python - How to run Keras on multiple cores?

Question

I'm using Keras with Tensorflow backend on a cluster (creating neural networks). How can I run it in a multi-threaded way on the cluster (on several cores) or is this done automatically by Keras? For example in Java one can create several threads, each thread running on a core.

If possible, how many cores should be used?

Answer 1

Tensorflow automatically runs the computations on as many cores as are available on a single machine.

If you have a distributed cluster, be sure you follow the instructions at https://www.tensorflow.org/how_tos/distributed/ to configure the cluster. (e.g. create the tf.ClusterSpec correctly, etc.)

To help debug, you can use the log_device_placement configuration options on the session to have Tensorflow print out where the computations are actually placed. (Note: this works for both GPUs as well as distributed Tensorflow.)

# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

Note that while Tensorflow's computation placement algorithm works fine for small computational graphs, you might be able to get better performance on large computational graphs by manually placing the computations in specific devices. (e.g. using with tf.device(...): blocks.)