tensorflow - Google Cloud TPU -- no TPU being used

Question

I am trying to run a simple program on TPU:

import tensorflow as tf

tpu = tf.distribute.cluster_resolver.TPUClusterResolver()
print("Device:", tpu.master())
tf.config.experimental_connect_to_cluster(tpu)
tf.tpu.experimental.initialize_tpu_system(tpu)
strategy = tf.distribute.experimental.TPUStrategy(tpu)

a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])

with strategy.scope():
    c = tf.matmul(a, b)
    print("c device: ", c.device)
with tf.Session(config=tf.ConfigProto(log_device_placement=True)) as sess:
    print(c.eval())

When I run this, it looks like the TPU is being found. However, none of the logged devices have 'TPU' in the name -- it is all on the CPU.

What am I doing wrong?

Answer 1

strategy.scope() is for model training.

If you want to run tf.matmul on a TPU you could use either this:

with tf.device('/TPU:0'):
  c = tf.matmul(a, b)

Or

@tf.function
def matmul_fn(x, y):
  z = tf.matmul(x, y)
  return z

z = strategy.run(matmul_fn, args=(a, b))
print(z)

Details are here.