Working with multiple graphs in TensorFlow

Question

Can someone explain to me how name_scope works in TensorFlow?

Suppose I have the following code:

import tensorflow as tf

g1 = tf.Graph()
with g1.as_default() as g:
    with g.name_scope( "g1" ) as scope:
        matrix1 = tf.constant([[3., 3.]])
        matrix2 = tf.constant([[2.],[2.]])
        product = tf.matmul(matrix1, matrix2)

tf.reset_default_graph()

g2 = tf.Graph()
with g2.as_default() as g:
    with g.name_scope( "g2" ) as scope:
        matrix1 = tf.constant([[4., 4.]])
        matrix2 = tf.constant([[5.],[5.]])
        product = tf.matmul(matrix1, matrix2)

tf.reset_default_graph()

with tf.Session( graph = g1 ) as sess:
    result = sess.run( product )
    print( result )

When I run this code I get the following error message:

Tensor Tensor("g2/MatMul:0", shape=(1, 1), dtype=float32) is not an element of this graph.

I agree "g2/MatMul" is not an element of graph g1, but why is it selecting "g2/MatMul" when the session graph is set to g1? Why doesn't it select "g1/MatMul"?

---

Edit

The following code seems to work:

import tensorflow as tf

g1 = tf.Graph()
with g1.as_default() as g:
    with g.name_scope( "g1" ) as g1_scope:
        matrix1 = tf.constant([[3., 3.]])
        matrix2 = tf.constant([[2.],[2.]])
        product = tf.matmul( matrix1, matrix2, name = "product")

tf.reset_default_graph()

g2 = tf.Graph()
with g2.as_default() as g:
    with g.name_scope( "g2" ) as g2_scope:
        matrix1 = tf.constant([[4., 4.]])
        matrix2 = tf.constant([[5.],[5.]])
        product = tf.matmul( matrix1, matrix2, name = "product" )

tf.reset_default_graph()

use_g1 = False

if ( use_g1 ):
    g = g1
    scope = g1_scope
else:
    g = g2
    scope = g2_scope

with tf.Session( graph = g ) as sess:
    tf.initialize_all_variables()
    result = sess.run( sess.graph.get_tensor_by_name( scope + "product:0" ) )
    print( result )

By flipping the switch use_g1, graph g1 or g2 will run in the session. Is this the way name scoping was meant to work?

Answer 1

Your product is a global variable, and you've set it to point to "g2/MatMul".

In particular

Try

print product

and you'll see

Tensor("g2/MatMul:0", shape=(1, 1), dtype=float32)

So the system takes "g2/MatMul:0" since that's the Tensor's name, and tries to find it in the graph g1 since that's the graph you set for the session. Incidentally you can see all nodes in the graph print [n.name for n in g1.as_graph_def().node]

Generally, using more than one graph is rarely useful. You can't merge them and can't pass tensors between them. I'd recommend just doing

tf.reset_default_graph()
a = tf.Constant(2)
sess = tf.InteractiveSession()
....

This way you'll have one default graph and one default session and you can omit specifying graph or session in most cases. If you ever need to refer to them explicitly, you can get them from tf.get_default_graph() or tf.get_default_session()