Question

What's the difference between Tensor and Variable in Tensorflow? I noticed in this stackoverflow answer, we can use Variable wherever Tensor can be used. However, I failed to do session.run() on a Variable:

A = tf.zeros([10])   # A is a Tensor
B = tf.Variable([111, 11, 11]) # B is a Variable
sess.run(A) # OK. Will return the values in A
sess.run(B) # Error.

Answer 1

Variable is basically a wrapper on Tensor that maintains state across multiple calls to run, and I think makes some things easier with saving and restoring graphs. A Variable needs to be initialized before you can run it. You provide an initial value when you define the Variable, but you have to call its initializer function in order to actually assign this value in your session and then use the Variable. A common way to do this is with tf.global_variables_initalizer().

For example:

import tensorflow as tf
test_var = tf.Variable([111, 11, 1])
sess = tf.Session()
sess.run(test_var)

# Error!

sess.run(tf.global_variables_initializer())  # initialize variables
sess.run(test_var)
# array([111, 11, 1], dtype=int32)

As for why you use Variables instead of Tensors, basically a Variable is a Tensor with additional capability and utility. You can specify a Variable as trainable (the default, actually), meaning that your optimizer will adjust it in an effort to minimize your cost function; you can specify where the Variable resides on a distributed system; you can easily save and restore Variables and graphs. Some more information on how to use Variables can be found here.