TensorFlow Basics - Second Part
Placeholder Variables
We already talked about Variables and Constants in the first part of TensorFlow Basics. In this part we extend the topic by introducing TensorFlow placeholders. The placeholders give us the chance to define the computation graph without specifying congrete input values. It is sufficient to declare the Tensortype of the data that we want to work with. As expected the placeholder variable can the be used for any further computation such as in a multiply operation as shown in the following script:
import tensorflow as tf input1 = tf.placeholder(tf.float32) input2 = tf.placeholder(tf.float32) output = tf.multiply(input1, input2) with tf.Session() as sess: out = sess.run([output], feed_dict={input1:[2.0], input2:[3.0]}) print(out) #=> [array([ 6.], dtype=float32)]
As you can see, using placeholder variables is intuitively easy. The one thing that is important to mention is that any placeholder need to replaced by a real value when it is processed in a Sesson.
To do that we simply use the feed concept, that we have discussed in the first part of the tutorial. In sess.run() we pass two different arguments. First we pass the variable whose output we want to fetch from the Session run (in this case we want to fetch the result of the placeholder computation -> output)
Variable Scope
As it is always in programming, small side-projects can grow to have hundreds and thousands of lines of code very quickly. And as in many other programming languages TensorFlow provides a feature called the Variable Scope mechanism to group all the variables in namespaces to avoid clashes.
The Tensorflow Varibale Scope feature consists of two main functions:
tf.get_variable(, , )
tf.variable_scope()
In a very complex Deep Learning Model you may have different variable scopes, one variable scope for the convolutional part, one for the fully connected part and maybe even a third or fourth for a recurrent part of your network architecture. The easy to use TensorFlow variable scope command helps you do organize your model structure in logical parts using the same names for similar concepts i.e W for a weight variable and b for biases without mixing different parts up.
- tf.get_variable(, , )
tf.get_variable(, , ) will create a varible with the specified name if such a variable does not already exist and it will access that variable if it finds it to exist. Accessing an existing varibale requires an
- tf.variable_scope()
tf.variable_scope() works closely with the tf.get_variable. It lets you define the namespaces mentioned above. In the next example you can see that in Tensorflow you cannot define two varibales of the same name in one namespace:
import tensorflow as tf v = tf.get_variable("var", [1]) v = tf.get_variable("var", [1]) #ERROR! #=> ValueError: Variable var already exists, disallowed.
Recall that Python namespaces are not TensorFlow namespaces! Changing the Python variable name does not solve the problem. In the next example there stil is a name clash in the Tensorflow computation graph:
import tensorflow as tf v = tf.get_variable("var", [1]) w = tf.get_variable("var", [1]) #Still an ERROR! #=> ValueError: Variable var already exists, disallowed.
Defining a namespace that encapsules one of the varibales is they key to solve this issue:
import tensorflow as tf v = tf.get_variable("var", [1]) print(v.name) #=> var:0 with tf.variable_scope("blah"): v= tf.get_variable("var", [1]) print(v.name) #=> blah/var:0
You can see that in this example there is no error. The reseon for this is that the second "var" variable now lives in the namespace "blah" which makes it adressable under the complete name namespace/name which in this case is blah/var.
Asserts
Tensorflow provides a simple Assert functionality to perform quick checks on varibales and outputs to make sure that no incorrect value is further processed. An Assert can be added anywhere in one single line of code:
import tensorflow as tf assert 1 == 1 # WORKS assert 2 == 1 # ERROR! # => AssertionError
assert tf.get_variable_scope().reuse == False # => AssertionError














