The 10 Most Important TensorFlow Classes
The TensorFlow API is immense, comprising hundreds of packages and thousands of modules. Given its size, newcomers may find it hard to know which classes to study closely. To remedy this confusion, here are TensorFlow’s ten most important classes with explanations about what the class accomplishes and why it’s so important.
Tensors play a central role in TensorFlow development and serve as the primary objects for storing and manipulating data. Optimizers only accept data contained in tensors, and image-processing functions require images to be provided as tensors. All neural network layers, from dense layers to dropout layers, accept tensors as input and return tensors as output.
A tensor serves as an N-dimensional array, where N can be zero or more. A tensor’s number of dimensions is called the tensor’s rank, and the size of each dimension is called the tensor’s shape. For example, a 3-x-5 matrix has shape [3, 5], and an RGB image whose size is 200 x 200 would be represented by a tensor with size [200, 200, 3].
TensorFlow provides hundreds of functions for creating, transforming, and processing tensors. You can create a tensor with constant values by calling
tf.constant or create a tensor with random values by calling
tf.random_uniform. You can reshape a tensor with
tf.reshape and extract part of a tensor with
When the Python interpreter reaches a function that operates on tensors, it doesn’t execute the operation immediately. Instead, it creates an instance of the
Operation class that represents the operation. Every
Operation has a property called
inputs that contains its input tensors and a property called
outputs that contains its output tensors.
Operation has a property called
type that is usually set to the function that created it. For example, if you call
tf.add, the corresponding operation will have its
type set to
Other math operations include
tf.round, and t
f.sqrt. TensorFlow also supports traditional matrix operations, including
TensorFlow creates a
Tensor instance for each tensor in your application and an
Operation for each operation involving tensors. It stores these
Operations in a data structure called a
Graph. Only one
Graph can be active at a time, and you can make a new Graph active by calling
Graph class provides a number of methods for accessing the data contained in the graph. You can access a particular tensor with
get_tensor_by_name or access all of the graph’s operations by calling
Graph stores data in a series of containers called collections. Every collection can be accessed through a particular key, and
get_all_collection_keys provides the full list of keys. For example, a graph stores its global variables in the collection whose key is
After you add tensors and operations to a graph, you can execute the graph’s operations by creating and running a session. You can create a session by calling
tf.Session and then launch the session by calling its run method.
The first argument of the run method tells the session what processing to perform. If this argument contains tensors, the session will compute the elements of each tensor and return the elements in a NumPy array. If this argument contains Operations, the session will perform each operation and return the appropriate result.
If the questions on
StackOverflow are any indication,
run's feed_dict confuses many developers. This parameter accepts a dictionary that associates values with tensors (usually placeholders) in the graph. But the dictionary’s values can’t be tensors. For this reason, it’s generally a good idea to store and process input data using NumPy arrays before executing a session.
Variables resemble tensors in many respects. They store values in N-dimensional arrays and can be operated upon using regular TensorFlow operations. But during training operations, applications rely on variables to store the state of the model. For example, if an application consists of a neural network, the network’s weights and biases will be stored as variables.
Another difference is that variables require a different set of methods than tensors. For example, after you create a
Variable, you need to initialize its value by running a special operation in the session. If your application has many variables, you can obtain a combined initialization operation by calling
At a low level, the goal of training is to set the application’s variables to values that will bring the model in line with observed data. These variables are critically important, so it’s a good idea to store them to checkpoint files with Savers.
The disparity between an application’s model and the observed data is called loss. A TensorFlow application reduces loss using an optimizer. In code, you can create an optimizer by instantiating a subclass of the
Optimizer class. Every optimizer has a
minimize method that returns an operation that can be executed in a session.
TensorFlow supports a number of different optimization algorithms, and each is represented by a different subclass of
Optimizer. As an example, the simplest optimization algorithm, the gradient descent method, is represented by the
GradientDescentOptimizer. But the simplest algorithm is rarely the most effective, and I recommend optimizing your applications with the
Estimators dramatically simplify the process of developing and deploying machine learning algorithms. When you use an estimator, you don’t have to worry about sessions and graphs. You simply need to know three methods of the
Another advantage of using estimators is that TensorFlow provides many subclasses of Estimator. These canned estimators, such as
DNNClassifier, make it easy to train and test machine learning. The
DNNLinearCombinedClassifier is particularly helpful because it lets you take advantage of wide and deep learning.
One of the most recent changes to the TensorFlow API is the promotion of the
tf.contrib.data package to
tf.data. This package provides the all-important
Dataset class, which TensorFlow recommends for loading and processing data. This class provides many powerful methods for batching and transforming data, and in many cases, you can perform these operations in a multithreaded manner.
Dataset class is also important because it’s the superclass of
TFRecordDataset. These two classes make it straightforward to read data from text files and TFRecord files.
Dataset class provides many powerful capabilities, but it doesn’t let you access its data directly. To extract tensors from a dataset, you need to create an instance of the
TensorFlow provides four different ways to iterate through a dataset’s content. The simplest is the one-shot iterator, which can iterate through a dataset only once. You can reuse initializable and reinitializable iterators, but you’ll need to run special initialization operations first. Feedable iterators are the most complicated, but you can associate them with multiple datasets and you don’t need to initialize them before each iteration.
The goal of training is to determine which variables produce the least possible loss. Training can take hours or days, so it’s crucial to store the variable’s values during and after training. TensorFlow makes this possible by providing the
Using this class is easy. After you create a
Saver instance, you can call save to store the model’s state in numbered checkpoint files. You can load the model’s variables from the checkpoint files by calling the