Over the past few years , research and innovation in the field of computer vision has been very promising. Ideas like face recognition, autonomous self driving cars, object recognition,speech recognition etc which was an imagination in the past has become a reality. Machines have reached a stage where they can actually see like a human. Thanks to the power of deep learning.
Deep learning is a branch of machine learning which involves developing algorithms to extract high level features also known as deep features and use these features to make decision similar to a human brain. Deep learning algorithm constitutes many connected neural layers consisting of connected neurons to extract features and recognize patterns from input data.
An artificial neural network consists of many connected neurons which works slightly like nerve connection in the human brain. Neurons are the constitutive units in the neural network.Deep Neural Networks (aka DNN) is a collection of multiple neural networks with definite layers to process information given an input. A typical DNN looks like figure 1 below.
Figure 1
At Aindra we are always experimenting to get inside this huge neural network, pull the features which the machine learns and a more practical understanding of DNN.
One of the ways of understanding DNN is to visualize each layer in the network. We opted to visualize the layers in a DNN by triggering the activation function in each layer. Each neuron in the network is defined with an activation function. An activation function defines the output given an input from the previous layer. By triggering the activation function, we get a tensor (n dimensional matrix array) which can be converted to an image to visualize.
To illustrate this method we conducted a small experiment where we developed a DNN and an image of digit (4) was given as input to recognize among 0 to 9. The following visuals show the outputs from each layer.
From the above experiment we would like to show certain valuable outcomes :-
-
The visuals clearly show that the features which the machine learns are very rich compared to any other traditional method of feature extraction. With deep learning, we need not have to specify the type of feature which should be extracted. Using the traditional method we have to choose a specific feature extraction technique like HOG etc which could be useful or it may not be useful. Using deep learning technique we only have to design the architecture of the network. The network itself picks up all features irrespective of any specification.
-
Extensive research and development is not required to build any particular detector like object detector or face detector. The layers in the neural network are so huge that at some node the machine recognizes the specific region of interest and creates a detector. In the above example the machine built a digit recognition engine. Visualization helps us identify the particular layer where the machine picked the region of interest.
On the whole visualization techniques have helped us understand deep neural networks better and not treat a DNN as a “blackbox”. Understanding neural networks will help us build more better models by extracting deep features and break more barriers.
References:
[1]Yosinski, Jason, et al. “Understanding neural networks through deep visualization.” arXiv preprint arXiv:1506.06579 (2015).
aindra systems 