One of the questions I often get from people after giving a talk is “How do I get started with deep neural nets?” Building new deep neural nets is a three-part process: learn the theoretical concepts of the model, play with toy models, and put it into practice by building your own models.
There is a lot of jargon surrounding deep neural nets: activation maps, convolution, pooling, etc. Luckily, some of the experts in the field are also excellent teachers. There is a beautiful overview of the convolutional neural nets and how their architecture fits together at Andrej Karpathy’s Stanford cs231n class. Not only will this provide you with the background information you need to build your own nets, it has animated illustrations of tricky key concepts such as convolutional layers.
Once you’ve mastered the basics, it’s important to read some of the key papers in the field to better understand how the models and concepts fit together. Yann Lecun’s LeNet–5, the convolutional neural net that successfully classified MNIST digits, is a great place to start. Another is Alex Krizhevzy’s Alexnet, the model that successfully won the 2012 ImageNet challenge. After those seminal papers, the next two papers that are incredibly valuable are the 2014 GoogleNet and VGG models. The GoogleNet model is one of the most recent ImageNet classification challenges and is readily available for use in the Caffe deep learning framework. The VGG model in particular is notable for both its flexibility for retraining and its winning score on the combination localization+classification task. Lastly, there’s a recent review article in Nature that provides an overview of recent developments in the field.
DIGITS and Caffe are my first recommendation because DIGITS provides an easy-to-use interface to Caffe’s model building capabilities. DIGITS will allow you to get started quickly in building your own models while the underlying Caffe framework will allow you to go deeper once you master the basics. In addition, there are docker images that will allow you to quickly start playing with the framework and a model zoo featuring many of the most recent ImageNet top models.
—Written by Aaron Sander