How to Get Started Using Deep Neural Nets

How to Get Started Using Deep Neural Nets

How to Get Started Using Deep Neural Nets

By July 22, 2015 Booz Allen, Data Science No Comments

How to Get Started Using Deep Neural Nets

One of the questions I often get from people after giving a talk is “How do I get started with deep neural nets?” Building new deep neural nets is a three-part process: learn the theoretical concepts of the model, play with toy models, and put it into practice by building your own models.

Learn

There is a lot of jargon surrounding deep neural nets: activation maps, convolution, pooling, etc. Luckily, some of the experts in the field are also excellent teachers. There is a beautiful overview of the convolutional neural nets and how their architecture fits together at Andrej Karpathy’s Stanford cs231n class. Not only will this provide you with the background information you need to build your own nets, it has animated illustrations of tricky key concepts such as convolutional layers.

Once you’ve mastered the basics, it’s important to read some of the key papers in the field to better understand how the models and concepts fit together. Yann Lecun’s LeNet–5, the convolutional neural net that successfully classified MNIST digits, is a great place to start. Another is Alex Krizhevzy’s Alexnet, the model that successfully won the 2012 ImageNet challenge. After those seminal papers, the next two papers that are incredibly valuable are the 2014 GoogleNet and VGG models. The GoogleNet model is one of the most recent ImageNet classification challenges and is readily available for use in the Caffe deep learning framework. The VGG model in particular is notable for both its flexibility for retraining and its winning score on the combination localization+classification task. Lastly, there’s a recent review article in Nature that provides an overview of recent developments in the field.

Play

By starting to play with Javascript convolutional nets examples, you will see how the concepts operate in real time with real examples that you can easily modify. Andrej Karpathy has created a collection of Javascript examples of convolutional neural nets including: classification, auto encoder, and reinforcement learning examples (including visualizations of the learning and layers of the neural nets).

Practice

DIGITS and Caffe are my first recommendation because DIGITS provides an easy-to-use interface to Caffe’s model building capabilities. DIGITS will allow you to get started quickly in building your own models while the underlying Caffe framework will allow you to go deeper once you master the basics. In addition, there are docker images that will allow you to quickly start playing with the framework and a model zoo featuring many of the most recent ImageNet top models.

—Written by Aaron Sander