DIY Deep Learning for Vision:
a Hands-On Tutorial with Caffe

This tutorial is designed to equip researchers and developers with the tools and know-how needed to incorporate deep learning into their work. Both the ideas and implementation of state-of-the-art deep learning models will be presented. While deep learning and deep features have recently achieved strong results in many tasks, a common framework and shared models are needed to advance further research and applications and reduce the barrier to entry.

To this end we present the Caffe framework that offers an open-source library, public reference models, and working examples for deep learning. Join our tour from the 1989 LeNet for digit recognition to today's top ILSVRC14 vision models and beyond to detection, vision + language, and segmentation models. Follow along with do-it-yourself code notebooks. While focusing on vision, general techniques are covered.

Schedule

This afternoon tutorial is held Sunday, June 7 at 2pm — 6pm in room 200. There will a break for open discussion and coffee at 3:30 – 4:15pm.

Cloud instances with Caffe were made available during the tutorial for attendees to follow along thanks to the NVIDIA qwiklabs infrastructure. No need to pre-register, just a machine with a web browser.

Caffe Tour Follow along with the tutorial slides!

• Introduction: deep learning and the need for frameworks
• Philosophy: expressive, fast, modular, and open
• Model Anatomy: nets, layers, blobs
• Forward / Backward computation
• Data
• Loss
• Solver / model optimization
• Interfaces for command line, Python, and MATLAB

Caffeine

• coffee!
• examples / demos
• discussion

The Latest Roast: new models and new code

• Detection with Fast R-CNN
• Sequences with LSTMs and Vision + Language with LRCN
• Pixelwise prediction with FCNs
• Framework design and future

Audience

Beginners: you will be equipped with the tools and know-how to begin or improve your deep learning toolkit through talks, worked examples, and demos.

Intermediate: follow along for a walkthrough of model definition and development, framework extension, and tips and tricks in practice will be covered.

Advanced: join for discussion of modeling and optimization as well as news of the latest developments brewing in Caffe.

Materials

Everything will be bundled in the next Caffe release.

Organizers

• Evan Shelhamer @shelhamer. PhD student, UC Berkeley CS.
• Jeff Donahue @jeffdonahue. PhD student, UC Berkeley CS.
• Jon Long @longjon. PhD student, UC Berkeley CS.
• Yangqing Jia @Yangqing. Research Scientist, Google.
• Ross Girshick @rbgirshick. Researcher, Microsoft Research.

Acknowledgements

The tutorial organizers would like to thank

• the Caffe team, the Berkeley Vision and Learning Center, and BVLC PI Trevor Darrell
• our open-source contributors and the Caffe community as a whole
• NVIDIA and the NVIDIA Academic program for collaboration and GPU donation
• A9 and Amazon Web Services for a research grant in support of Caffe development and reproducible research

for all the opportunities we have had in brewing deep nets in Caffe.