CS551: Introduction to Deep Learning

This course will provide a basic understanding of deep learning and how to solve problems from varied domains. Open source tools will be used to demonstrate different applications.

• Monday - 0800-0900
• Tuesday - 0800-0900
• Wednesday - 1700-1800

Brief introduction of big data problem. Overview of linear algebra, probability, numerical computation. Basics of Machine learning/Feature engineering. Neural network. Tutorial for Tools. Deep learning network - Shallow vs Deep network, Deep feedforward network, Gradient based learning - Cost function, soft max, sigmoid function, Hidden unit - ReLU, Logistic sigmoid, hyperbolic tangent Architecture design, SGD, Unsupervised learning - Deep Belief Network, Deep Boltzmann Machine, Factor analysis, Autoencoders. Regularization. Optimization for training deep model. Advanced topics - Convolutional Neural Network, Recurrent Neural Network/ Sequence modeling, LSTM, Reinforcement learning. Practical applications – Vision, speech, NLP, etc.

• Ian Goodfellow, Yoshua Bengio and Aaron Courville, “Deep Learning”, Book in preparation for MIT Press, 2016. (available online)
• Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie, “The elements of statistical learning”, Springer Series in Statistics, 2009.
• Charu C Aggarwal, “Neural Networks and Deep Learning”, Springer.

• Introduction
• Linear Algebra and Probability
• Feature engineering
• Neural network
• Introdution to Keras - Presented by Niraj
• DNN
• Regularization
• Optimization
• CNN
• RNN
• Practical methods
• Reinforcement learning

• Matrix differentiation by Randal J. Barnes, Department of Civil Engineering, University of Minnesota, Minneapolis, Minnesota, USA
• A geometric interpretation of the covariance matrix
• Deep Learning, Nature
• Artificial Neural Networks: A tutorial
• The neural network zoo
• A Step by Step Backpropagation Example
• Backpropagation in convolutional neural network
• Andrej Karpathy blog
• Blog on LSTM-RNN
• Ill-Conditioning in Neural Networks

• General information

• Previous year midsem paper