CS551: Introduction to Deep Learning

This course will provide basic understanding of deep learning and how to solve problems from varied domains. Open source tools will be used to demonstrate different applications.

• Monday - 1000-1100
• Tuesday - 1000-1100
• Wednesday - 1000-1100

Brief introduction of big data problem. Overview of linear algebra, probability, numerical computation. Basics of Machine learning/Feature engineering. Neural network. Tutorial for Tools. Deep learning network - Shallow vs Deep network, Deep feedforward network, Gradient based learning - Cost function, soft max, sigmoid function, Hidden unit - ReLU, Logistic sigmoid, hyperbolic tangent Architecture design, SGD, Unsupervised learning - Deep Belief Network, Deep Boltzmann Machine, Factor analysis, Autoencoders. Regularization. Optimization for training deep model. Advanced topics - Convolutional Neural Network, Recurrent Neural Network/ Sequence modeling, LSTM, Reinforcement learning. Practical applications – Vision, speech, NLP, etc.

• Ian Goodfellow, Yoshua Bengio and Aaron Courville, “Deep Learning”, Book in preparation for MIT Press, 2016. (available online)
• Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie, “The elements of statistical learning”, Springer Series in Statistics, 2009.

• Introduction to Deep Learning
• Overview of Linear Algebra & Probability
• Feature Engineering
• Neural Network
• Demonstration of Keras, Installation guide for Keras - Presented by Niraj
• DNN
• Regularization
• Optimization
• CNN
• RNN
• Deep Reinforcement Learning

• Matrix differentiation by Randal J. Barnes, Department of Civil Engineering, University of Minnesota, Minneapolis, Minnesota, USA
• A geometric interpretation of the covariance matrix
• Deep Learning, Nature
• Artificial Neural Networks: A tutorial
• The neural network zoo
• A Step by Step Backpropagation Example
• Backpropagation in convolutional neural network
• Andrej Karpathy blog
• Blog on LSTM-RNN
• Ill-Conditioning in Neural Networks

• Some project ideas and data resources
• General information for project

• Previous year mid-sem