======CS551: Introduction to Deep Learning======
This course will provide basic understanding of deep learning and how to solve problems from varied domains. Open source tools will be used to demonstrate different applications.

=====Class timings=====
  * Monday - 1000-1100
  * Tuesday - 1000-1100
  * Wednesday  - 1000-1100

=====Syllabus=====
Brief introduction of big data problem. Overview of linear algebra, probability, numerical computation. Basics of Machine learning/Feature engineering. Neural network. Tutorial for Tools. Deep learning network - Shallow vs Deep network, Deep feedforward network, Gradient based learning - Cost function, soft max, sigmoid function, Hidden unit - ReLU, Logistic sigmoid, hyperbolic tangent Architecture design, SGD, Unsupervised learning - Deep Belief Network, Deep Boltzmann Machine, Factor analysis, Autoencoders. Regularization. Optimization for training deep model. Advanced topics - Convolutional Neural Network, Recurrent Neural Network/ Sequence modeling, LSTM, Reinforcement learning. Practical applications  – Vision, speech, NLP, etc. 

=====Books=====
  * Ian Goodfellow, Yoshua Bengio and Aaron Courville, “Deep Learning”, Book in preparation for MIT Press, 2016. (available online) 
  * Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie, “The elements of statistical learning”, Springer Series in Statistics, 2009.

=====Slides=====
  * {{ :courses:2018:cs551:01_intro_to_dl.pdf |Introduction to Deep Learning}}
  * {{ :courses:2018:cs551:02_linear_algebra.pdf | Overview of Linear Algebra & Probability}}
  * {{ :courses:2018:cs551:03_feature_engg.pdf | Feature Engineering}}
  * {{ :courses:2018:cs551:04_neural_network.pdf | Neural Network}}
  * {{ :courses:2018:cs551:keras.pdf |Demonstration of Keras}}, {{ :courses:2018:cs551:installationguidekerastensorflow.txt |Installation guide for Keras}} - Presented by Niraj
  * {{ :courses:2018:cs551:05_dl_network.pdf |DNN}}
  * {{ :courses:2018:cs551:06_regularization.pdf |Regularization}}
  * {{ :courses:2018:cs551:07_optimization.pdf |Optimization}}
  * {{ :courses:2018:cs551:08_cnn.pdf |CNN}}
  * {{ :courses:2018:cs551:09_rnn.pdf |RNN}}
  * {{ :courses:2018:cs551:11_drl.pdf |Deep Reinforcement Learning}}


  

=====Additional Reading Materials=====
  * {{:courses:2017:matrixcalculus.pdf|Matrix differentiation}} by Randal J. Barnes, Department of Civil Engineering, University of Minnesota, Minneapolis, Minnesota, USA
  * [[http://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix|A geometric interpretation of the covariance matrix]]
  * [[http://www.nature.com/nature/journal/v521/n7553/full/nature14539.html|Deep Learning, Nature]]
  * [[http://ieeexplore.ieee.org/abstract/document/485891/ | Artificial Neural Networks: A tutorial]]
  * [[http://www.asimovinstitute.org/neural-network-zoo/|The neural network zoo]]
  * [[https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ | A Step by Step Backpropagation Example]]
  * [[http://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/|Backpropagation in convolutional neural network]]
  * [[http://karpathy.github.io/2015/05/21/rnn-effectiveness/ | Andrej Karpathy blog]]
  * [[https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/ | Blog on LSTM-RNN]]
  * [[ftp://ftp.sas.com/pub/neural/illcond/illcond.html| Ill-Conditioning in Neural Networks]]


=====Information related to projects=====
  * {{ :courses:2018:cs551:projectlist.pdf | Some project ideas and data resources}}
  * [[ :courses:2018:cs551:info_for_project| General information for project]]

=====Other information=====
  * {{ :courses:2018:cs551:2017_cs514_midsem.pdf |Previous year mid-sem}}