Readings

Weekly Readings (Subject to Change)

01/23/2019 (Introduction, Backprop, Shallow Vs Deep, Policies)
Required
  • Reducing the dimensionality of data with neural networks by Hinton et. al. (2006) Here
  • How to read a technical paper by Jason Eisner (2009) [HTML]
  • Chapter 6 of Deep Learning Book by Goodfellow Here
Optional
  • Chapters 1-5 of Deep Learning Book by Goodfellow Here
01/30/2019 (Convolutional Neural Networks, Vision (ImageNet) )
Required
  • Chapter 9 (Convolutional Neural Networks) of Deep Learning Book by Goodfellow Here
  • AlexNet
    • (Presenter: )
Paper Critique and Summary before Class I (Read 1 out of the 3)
  1. VGGNet Here
  2. GoogleNet Here
  3. ResNet Here
Optional
02/06/2019 (Advanced Training)
Required
  • Dropout: a simple way to prevent neural networks from overfitting by Srivastava et. al. Here
  • Chapters 7 and 8 of Deep Learning Book by Goodfellow Here
  • Net2net: Accelerating learning via knowledge transfer." arXiv preprint arXiv:1511.05641 (2015) by Chen et. al. Here (Presenter: )
  • Adam: A method for stochastic optimization by Diederik et. al. (2014) Here (Presenter: )
Optional
  • Sec 2.2 of DL Papers Reading Roadmap: Here
  • Sec 2.1 of DL Papers Reading Roadmap: Here
02/13/2019 (Unsupervised Learning/Deep Generative Models)
Required
  • Generative Adverserial Networks by Goodfellow (2014) Here (Presenter: )
  • UNSUPERVISED REPRESENTATION LEARNING WITH DEEP CONVOLUTIONAL GENERATIVE ADVERSARIAL NETWORKS by Radford et. al. (2015) (Presenter: ) Here
Optional
  • Chapter 14 (Autoencoders) of deep learning book Here
  • Papers on Unsupervised/Generative learning Here
02/20/2019 (Sequence Modeling)
Required
  • Chapter 10 of Deep Learning Book on Sequence Modeling Here
Paper Critique and Summary before Class II (Read 1 out of the 4)
  1. Generating sequences with recurrent neural networks." arXiv preprint arXiv:1308.0850 (2013). by Graves Here
  2. Sequence to sequence learning with neural networks." Advances in neural information processing systems. 2014 by Sutskever (2014) Here
  3. Visualizing and Understanding Recurrent Neural Networks by Karpathy et. al. ICLR (2015) Here
  4. PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs (NIPS 2017) by Wang et. al. Here
  5. Gated Feedback Recurrent Neural Networks by Chung et. al. (2015) Here
Optional
  • Papers on Sequence Modeling Here
02/27/2019 (Applications: NLP )
Required
  • Distributed Representations of Words and Phrases and their Compositionality by Mikolov et. al. (2013) Here
Paper Critique and Summary before Class III (Read 1 out of the 4)
  1. Attention is All you Need by Vaswani et. al. (2017) Here
  2. Ask Me Anything:Dynamic Memory Networks for Natural Language Processing by Kumar et. al. (2016) Here
  3. Global Vectors for Word Representation by Pennington et. al. Here
  4. Adversarial Multi-task Learning for Text Classification by Liu et. al. (2017)
  5. An Empirical Exploration of Recurrent Network Architectures by Jozefowicz et. al. 2015 Here
Optional
  • Efficient Estimation of Word Representations in Vector Space Here
  • See class by Richard Socher Here
03/06/2019 (Applications: Recommender Systems)
Required
  • Deep Learning based Recommender System: A Survey and New Perspectives by Zhang et. al. (2018) Here (Presenter: )
  • Neural Collaborative Filtering by He et. al. (2017) Here (Presenter: )
Optional
  • Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention by Chen et. al. 2017 Here
03/20/2019 (Deep Learning in X)
Required
  • Opportunities and obstacles for deep learning in biology and medicine by Ching et. al. (2018) Here (Presenter: )
  • A Neural Algorithm of Artistic Style by Gatys et. al. (2015) Here (Presenter: )
  • Deep Learning in Medical Image Analysis by Shen et. al. (2017) Here (Presenter: )
Optional
03/27/2019 (Deep Reinforcement Learning)
Required
  • Human-level control through deep reinforcement learning by Mnih et. al. (2015) Here
  • Dueling network architectures for deep reinforcement learning by Ziyu et. al. (2015) Here
  • Mastering the game of Go with deep neural networks and tree search." Nature 529.7587 (2016): 484-489 Here
Optional
04/03/2019 (Transfer/One-Shot DL)
Required
  • Progressive Neural Networks by Rusu et. al. (2016) Here (Presenter: )
  • Show, Attend and Tell: Neural Image Caption Generation with Visual Attention (2016) by Xu. et. al. Here (Presenter: )
  • How Transferable are features in deep neural networks Here
Optional
04/10/2019 (Resource-Aware Deep Learning)
Required
  • Deep Learning for IoT Big Data and Streaming Analytics: A Survey by Mohammadi et. al. Here
Paper Critique and Summary before Class IV (Read 1 out of the 4)
  1. Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding by Song et. al. (2015) Here
  2. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and less than 1MB model size. arXiv preprint arXiv:1602.07360 by Forrest et. al. (2016). Here
  3. Deep Learning for the Internet of Things by Yao et. al. (2018) Here
  4. Binarized Neural Networks: Training Neural Networks with Weights and Activations Constrained to+ 1 or−1 by Matthieu et. al. Here
Optional
04/17/2019 (Free/Work on Projects)
04/24/2019 (Project Presentations I)
05/01/2019 (Project Presentations II)