Certificate Programme in Applied AI: Advanced Deep Learning Techniques by IISc Bangalore

Module 3: Foundations of Neural Networks

• Nonlinear models for classification and regression
• Multilayer perceptions
• Gradient descent on squared error loss, Backpropagation (BP), automatic differentiation
• Stochastic gradient descent and the mini-batch algorithm
• Optimization issues – BP with momentum, weight decay, ADAM algorithm, dropout
• Cross-entropy loss
• Issues of Generalization

Learning Outcomes:

The student will get the basic foundations of neural network models for classification and regression after this module. The back propagation procedure will be covered in detail and issues relevant to learning algorithms would be discussed.

Module 4: Deep Learning and Models

• Overview of Convolutional Neural Networks (CNNs)
• CNNs in computer vison and some standard models (e.g.,VGG net, Resnet)
• Simple Overview of recurrent neural networks (GRU and LSTM)
• Simple overview of Encoder-Decoder architectures, Transformers, Attention models
• Deep Generative models: Simple overview of RBM, VAE, GAN

Learning Outcomes:

The student will learn important models for deep learning and will get a good practical view of these models.

Module 5: Foundations of Reinforcement Learning

• Multi-armed bandits
• Markov decision processes (MDPs)
• Examples
• Numerical solution

Learning Outcomes:

Formulate a real-world problem in the RL framework and apply model-based algorithms.

Module 6: Model-Free Approaches under Full-State Information

• Monte-Carlo Algorithms for Prediction and Control
• On-policy and Off-policy Algorithms
• Temporal Difference Learning
• Q-learning/SARSA/Expected SARSA Algorithms
• Double Q-learning

Learning Outcomes:

Get a good handle on basic RL algorithms that can be applied in moderate state-action spaces.

Module 7: Model-Free Approaches with Parametric Optimisation

• Temporal Difference Learning with Function Approximation
• Linear and Non-linear Function Approximation Architectures
• Q-learning/SARSA/Double Q-Learning with Function Approximation
• Policy Gradient Methods
• REINFORCE Algorithm
• Actor-critic Algorithms

Learning Outcomes:

Understand and apply various function approximation architectures and get a good understanding of which algorithms will work better in what settings.

Module 8: Deep Reinforcement Learning Algorithms

• Deep Value and Policy-based Algorithms
• The Deep Q-network Algorithm (DQN)
• Trust Region Policy Optimisation (TRPO)
• Proximal Policy Optimisation (PPO)
• Asynchronous Advantage Actor Critic (A3C)
• Generalised Advantage Estimation Algorithm (GAE)

Learning Outcomes:

Get fully acquainted to almost all the recent Deep RL algorithms and will be able to apply these algorithms in real-world problem scenarios.

Assignments and Projects

• Assignment 1: Implementing linear regression and classification
• Assignment 2: Implementing MLP using backpropagation
• Assignment 3: Image classification using CNNs and implementing different CNN models.
• Assignment 4: Implementation of value/policy iteration algorithms on Grid World setting
• Assignment 5: Studying implementations of Monte-Carlo and temporal difference algorithms on Grid World.
• Assignment 6: Use stable baselines and hugging face together. Using models as black boxes, fine-tune these already available models and deploy them over hugging faces.
• Capstone Projects: Implementation of Deep learning and Deep RL algorithms on various gym/Atari environments and studying detailed performance comparisons. Hyper-parameter tuning and algorithm tweaking for enhanced performance.

Tools

• Python
• Pytorch