Artificial Intelligence

March 3, 2018
6 weeks

This course is also offered in other languages.  For language in 中文, please refer to this webpage(  for details.

Modular 1 – Machine Learning

Class 1 Regression

  1. Basic concept of Regression
  2. Bias-Variance trade off
  3. Underfitting vs. Overfitting
  4. Linear regression analytical solution
  5. Regularization:




Pros and cons of L1 and L2 regularization

  1. Advanced techniques in regression

Gradient Descendent

Coordinated Descendent

Stochastic Gradient Descendent

Random sample consensus (RANSAC)


Class 2 Classification

  1. Evaluation Methods of classification
  2. Basic classification model: logistic regression, decision tree
  3. Classification Types (how binary and multi-class works)
  4. Ensemble model method:





Class 3 Dimension Reduction

  1. Dimension reduction overview
  2. Dimension reduction methods

Randomized Projection

Principal Component Analysis

PCA Calculation

Randomized PCA

Sparse PCA

  1. Manifold learning
  2. Multidimensional Scaling



Class 4 Clustering

  1. Unsupervised learning introduction
  2. Clustering methods & techniques

K-mean Algorithm

Hierarchical Clustering Algorithm

DBSCAN algorithm

Outlier and anomaly detection


Modular 2 – Neural Network & Deep Learning

Class 1

Neural network basic (which maybe duplicate with NLP part)

  • Introduction to neural network, include some basic concept like neuron, weights, bias, activation function.
  •  Forward propagation for inference
  •  Training algorithm: backpropagation (use 1 hidden layer neural network with binary output as example)

Deep neural network

  • Fully connected layer
  • Shows how to use DNN for MNIST digit recognition problem

Convolutional neural network

  • Motivation: why use CNN in computer vision problem: position invariance.
  • Convolution
  • Intro to convolutional layer + pooling layer
  •  Revisit MNIST problem and show how to use CNN to improve it. #params reduced.

Class 2

Short recap to fully connected layer, convolutional layer and pooling layer.

Introduction to famous vision problems and corresponding networks

  •      Image classification: Alexnet
  •      Object detection: R-CNN
  •      Image segmentation: U-Net

Useful technicals for neural network training

  •      Performance: try different network structure, different number of layers and different number of hidden units in each layer
  •      Converge:  sensitive to learning rate
  •      Speed up training: Stochastic Gradient Descent, Momentum
  •      Gradient vanishing problem: Batch Normalization

Recurrent Neural Network for video learning

  •      RNN basic (this maybe duplicate with NLP part)
  •      Use example to show how to use RNN for video analysis.

Reinforcement learning

  •      Deep Q-learning
  •      Playing Atari game by DeepMind

(Brief introduction installation tensorflow)

Class 3 – Tensorflow and Facial Recognition

Brief introduction to Tensorflow: Tensor, operator concept.

Shows one small network structure and shows how to write it in Tensorflow.

Lab problem:

Face recognition: given face images for 40 person, each have 10 images, use 9 images of each person for training. Target is to label the left 40 images (1 per person) to the right person.

Face recognition is widely used technologies, such as photo softwares, surveillance.

Class 4

Finish the basic version for the Face recognition.

Improve network:

  •      Try different learning rate
  •      Add more layer
  •      Add more neurons for each layer
  •      Compare DNN and CNN
  •      Try different optimizer


Modular 3 – Deep Learning and Natural Language Processing

Class 1

  • Intro to NLP and Deep Learning
  • what is NLP?
  • NLP difficulty level
  • Industry applications
  • Deep neural network for NLP
  • Phonology and Morphology
  • Syntax and Semantics
  • Question Answering

Class 2

  • Simple Word Vector representations: word2vec, GloVe
  • Vector(discrete) Representation
  • Problem with discrete representation
  • Cooccurence Matrix
  • Main idea of word2vec
  • Main idea of Glove

Class 3

  • Complicated Models for NLP
  • Recurrent Neural Networks
  • Alignment
  • Gated Recurrent Units
  • Long-short-term-memories (LSTMs)

Class 4

  • TensorFlow for NLP
  • A recap of tensorflow
  • NLP specific tensorflow
  • Build a tensorflow based chatbot from scratch