This course is also offered in other languages. For language in 中文, please refer to this webpage(dataapplab.com/ai) for details.
Modular 1 – Machine Learning
Class 1 Regression
- Basic concept of Regression
- Bias-Variance trade off
- Underfitting vs. Overfitting
- Linear regression analytical solution
Pros and cons of L1 and L2 regularization
- Advanced techniques in regression
Stochastic Gradient Descendent
Random sample consensus (RANSAC)
Class 2 Classification
- Evaluation Methods of classification
- Basic classification model: logistic regression, decision tree
- Classification Types (how binary and multi-class works)
- Ensemble model method:
Class 3 Dimension Reduction
- Dimension reduction overview
- Dimension reduction methods
Principal Component Analysis
- Manifold learning
- Multidimensional Scaling
Class 4 Clustering
- Unsupervised learning introduction
- Clustering methods & techniques
Hierarchical Clustering Algorithm
Outlier and anomaly detection
Modular 2 – Neural Network & Deep Learning
Neural network basic (which maybe duplicate with NLP part)
- Introduction to neural network, include some basic concept like neuron, weights, bias, activation function.
- Forward propagation for inference
- Training algorithm: backpropagation (use 1 hidden layer neural network with binary output as example)
Deep neural network
- Fully connected layer
- Shows how to use DNN for MNIST digit recognition problem
Convolutional neural network
- Motivation: why use CNN in computer vision problem: position invariance.
- Intro to convolutional layer + pooling layer
- Revisit MNIST problem and show how to use CNN to improve it. #params reduced.
Short recap to fully connected layer, convolutional layer and pooling layer.
Introduction to famous vision problems and corresponding networks
- Image classification: Alexnet
- Object detection: R-CNN
- Image segmentation: U-Net
Useful technicals for neural network training
- Performance: try different network structure, different number of layers and different number of hidden units in each layer
- Converge: sensitive to learning rate
- Speed up training: Stochastic Gradient Descent, Momentum
- Gradient vanishing problem: Batch Normalization
Recurrent Neural Network for video learning
- RNN basic (this maybe duplicate with NLP part)
- Use example to show how to use RNN for video analysis.
- Deep Q-learning
- Playing Atari game by DeepMind
(Brief introduction installation tensorflow)
Class 3 – Tensorflow and Facial Recognition
Brief introduction to Tensorflow: Tensor, operator concept.
Shows one small network structure and shows how to write it in Tensorflow.
Face recognition: given face images for 40 person, each have 10 images, use 9 images of each person for training. Target is to label the left 40 images (1 per person) to the right person.
Face recognition is widely used technologies, such as photo softwares, surveillance.
Finish the basic version for the Face recognition.
- Try different learning rate
- Add more layer
- Add more neurons for each layer
- Compare DNN and CNN
- Try different optimizer
Modular 3 – Deep Learning and Natural Language Processing
- Intro to NLP and Deep Learning
- what is NLP?
- NLP difficulty level
- Industry applications
- Deep neural network for NLP
- Phonology and Morphology
- Syntax and Semantics
- Question Answering
- Simple Word Vector representations: word2vec, GloVe
- Vector(discrete) Representation
- Problem with discrete representation
- Cooccurence Matrix
- Main idea of word2vec
- Main idea of Glove
- Complicated Models for NLP
- Recurrent Neural Networks
- Gated Recurrent Units
- Long-short-term-memories (LSTMs)
- TensorFlow for NLP
- A recap of tensorflow
- NLP specific tensorflow
- Build a tensorflow based chatbot from scratch