In [1]:
# %load /Users/facai/Study/book_notes/preconfig.py
%matplotlib inline

#import matplotlib.pyplot as plt
#import seaborn as sns
#sns.set(color_codes=True)
#sns.set(font='SimHei', font_scale=2.5)
#plt.rcParams['axes.grid'] = False

#import numpy as np

#import pandas as pd
#pd.options.display.max_rows = 20

#import sklearn

#import itertools

Chapter 12 Application

12.1 Large Scale Deep Learning

  • Fast CPU Implementations
    • specializated numerical computation routines
    • optimizing data struture (avoid cache misses)
    • using vector instructions
  • GPU Implementations
  • Model Compression fit bigger model -> populate samples -> train a smaller model
  • Dynamic Structure: accelerating inference => cascade of classifiers
  • lower precision to accelerate train/inference

12.2 Computer Vision

  • Preprocessing:
    • Standardization
    • Contrast Normalization
      1. global contrast normalization
      2. local contrast normalization
    • Dataset Augmentation

12.3 Speech Recognition

  • LSTM RNN
  • CTC framework

12.4 Natural Language Processing

  • n-grams
  • Neural Language Models
    • word embeddings

12.5 Other Applications

  • Recommender Systems
    • reinforcement learning
  • Knowledge Representation, Reasoning and Question Answering