Apache Singa

A Tutorial of SINGA V1

Welcome to this tutorial for Apache Incubator-singa. Please install conda and SINGA before running the notebooks.

Outline

Introduction

Training and Inference of deep learning models

Training objective

$\min_{\theta} \sum_{<x, y>\in D}L(f(x|\theta), y)$

Training procedure

  1. randomly initialize $\theta$
  2. get a mini-batch of data $\{<x, y>\}$ and compute $\frac{\partial L}{\partial \theta}$
  3. $\theta \leftarrow \theta - \alpha \frac{\partial L}{\partial \theta}$
  4. repeat 2-3 until converge, i.e., $||\frac{\partial L}{\partial \theta}|| < \epsilon$

Back-propagation for gradient computation

SINGA V1

Software Stack

Progress

Components Feature Status
Core Tensor implementation V1.0
Device implementation (CppCPU, CudaGPU, OpenclGPU) V1.1
Model Basic layers for CNN and RNN V1.0
Basic loss and metric functions V1.0
Basic optimizer and initializers V1.0
IO Message paasing via socket V1.0
Installation From source on Linux and Mac V1.0
From source on Windows V1.1
From wheel on Linux and Mac V1.0
From debian package on Ubuntu V1.1
Docker images V1.1
AWS AMI V1.1

Schedule

Components Feature Status
Core Tensor API = Numpy API V1.2
Operation scheduling and memory optimization V1.3
Model To support more vison, NLP examples V1.2
General neural net API V1.2
To support reinforcement learning examples V1.3
Auto-grad + Functional API TBD
IO HDFS integration V1.3
Server Consistency and communication optimization V1.2
Message passing via Infiniband V1.2
Installation From wheel on Windows V1.2
From debian on Linux V1.2