SINGA Core Classes

Device

A device instance represents a hardware device with multiple execution units, e.g.,

  • A GPU which has multile cuda streams
  • A CPU which has multiple threads

All data structures (variables) are allocated on a device instance. Consequently, all operations are executed on the resident device.

Create a device instance

In [1]:
from singa import device
default_dev = device.get_default_device()
gpu = device.create_cuda_gpu()  # the first gpu device
gpu
Out[1]:
<singa.singa_wrap.Device; proxy of <Swig Object of type 'std::shared_ptr< singa::Device > *' at 0x7f69a05ff330> >

NOTE: currently we can only call the creating function once due to the cnmem restriction.

In [ ]:
gpu = device.create_cuda_gpu_on(1)  # use the gpu device with the specified GPU ID
gpu_list1 = device.create_cuda_gpus(2)  # the first two gpu devices
gpu_list2 = device.create_cuda_gpus([0,2]) # create the gpu instances on the given GPU IDs
opencl_gpu = device.create_opencl_device()  # valid if SINGA is compiled with USE_OPENCL=ON
In [2]:
device.get_num_gpus()
Out[2]:
3
In [3]:
device.get_gpu_ids()
Out[3]:
(0, 1, 2)

Tensor

A tensor instance represents a multi-dimensional array allocated on a device instance. It provides linear algbra operations, like +, -, *, /, dot, pow ,etc

NOTE: class memeber functions are inplace; global functions are out-of-place.

Create tensor instances

In [4]:
from singa import tensor
import numpy as np
a = tensor.Tensor((2, 3))
a.shape
Out[4]:
(2, 3)
In [5]:
a.device
Out[5]:
<singa.singa_wrap.Device; proxy of <Swig Object of type 'std::shared_ptr< singa::Device > *' at 0x7f69a02f8a50> >
In [6]:
gb = tensor.Tensor((2, 3), gpu)
In [7]:
gb.device
Out[7]:
<singa.singa_wrap.Device; proxy of <Swig Object of type 'std::shared_ptr< singa::Device > *' at 0x7f69a05ff330> >

Initialize tensor values

In [8]:
a.set_value(1.2)
gb.gaussian(0, 0.1)

To and from numpy

In [9]:
tensor.to_numpy(a)
Out[9]:
array([[ 1.20000005,  1.20000005,  1.20000005],
       [ 1.20000005,  1.20000005,  1.20000005]], dtype=float32)
In [10]:
tensor.to_numpy(gb)
Out[10]:
array([[ 0.24042693, -0.21337385, -0.0969397 ],
       [-0.010797  , -0.07642138, -0.09220808]], dtype=float32)
In [11]:
c = tensor.from_numpy(np.array([1,2], dtype=np.float32))
c.shape
Out[11]:
(2,)
In [12]:
c.copy_from_numpy(np.array([3,4], dtype=np.float32))
tensor.to_numpy(c)
Out[12]:
array([ 3.,  4.], dtype=float32)

Move tensor between devices

In [13]:
gc = c.clone()
gc.to_device(gpu)
gc.device
Out[13]:
<singa.singa_wrap.Device; proxy of <Swig Object of type 'std::shared_ptr< singa::Device > *' at 0x7f69a05ff330> >
In [14]:
b = gb.clone()
b.to_host()  # the same as b.to_device(default_dev)
b.device
Out[14]:
<singa.singa_wrap.Device; proxy of <Swig Object of type 'std::shared_ptr< singa::Device > *' at 0x7f69a02f8a50> >

Operations

NOTE: tensors should be initialized if the operation would read the tensor values

Summary

In [15]:
gb.l1()
Out[15]:
0.12169448286294937
In [16]:
a.l2()
Out[16]:
0.4898979663848877
In [17]:
e = tensor.Tensor((2, 3))
e.is_empty()
Out[17]:
False
In [18]:
gb.size()
Out[18]:
6L
In [19]:
gb.memsize()
Out[19]:
24L
In [20]:
# note we can only support matrix multiplication for tranposed tensors; 
# other operations on transposed tensor would result in errors
c.is_transpose()  
Out[20]:
False
In [21]:
et=e.T()
et.is_transpose()
Out[21]:
True
In [22]:
et.shape
Out[22]:
(3L, 2L)
In [23]:
et.ndim()
Out[23]:
2L

Member functions (in-place)

These functions would change the content of the tensor

In [24]:
a += b
tensor.to_numpy(a)
Out[24]:
array([[ 1.44042695,  0.98662621,  1.10306036],
       [ 1.18920302,  1.12357867,  1.10779202]], dtype=float32)
In [25]:
a -= b
tensor.to_numpy(a)
Out[25]:
array([[ 1.20000005,  1.20000005,  1.20000005],
       [ 1.20000005,  1.20000005,  1.20000005]], dtype=float32)
In [26]:
a *= 2
tensor.to_numpy(a)
Out[26]:
array([[ 2.4000001,  2.4000001,  2.4000001],
       [ 2.4000001,  2.4000001,  2.4000001]], dtype=float32)
In [27]:
a /= 3
tensor.to_numpy(a)
Out[27]:
array([[ 0.80000007,  0.80000007,  0.80000007],
       [ 0.80000007,  0.80000007,  0.80000007]], dtype=float32)
In [28]:
d = tensor.Tensor((3,))
d.uniform(-1,1)
tensor.to_numpy(d)
Out[28]:
array([ 0.62944734, -0.72904599,  0.81158388], dtype=float32)
In [29]:
a.add_row(d)
tensor.to_numpy(a)
Out[29]:
array([[ 1.42944741,  0.07095408,  1.61158395],
       [ 1.42944741,  0.07095408,  1.61158395]], dtype=float32)

Global functions (out of place)

These functions would not change the memory of the tensor, instead they return a new tensor

Unary functions

In [30]:
h = tensor.sign(d)
tensor.to_numpy(h)
Out[30]:
array([ 1., -1.,  1.], dtype=float32)
In [31]:
tensor.to_numpy(d)
Out[31]:
array([ 0.62944734, -0.72904599,  0.81158388], dtype=float32)
In [32]:
h = tensor.abs(d)
tensor.to_numpy(h)
Out[32]:
array([ 0.62944734,  0.72904599,  0.81158388], dtype=float32)
In [33]:
h = tensor.relu(d)
tensor.to_numpy(h)
Out[33]:
array([ 0.62944734,  0.        ,  0.81158388], dtype=float32)
In [34]:
g = tensor.sum(a, 0)
g.shape
Out[34]:
(3L,)
In [35]:
g = tensor.sum(a, 1)
g.shape
Out[35]:
(2L,)
In [36]:
tensor.bernoulli(0.5, g)
tensor.to_numpy(g)
Out[36]:
array([ 1.,  0.], dtype=float32)
In [37]:
g.gaussian(0, 0.2)
tensor.gaussian(0, 0.2, g)
tensor.to_numpy(g)
Out[37]:
array([-0.12226005, -0.05827543], dtype=float32)

Binary functions

In [38]:
f = a + b
tensor.to_numpy(f)
Out[38]:
array([[ 1.66987431, -0.14241977,  1.51464427],
       [ 1.41865039, -0.0054673 ,  1.51937592]], dtype=float32)
In [39]:
g = a < b
tensor.to_numpy(g)
Out[39]:
array([[ 0.,  0.,  0.],
       [ 0.,  0.,  0.]], dtype=float32)
In [40]:
tensor.add_column(2, c, 1, f)   # f = 2 *c + 1* f
tensor.to_numpy(f)
Out[40]:
array([[ 7.66987419,  5.85758018,  7.51464415],
       [ 9.41865063,  7.99453259,  9.5193758 ]], dtype=float32)

BLAS

BLAS function may change the memory of input tensor

In [41]:
tensor.axpy(2, a, f)  # f = 2a + f
tensor.to_numpy(b)
Out[41]:
array([[ 0.24042693, -0.21337385, -0.0969397 ],
       [-0.010797  , -0.07642138, -0.09220808]], dtype=float32)
In [42]:
f = tensor.mult(a, b.T())
tensor.to_numpy(f)
Out[42]:
array([[ 0.17231143, -0.16945721],
       [ 0.17231143, -0.16945721]], dtype=float32)
In [43]:
tensor.mult(a, b.T(), f, 2, 1)  # f = 2a*b.T() + 1f
tensor.to_numpy(f)
Out[43]:
array([[ 0.51693428, -0.50837165],
       [ 0.51693428, -0.50837165]], dtype=float32)