Numba 0.44.0 Release Demo

This notebook contains a demonstration of new features present in the 0.44.0 release of Numba. Whilst release notes are produced as part of the CHANGE_LOG, there's nothing like seeing code in action!


🔥IMPORTANT: Numba is officially deprecating some features and behaviours in this release!🔥

More information about this is given below.


It is also worth noting that this release of Numba is now backed by LLVM 8 on all platforms except ppc64le (due to numerous issues present solely on that platform). As a result of the ppc64le issues, LLVM 7.0.x and 7.1.x are also still supported on all platforms. This release also fully accommodates new and changed functionality required to be compatible with NumPy 1.16.

Also included are demonstrations of:

  • Type inferred dictionaries
  • New unicode features
  • Newly supported NumPy functions
  • A few new miscellaneous features!

First, import the necessary from Numba and NumPy...

In [ ]:
from numba import jit, njit, config, __version__
from numba.extending import overload
import numpy as np
assert tuple(int(x) for x in __version__.split('.')[:2]) >= (0, 44)

Deprecated features and behaviours

Numba Version 0.44 is deprecating a number of features and issuing pending-deprecation notices for others. Extensive documentation including examples, recommendations and removal schedules can be found here. The deprecations/pending deprecations most likely to impact your code relate to:

NOTE: Suppressing these warnings is easy and explained along with example code here.

Deprecation of List and Set reflection

Reflection is the jargon used in Numba to describe the process of ensuring that changes made by compiled code to arguments that are mutable Python container data types are visible in the Python interpreter when the compiled function returns. Numba has for some time supported reflection of list and set data types and it is support for this reflection that is scheduled for deprecation with view to replace with a better implementation.

Here is an example of code that would be impacted by such a change:

In [ ]:
from numba import njit

@njit
def foo(x):
  x.append(10) # changes made here need "reflecting" back to `a` in the outer scope

a = [1, 2, 3]
foo(a)

Deprecation of object mode fall-back behaviour when using @jit

The numba.jit decorator has for a long time followed the behaviour of first attempting to compile the decorated function in nopython mode and should this compilation fail it will fall-back and try again to compile but this time in object mode. It it this fall-back behaviour which is being deprecated, the result of which will be that numba.jit will by default compile in nopython mode and object mode compilation will become opt-in only.

Here is an example of code that would be impacted by such a change:

In [ ]:
@jit # `nopython` mode was not explicitly requested, so fall-back is permitted
def bar():
    l = []
    for x in range(10):
        l.append(x)
    # no "reversed" support in nopython mode so compilation will fall-back to object mode
    reversed(l)

bar()

Other deprecations

Along with the above, some lesser used features and behaviours are being deprecated, namely:

For developers writing Numba extensions/leveraging the Numba internal APIs, the following deprecation is made:

Dictionary Support Enhancements

Initial support for dictionaries was implemented in Version 0.43 via the "typed" dictionary. As a limitation of the initial work the key and value types had to be specified on construction. Numba Version 0.44 removes this restriction and implements type inference for the typed dictionary.

Specifying the type still works as before:

In [ ]:
from numba.typed import Dict
from numba import int32, float32

a_dictionary = Dict.empty(int32, float32)

Type infererence also now works, whereby the type of the dictionary is inferred at compile time by how it is used:

In [ ]:
@njit
def infer_dict():
    d1 = {}
    d1[10] = 's' # Numba works out that d1{} is equivalent to Dict.empty(int64, unicode_type)
    return d1

d = infer_dict()
for k, v in d.items():
    print("%s:%s" % (k, v))

from numba import typeof
print(typeof(d))

The curly brace initialisation syntax and the dict reserved word are also both supported (Note: dict() cannot take arguments yet).

In [ ]:
@njit
def curly_braces_and_dict():
    curly = {'a': 1.0, 'b': 2.0} # inferred as DictType[unicode_type,float64]
    reserved = dict()
    reserved[10] = np.zeros(5) # inferred as DictType[int64,array(float64, 1d, C)]
    return curly, reserved
    
for x in curly_braces_and_dict():
    print(typeof(x))
    for k, v in x.items():
        print(k, v)

This is an update to the snake graph example from the previous release notebook, making use of the dictionary type inference enhancement, also, Version 0.44 supports unicode string iteration, so the example now makes use of that too!

In [ ]:
@njit
def snake_graph():
    """
    This function creates a dictionary (str->int) and then randomly increases the integer
    values and finally prints the results as a snake-y graph.
    """
    # set up
    x = dict() # <-- the type of x will be inferred
    bins = "numba"
    for v in bins: # <-- iterate a string directly
        x[v] = 0 # <-- Numba works out that x is a DictType[unicode_type,int64]
    
    # add values to bins at random
    it = 100
    for i in range(it):
        key = bins[np.random.randint(len(bins))]
        x[key] += 1
    
    # iterate the key space and print a "bar" of snakes
    for k in x.keys():
        print(k + ':' + ''.join(['🐍' for _ in range(x[k])]))

    return x
        
r = snake_graph()

r

New Unicode features

A few more features were added to Numba's unicode support bringing complete support ever closer. The added string instance methods are:

  • .zfill()
  • .ljust()
  • .rjust()
  • .center()
  • .strip()
  • .lstrip()
  • .rstrip()

Support for string multiplication and iteration is also added. This example includes both the new dictionary features and new string features:

In [ ]:
@njit
def decode():
    char = '#'
    w = 35
    encode = [(0, 0, 0, False), (0, 0, 0, False), (17, 6, 11, False),
              (13, 12, 9, True), (10, 15, 9, False), (8, 14, 12, False),
              (6, 13, 15, False), (5, 26, 3, True), (3, 30, 1, True),
              (3, 30, 1, True), (2, 31, 1, True), (3, 30, 1, True),
              (3, 29, 2, True), (4, 27, 3, True), (8, 21, 5, True),
              (19, 8, 7, False), (16, 9, 9, False), (14, 15, 5, False),
              (18, 9, 7, False), (17, 7, 10, False), (16, 6, 12, False),
              (15, 4, 15, False), (14, 3, 17, False), (13, 2, 19, False)]
    slices = {9: '      #######       ################   ',
              3: ' ######  ####  ',
              10: '               #######    ####################',
              13: '    ###########        ######## ',
              14: '#            ########                  ',
              11: '  ####################  ########',
              7: '              ##########          ######',
              8: '  #########        #############   ',
              12: '################      #######    ',}
    ret = dict() # newly supported dictionary use case
    for ix, enc in enumerate(encode):
        lpad, width, rpad, lookup = enc
        if lookup:
            if ix % 2:
                s = slices[ix].strip()
            else:
                s = slices[ix].lstrip().rstrip()
            buf = ' ' * lpad + s + ' ' * rpad # string multiplication
        elif ix % 2:
            buf = (char * width).rjust(lpad + width) # string rjust
        else:
            buf = ' ' * lpad + (char * width).ljust(lpad + rpad) # string ljust
        ret[ix] = buf

    title = ("Numba " + '.44'.zfill(4)).center(w, '~') # string zfill and center
    newtitle = ""
    for ix, t in enumerate(title): # string iteration
        if t == '~':
            if ix % 2:
                newtitle += '⚡'
            elif ix % 3:
                newtitle += '🐍'
            else:
                newtitle += t
        else:
            newtitle += t
    ret[0] = newtitle
    return ret
In [ ]:
for k, v in sorted(decode().items()):
    print(k, v)

Newly supported NumPy functions

This release contains a number of newly supported NumPy functions:

  • Repetition of arrays: np.repeat
  • Deleting parts of an array: np.delete
  • np.ndarray.shape wiring for completeness: np.shape
  • Quantiles: np.quantile and np.nanquantile
In [ ]:
@njit
def numpy_new():

    arr = np.array([[1,2],[3,4]])
    
    # np.repeat
    repeated = np.repeat(arr, 4)
    print(repeated)
    
    # np.delete
    deleted = np.delete(repeated, 3)
    print(deleted)
    
    # np.shape
    print(np.shape(deleted))
    
    # np.quantile
    print(np.quantile([1, 2, 3, 4], 0.5))
    
    # np.nanquantile
    print(np.nanquantile([np.nan, 1, 2, 3, 4], 0.5))
    
numpy_new()

Miscellaneous features

Some new features were added that don't fit anywhere in particular but are still very useful. It is now possible to .view() a scalar NumPy type as another NumPy type of the same width, which is particularly helpful if writing low level numerical routines.

In [ ]:
import struct
@njit
def extract_fraction_float64(x):
    """ extracts the fraction part of an IEEE754 double precision (float64) representation
    """
    x_as_int = np.float64(x).view(np.uint64)
    w = 64
    frac_bits = 52
    return np.uint64(x_as_int & (np.uint64(-1) >> w - frac_bits))   
    
val = -np.pi
print("hex bits          :", hex(struct.unpack('Q', struct.pack('d', val))[0]))
print("extracted fraction:   ", hex(extract_fraction_float64(val)))

Also, max and min now work on iterables, a small but hugely useful addition e.g.

In [ ]:
@njit
def foo():
    x = [10, 3, -4]
    print(min(x), max(x))
    
foo()