High Performance Python

Python is slower than compiled languages for a variety of reasons:

Python is Dynamically Typed rather than Statically Typed.

What this means is that at the time the program executes, the interpreter doesn't know the type of the variables that are defined. For example, the difference between a C variable and a Python variable is summarized by this diagram:

For a variable in C, the compiler knows the type by its very definition. For a variable in Python, all you know at the time the program executes is that it's some sort of Python object.

So if you write the following in C:

int a = 1;
int b = 2;
int c = a + b;

the C compiler knows from the start that a and b are integers: they simply can't be anything else! With this knowledge, it can call the routine which adds two integers, returning another integer which is just a simple value in memory. As a rough schematic, the sequence of events looks like this:

C Addition

  1. Assign 1 to a
  2. Assign 2 to b
  3. call binary_add<int, int>(a, b)
  4. Assign the result to c

The equivalent code in Python looks like this:

a = 1
b = 2
c = a + b

here the interpreter knows only that 1 and 2 are objects, but not what type of object they are. So the The interpreter must inspect PyObject_HEAD for each variable to find the type information, and then call the appropriate summation routine for the two types. Finally it must create and initialize a new Python object to hold the return value. The sequence of events looks roughly like this:

Python Addition

  1. Assign 1 to a
    • Set a->PyObject_HEAD->typecode to integer
    • Set a->val = 1
  2. Assign 2 to b
    • Set b->PyObject_HEAD->typecode to integer
    • Set b->val = 2
  3. call binary_add(a, b)
    • find typecode in a->PyObject_HEAD
    • a is an integer; value is a->val
    • find typecode in b->PyObject_HEAD
    • b is an integer; value is b->val
    • call binary_add<int, int>(a->val, b->val)
    • result of this is result, and is an integer.
  4. Create a Python object c
    • set c->PyObject_HEAD->typecode to integer
    • set c->val to result

The dynamic typing means that there are a lot more steps involved with any operation. This is a primary reason that Python is slow compared to C for operations on numerical data.

Python is interpreted rather than compiled.

We saw above one difference between interpreted and compiled code. A smart compiler can look ahead and optimize for repeated or unneeded operations, which can result in speed-ups. Compiler optimization is its own beast, and I'm personally not qualified to say much about it, so I'll stop there.

Python's object model can lead to inefficient memory access

We saw above the extra type info layer when moving from a C integer to a Python integer. Now imagine you have many such integers and want to do some sort of batch operation on them. In Python you might use the standard List object, while in C you would likely use some sort of buffer-based array.

A NumPy array in its simplest form is a Python object build around a C array. That is, it has a pointer to a contiguous data buffer of values. A Python list, on the other hand, has a pointer to a contiguous buffer of pointers, each of which points to a Python object which in turn has references to its data (in this case, integers). This is a schematic of what the two might look like:

You can see that if you're doing some operation which steps through data in sequence, the numpy layout will be much more efficient than the Python layout, both in the cost of storage and the cost of access.

Speeding up statistical computations in Python

In the age of "big data" and sophisitcated Bayesian and statistical learning algorithms, many are interested in optimizing the performance of the high-level languages that we use to analyse data.

NumPy gets us part of the way there on Python:

  • Storage of multidimensional data
  • Efficient data access
  • Efficient in-memory storage
  • Fast methods and functions for data manipulation

Ffor many applications, this is sufficient to drastically improve performance. However, there is plenty of scope for improving Python's performance in situations where speed matters.

Pure Python and Python with NumPy are not particularly fast. Below are some recent performance benchmarks comparing several computing languages (taken directly from the Julia website):

FortranJuliaPythonRMatlabOctaveMathematicaJavaScriptGoLuaJITJava
gcc 5.1.1 0.4.0 3.4.3 3.2.2 R2015b 4.0.0 10.2.0 V8 3.28.71.19 go1.5 gsl-shell 2.3.1 1.8.0_45
fib0.702.1177.76533.5226.899324.35118.533.361.861.711.21
parse_int5.051.4517.0245.73802.529581.4415.026.061.205.773.35
quicksort1.311.1532.89264.544.921866.0143.232.701.292.032.60
mandel0.810.7915.3253.167.58451.815.130.661.110.671.35
pi_sum1.001.0021.999.561.00299.311.691.011.001.001.00
rand_mat_stat1.451.6617.9314.5614.5230.935.952.302.963.273.92
rand_mat_mul3.481.021.141.571.121.121.3015.071.421.162.36

Figure: benchmark times relative to C (smaller is better, C performance = 1.0).

So, while fast relative to some scientific compution choices (e.g. R, Matlab), Python sometimes needs to be tweaked in order to make it a competitive choice for implementing modern statistical methods. We will cover two approachable ways of improving the performance of Python.

One does have to be careful when interpreting benchmarks. They involve many assumptions about how algorithms are implemented that may not be applicable to real-world applications in a particular language.

Profiling

Before you barrel ahead and prematurely optimize your Python code, it is important to understand why and where your code is slow. This is achieved by systematically accounting for the resources that your code is using, such as memory, CPU time or data transfer. This process is broadly referred to as Profiling, and it allows you to identify where the performance bottlenecks in your code lie.

Here, we will concentrate on optimizing performance for CPU-bound problems.

There are a number of tools to help you profile your code.

time

For those of you on UNIX platforms, the built-in utility time can be used to assess how long your code takes to run.

In [1]:
!time python ../examples/abc.py
/bin/sh: 1: time: not found

The output from time can be interpreted as:

  • real: elapsed (wall) time
  • user: time spent in your code
  • sys: time spent in system (kernel) functions

The last 2 quantities account for the cycles used by your program. The remaining real time is often due to waiting for information either from disk or a network connection (I/O).

Python also has a time module (and function) that is more rudimentary; it simply returns the time, in seconds from the Epoch (1/1/1970).

In [2]:
import time
time.time()
Out[2]:
1545190399.5904741

We can use this for profiling by differencing the times before and after running some code of interest:

In [3]:
import numpy as np
start_time = time.time()
np.product(range(1, 100000))
end_time = time.time()

end_time - start_time
Out[3]:
0.03713417053222656

Note, however that it does not provide a breakdown of where the code spends its time.

IPython magic: %timeit, %run and %prun

IPython has three built-in "magic" functions that are useful for profiling your code.

The %timeit magic executes a Python statement or expressions in a loop to see how long we expect it to take for any given call. Additionally, it repeats the loop a certain number of times, and returns the best result.

As an example, consider a Python implementation of the trapezoidal rule, a method from numerical analysis for approximating a definite integral. Specifically, it allows us to approximate:

$$\int_a^b f(x) dx$$

using the approximation:

$$\int_a^b f(x) dx \approx (b-a) \frac{f(b) + f(a)}{2}$$

Rather than use a single interval for this estimate, we break the interval down into $n$ subintervals, to obtain a more accurate approximation.

In [4]:
def f(x):
    return 2*x*x + 3*x + 1
      
def trapez(a, b, n):
    h = (b-a)/n 
    sumy = 0
    x = a
    for i in range(n):
        x += h
        sumy += f(x)
    sumy += 0.5*(f(a) + f(b))
    return sumy*h
In [5]:
trapez(1, 5, 10000)
Out[5]:
122.6930668799846

To confirm that this works, we can compare this to the symbolic solution, using Sympy:

In [6]:
import sympy as sym

xs = sym.symbols('xs')

fx = 2*xs*xs + 3*xs + 1

ifx = sym.integrate(fx, (xs, 1, 5))
ifx.evalf()
Out[6]:
122.666666666667
In [7]:
%timeit -n 50 -r 5 trapez(1, 5, 10000)
3.25 ms ± 338 µs per loop (mean ± std. dev. of 5 runs, 50 loops each)

%timeit tries to pick suitable values for the number of loops and repeats; these values can be overriden by specifying -n and -r values, respectively.

The %run command with a -p option allows you to run complete programs under the control of the Python profiler. It writes the output to the help pane, which opens at the bottom of the page.

In [8]:
# This code redirects pager output to a regular cell
from IPython.core import page
page.page = print
In [9]:
%run -p ../examples/abc.py
mu       4.617512
sigma    1.699296
dtype: float64
         1748732 function calls (1744952 primitive calls) in 6.083 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    48742    1.228    0.000    2.467    0.000 _methods.py:91(_var)
   146233    0.594    0.000    0.594    0.000 {method 'reduce' of 'numpy.ufunc' objects}
    48742    0.476    0.000    1.236    0.000 _methods.py:58(_mean)
    48742    0.389    0.000    2.951    0.000 _methods.py:138(_std)
    97484    0.369    0.000    0.458    0.000 _methods.py:48(_count_reduce_items)
    48743    0.320    0.000    0.320    0.000 {method 'normal' of 'mtrand.RandomState' objects}
        1    0.305    0.305    5.157    5.157 abc.py:4(abc)
299336/299215    0.304    0.000    0.312    0.000 {built-in method builtins.isinstance}
   146226    0.256    0.000    0.432    0.000 numeric.py:504(asanyarray)
197736/197604    0.191    0.000    0.201    0.000 {built-in method builtins.issubclass}
   146235    0.176    0.000    0.176    0.000 {built-in method numpy.core.multiarray.array}
   149840    0.162    0.000    0.162    0.000 {built-in method builtins.hasattr}
    48742    0.108    0.000    3.059    0.000 {method 'std' of 'numpy.ndarray' objects}
    48742    0.108    0.000    1.343    0.000 {method 'mean' of 'numpy.ndarray' objects}
      170    0.075    0.000    0.075    0.000 {built-in method marshal.loads}
    48805    0.072    0.000    0.072    0.000 {built-in method builtins.max}
      213    0.058    0.000    0.062    0.000 <frozen importlib._bootstrap>:318(__exit__)
    24371    0.056    0.000    0.056    0.000 {method 'uniform' of 'mtrand.RandomState' objects}
    48742    0.047    0.000    0.047    0.000 {built-in method builtins.abs}
    43/27    0.046    0.001    0.134    0.005 {built-in method _imp.create_dynamic}
      613    0.034    0.000    0.125    0.000 textwrap.py:414(dedent)
      883    0.033    0.000    0.033    0.000 {method 'sub' of '_sre.SRE_Pattern' objects}
  613/593    0.033    0.000    0.203    0.000 {built-in method builtins.__build_class__}
28817/28680    0.031    0.000    0.031    0.000 {built-in method builtins.len}
    18015    0.030    0.000    0.030    0.000 {method 'startswith' of 'str' objects}
      613    0.025    0.000    0.025    0.000 {method 'findall' of '_sre.SRE_Pattern' objects}
      958    0.019    0.000    0.019    0.000 {built-in method posix.stat}
      170    0.015    0.000    0.015    0.000 {method 'read' of '_io.FileIO' objects}
      420    0.015    0.000    0.072    0.000 <frozen importlib._bootstrap_external>:1233(find_spec)
   149/58    0.011    0.000    0.035    0.001 sre_parse.py:470(_parse)
     2129    0.011    0.000    0.031    0.000 <frozen importlib._bootstrap_external>:57(_path_join)
      333    0.011    0.000    0.109    0.000 <frozen importlib._bootstrap>:870(_find_spec)
      170    0.011    0.000    0.025    0.000 <frozen importlib._bootstrap_external>:830(get_data)
     2129    0.010    0.000    0.017    0.000 <frozen importlib._bootstrap_external>:59(<listcomp>)
     4126    0.010    0.000    0.010    0.000 {built-in method builtins.getattr}
    399/2    0.009    0.000    0.923    0.461 <frozen importlib._bootstrap>:966(_find_and_load)
    209/1    0.008    0.000    6.084    6.084 {built-in method builtins.exec}
      765    0.007    0.000    0.014    0.000 <frozen importlib._bootstrap>:157(_get_module_lock)
     4601    0.007    0.000    0.007    0.000 {method 'rstrip' of 'str' objects}
3167/3163    0.007    0.000    0.010    0.000 {method 'join' of 'str' objects}
1246/1035    0.006    0.000    0.366    0.000 <frozen importlib._bootstrap>:997(_handle_fromlist)
    399/2    0.006    0.000    0.922    0.461 <frozen importlib._bootstrap>:936(_find_and_load_unlocked)
      765    0.006    0.000    0.007    0.000 <frozen importlib._bootstrap>:78(acquire)
   261/53    0.006    0.000    0.018    0.000 sre_compile.py:64(_compile)
      430    0.006    0.000    0.019    0.000 copy.py:66(copy)
      170    0.005    0.000    0.008    0.000 <frozen importlib._bootstrap_external>:430(_validate_bytecode_header)
      765    0.005    0.000    0.007    0.000 <frozen importlib._bootstrap>:103(release)
     4367    0.005    0.000    0.005    0.000 {method 'append' of 'list' objects}
      547    0.005    0.000    0.122    0.000 _decorators.py:300(__call__)
      900    0.005    0.000    0.035    0.000 __init__.py:829(__setitem__)
      340    0.005    0.000    0.016    0.000 <frozen importlib._bootstrap_external>:263(cache_from_source)
      333    0.005    0.000    0.084    0.000 <frozen importlib._bootstrap_external>:1117(_get_spec)
      213    0.005    0.000    0.023    0.000 <frozen importlib._bootstrap>:504(_init_module_attrs)
      170    0.005    0.000    0.131    0.001 <frozen importlib._bootstrap_external>:743(get_code)
     2273    0.005    0.000    0.005    0.000 <frozen importlib._bootstrap>:222(_verbose_message)
     1113    0.005    0.000    0.005    0.000 {method 'format' of 'str' objects}
    213/1    0.005    0.000    0.921    0.921 <frozen importlib._bootstrap>:651(_load_unlocked)
     1543    0.004    0.000    0.004    0.000 {built-in method builtins.setattr}
      677    0.004    0.000    0.072    0.000 re.py:286(_compile)
     1910    0.004    0.000    0.004    0.000 {method 'rpartition' of 'str' objects}
     1769    0.004    0.000    0.004    0.000 sre_parse.py:232(__next)
     1391    0.004    0.000    0.007    0.000 sre_parse.py:253(get)
     2279    0.003    0.000    0.003    0.000 {built-in method _imp.acquire_lock}
     2026    0.003    0.000    0.003    0.000 {method 'get' of 'dict' objects}
      429    0.003    0.000    0.008    0.000 copy.py:268(_reconstruct)
     2279    0.003    0.000    0.003    0.000 {built-in method _imp.release_lock}
      126    0.003    0.000    0.006    0.000 expr.py:152(<genexpr>)
     1119    0.003    0.000    0.005    0.000 <frozen importlib._bootstrap>:847(__exit__)
      982    0.003    0.000    0.005    0.000 sre_parse.py:163(__getitem__)
      925    0.003    0.000    0.021    0.000 <frozen importlib._bootstrap_external>:75(_path_stat)
      395    0.003    0.000    0.005    0.000 <frozen importlib._bootstrap>:176(cb)
     1119    0.003    0.000    0.005    0.000 <frozen importlib._bootstrap>:843(__enter__)
      122    0.003    0.000    0.005    0.000 sre_compile.py:250(_optimize_charset)
      366    0.003    0.000    0.012    0.000 <frozen importlib._bootstrap>:194(_lock_unlock_module)
  399/192    0.003    0.000    0.004    0.000 sre_parse.py:173(getwidth)
   112/14    0.003    0.000    0.008    0.001 abc.py:196(__subclasscheck__)
    188/7    0.003    0.000    0.010    0.001 pyparsing.py:1380(_parseNoCache)
      107    0.003    0.000    0.005    0.000 functools.py:44(update_wrapper)
     1530    0.002    0.000    0.002    0.000 {built-in method _thread.get_ident}
      213    0.002    0.000    0.004    0.000 <frozen importlib._bootstrap_external>:524(spec_from_file_location)
      395    0.002    0.000    0.004    0.000 <frozen importlib._bootstrap>:58(__init__)
      170    0.002    0.000    0.079    0.000 <frozen importlib._bootstrap_external>:485(_compile_bytecode)
        1    0.002    0.002    0.005    0.005 __init__.py:978(_rc_params_in_file)
      399    0.002    0.000    0.016    0.000 <frozen importlib._bootstrap>:147(__enter__)
        5    0.002    0.000    0.042    0.008 _collections_abc.py:824(update)
       68    0.002    0.000    0.018    0.000 ops.py:1486(_arith_method_FRAME)
     1274    0.002    0.000    0.002    0.000 {method 'strip' of 'str' objects}
  213/190    0.002    0.000    0.159    0.001 <frozen importlib._bootstrap>:564(module_from_spec)
        1    0.002    0.002    0.002    0.002 {method 'readlines' of '_io._IOBase' objects}
   115/53    0.002    0.000    0.036    0.001 sre_parse.py:407(_parse_sub)
     1206    0.002    0.000    0.002    0.000 sre_parse.py:248(match)
       29    0.002    0.000    0.002    0.000 {built-in method posix.listdir}
      235    0.002    0.000    0.003    0.000 {built-in method builtins.any}
      213    0.002    0.000    0.011    0.000 <frozen importlib._bootstrap_external>:361(_get_cached)
      340    0.002    0.000    0.003    0.000 <frozen importlib._bootstrap_external>:63(_path_split)
    170/1    0.002    0.000    0.921    0.921 <frozen importlib._bootstrap_external>:672(exec_module)
      232    0.002    0.000    0.002    0.000 pyparsing.py:1154(__init__)
      399    0.002    0.000    0.006    0.000 <frozen importlib._bootstrap>:151(__exit__)
       11    0.002    0.000    0.008    0.001 ops.py:861(add_methods)
      619    0.002    0.000    0.002    0.000 {method 'update' of 'dict' objects}
      429    0.002    0.000    0.021    0.000 pyparsing.py:1177(copy)
      547    0.002    0.000    0.002    0.000 _decorators.py:293(__init__)
      158    0.002    0.000    0.002    0.000 _decorators.py:247(__call__)
      213    0.002    0.000    0.006    0.000 <frozen importlib._bootstrap_external>:1228(_get_spec)
      587    0.002    0.000    0.002    0.000 {built-in method __new__ of type object at 0x7c1b3b1589e0}
      265    0.002    0.000    0.017    0.000 re.py:184(sub)
       11    0.002    0.000    0.041    0.004 ops.py:800(_create_methods)
      335    0.002    0.000    0.009    0.000 <frozen importlib._bootstrap_external>:85(_path_is_mode_type)
      333    0.002    0.000    0.086    0.000 <frozen importlib._bootstrap_external>:1149(find_spec)
        3    0.001    0.000    0.003    0.001 expr.py:216(disallowed)
      263    0.001    0.000    0.009    0.000 abc.py:180(__instancecheck__)
      790    0.001    0.000    0.001    0.000 {built-in method _thread.allocate_lock}
      442    0.001    0.000    0.003    0.000 generic.py:7(_check)
      354    0.001    0.000    0.006    0.000 re.py:169(match)
      440    0.001    0.000    0.004    0.000 <frozen importlib._bootstrap_external>:1080(_path_importer_cache)
      429    0.001    0.000    0.002    0.000 {method '__reduce_ex__' of 'object' objects}
    364/2    0.001    0.000    0.921    0.461 <frozen importlib._bootstrap>:211(_call_with_frames_removed)
      383    0.001    0.000    0.012    0.000 <frozen importlib._bootstrap>:403(cached)
      694    0.001    0.000    0.001    0.000 {method 'split' of 'str' objects}
        5    0.001    0.000    0.002    0.000 textwrap.py:233(_wrap_chunks)
      541    0.001    0.000    0.001    0.000 _weakrefset.py:70(__contains__)
       53    0.001    0.000    0.006    0.000 sre_compile.py:482(_compile_info)
      248    0.001    0.000    0.002    0.000 ops.py:326(_get_op_name)
       53    0.001    0.000    0.065    0.001 sre_compile.py:557(compile)
      122    0.001    0.000    0.007    0.000 sre_compile.py:223(_compile_charset)
       83    0.001    0.000    0.004    0.000 pyparsing.py:3326(__init__)
      635    0.001    0.000    0.001    0.000 {built-in method builtins.min}
      333    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:369(__init__)
      429    0.001    0.000    0.002    0.000 copyreg.py:87(__newobj__)
      852    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:321(<genexpr>)
      108    0.001    0.000    0.009    0.000 rcsetup.py:358(validate_color)
      340    0.001    0.000    0.002    0.000 <frozen importlib._bootstrap_external>:52(_r_long)
      414    0.001    0.000    0.002    0.000 sre_parse.py:171(append)
      170    0.001    0.000    0.001    0.000 {built-in method _imp._fix_co_filename}
      939    0.001    0.000    0.001    0.000 {built-in method builtins.ord}
      251    0.001    0.000    0.002    0.000 <frozen importlib._bootstrap>:416(parent)
      389    0.001    0.000    0.001    0.000 {method 'match' of '_sre.SRE_Pattern' objects}
      351    0.001    0.000    0.002    0.000 __init__.py:217(bind_method)
       22    0.001    0.000    0.007    0.000 traceback.py:319(extract)
      133    0.001    0.000    0.002    0.000 ops.py:274(_get_opstr)
      306    0.001    0.000    0.009    0.000 <frozen importlib._bootstrap_external>:94(_path_isfile)
        1    0.001    0.001    0.014    0.014 expr.py:2(<module>)
      333    0.001    0.000    0.002    0.000 <frozen importlib._bootstrap>:780(find_spec)
       48    0.001    0.000    0.003    0.000 config.py:414(register_option)
      106    0.001    0.000    0.003    0.000 enum.py:803(__and__)
      192    0.001    0.000    0.004    0.000 colors.py:116(_is_nth_color)
      381    0.001    0.000    0.001    0.000 sre_parse.py:159(__len__)
      351    0.001    0.000    0.001    0.000 sre_parse.py:285(tell)
       23    0.001    0.000    0.001    0.000 generic.py:6(create_pandas_abc_type)
      231    0.001    0.000    0.002    0.000 enum.py:267(__call__)
      568    0.001    0.000    0.001    0.000 {built-in method posix.fspath}
      399    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:143(__init__)
   113/42    0.001    0.000    0.629    0.015 {built-in method builtins.__import__}
        1    0.001    0.001    0.126    0.126 rcsetup.py:15(<module>)
       87    0.001    0.000    0.011    0.000 rcsetup.py:69(f)
      403    0.001    0.000    0.001    0.000 {method 'replace' of 'str' objects}
      170    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:393(_check_name_wrapper)
      214    0.001    0.000    0.001    0.000 rcsetup.py:118(validate_bool)
       87    0.001    0.000    0.005    0.000 rcsetup.py:90(<listcomp>)
      170    0.001    0.000    0.004    0.000 <frozen importlib._bootstrap_external>:840(path_stats)
  157/155    0.001    0.000    0.001    0.000 pyparsing.py:382(__init__)
        1    0.001    0.001    0.096    0.096 pyparsing.py:76(<module>)
       34    0.001    0.000    0.001    0.000 groupby.py:3348(_whitelist_method_generator)
        1    0.001    0.001    0.194    0.194 __init__.py:101(<module>)
      333    0.001    0.000    0.001    0.000 {built-in method _imp.is_frozen}
       53    0.001    0.000    0.038    0.001 sre_parse.py:844(parse)
       84    0.001    0.000    0.002    0.000 ops.py:623(_make_flex_doc)
      226    0.001    0.000    0.001    0.000 {method 'extend' of 'list' objects}
      340    0.001    0.000    0.001    0.000 {built-in method from_bytes}
   121/59    0.001    0.000    0.017    0.000 pyparsing.py:3429(copy)
      170    0.001    0.000    0.001    0.000 ops.py:854(<lambda>)
        1    0.001    0.001    0.010    0.010 generic.py:102(NDFrame)
     33/5    0.001    0.000    0.003    0.001 pyparsing.py:3385(streamline)
       66    0.001    0.000    0.002    0.000 pyparsing.py:3785(__init__)
      404    0.001    0.000    0.001    0.000 {method 'find' of 'bytearray' objects}
       24    0.001    0.000    0.004    0.000 deprecation.py:178(deprecate)
        1    0.001    0.001    0.001    0.001 stata.py:720(StataMissingValue)
        1    0.001    0.001    0.012    0.012 frame.py:248(DataFrame)
       67    0.001    0.000    0.004    0.000 pyparsing.py:3456(__init__)
      270    0.001    0.000    0.001    0.000 {method 'endswith' of 'str' objects}
      170    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:35(_new_module)
      231    0.001    0.000    0.001    0.000 enum.py:517(__new__)
      265    0.001    0.000    0.001    0.000 sre_parse.py:111(__init__)
      103    0.001    0.000    0.001    0.000 ops.py:182(_gen_eval_kwargs)
      157    0.001    0.000    0.001    0.000 pyparsing.py:373(__new__)
        1    0.001    0.001    0.024    0.024 parsers.py:3(<module>)
       42    0.001    0.000    0.012    0.000 ops.py:1367(_flex_method_SERIES)
       80    0.001    0.000    0.002    0.000 _weakrefset.py:58(__iter__)
       74    0.001    0.000    0.001    0.000 _weakrefset.py:36(__init__)
      213    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:311(__enter__)
     39/7    0.001    0.000    0.009    0.001 pyparsing.py:3463(parseImpl)
       29    0.001    0.000    0.003    0.000 <frozen importlib._bootstrap_external>:1067(_path_hooks)
   121/59    0.001    0.000    0.013    0.000 pyparsing.py:3431(<listcomp>)
       11    0.001    0.000    0.002    0.000 ops.py:857(<dictcomp>)
      137    0.001    0.000    0.001    0.000 expr.py:145(<lambda>)
      144    0.001    0.000    0.001    0.000 __init__.py:293(set_function_name)
        3    0.001    0.000    0.011    0.004 generic.py:8747(_add_numeric_operations)
      420    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:41(_relax_case)
    58/10    0.001    0.000    0.018    0.002 pyparsing.py:3354(leaveWhitespace)
      333    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:707(find_spec)
        1    0.001    0.001    0.011    0.011 series.py:126(Series)
        1    0.001    0.001    0.001    0.001 __init__.py:933(<dictcomp>)
     45/1    0.001    0.000    0.001    0.001 copy.py:132(deepcopy)
      120    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:861(_find_spec_legacy)
       14    0.001    0.000    0.003    0.000 abc.py:132(__new__)
       93    0.001    0.000    0.001    0.000 sre_compile.py:388(_simple)
        1    0.001    0.001    0.006    0.006 groupby.py:1171(GroupBy)
       25    0.001    0.000    0.001    0.000 {method 'split' of '_sre.SRE_Pattern' objects}
      200    0.001    0.000    0.001    0.000 sre_parse.py:81(groups)
    43/27    0.001    0.000    0.135    0.005 <frozen importlib._bootstrap_external>:919(create_module)
      361    0.001    0.000    0.001    0.000 {method 'lower' of 'str' objects}
       23    0.001    0.000    0.008    0.000 pyparsing.py:1049(_trim_arity)
      105    0.001    0.000    0.007    0.000 colors.py:121(is_color_like)
        1    0.001    0.001    0.004    0.004 base.py:183(Index)
      170    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:800(__init__)
       80    0.001    0.000    0.001    0.000 ops.py:249(_get_frame_op_default_axis)
      213    0.001    0.000    0.001    0.000 <frozen importlib._bootstrap>:307(__init__)
       27    0.001    0.000    0.035    0.001 pyparsing.py:2816(__init__)
       87    0.000    0.000    0.003    0.000 colors.py:150(to_rgba)
    66/50    0.000    0.000    0.001    0.000 sre_compile.py:414(_get_literal_prefix)
       66    0.000    0.000    0.005    0.000 pyparsing.py:1843(__add__)
      151    0.000    0.000    0.001    0.000 {built-in method builtins.all}
    22/10    0.000    0.000    0.004    0.000 pyparsing.py:3613(parseImpl)
       54    0.000    0.000    0.001    0.000 pyparsing.py:2870(__str__)
        9    0.000    0.000    0.002    0.000 accessor.py:56(_add_delegate_accessors)
       14    0.000    0.000    0.006    0.000 expr.py:150(_filter_nodes)
        1    0.000    0.000    0.002    0.002 strings.py:1882(StringMethods)
       29    0.000    0.000    0.002    0.000 <frozen importlib._bootstrap_external>:1281(_fill_cache)
        6    0.000    0.000    0.021    0.004 ops.py:875(add_special_arithmetic_methods)
      158    0.000    0.000    0.000    0.000 _decorators.py:241(__init__)
       53    0.000    0.000    0.024    0.000 sre_compile.py:542(_code)
       14    0.000    0.000    0.002    0.000 colors.py:184(_to_rgba_no_colorcycle)
       12    0.000    0.000    0.002    0.000 ops.py:1566(_flex_comp_method_FRAME)
       29    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1196(__init__)
       75    0.000    0.000    0.001    0.000 sre_parse.py:342(_escape)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.compile}
       54    0.000    0.000    0.001    0.000 sre_parse.py:223(__init__)
      101    0.000    0.000    0.000    0.000 functools.py:74(wraps)
      159    0.000    0.000    0.000    0.000 expr.py:197(_node_not_implemented)
        5    0.000    0.000    0.005    0.001 __init__.py:357(namedtuple)
        2    0.000    0.000    0.921    0.461 __init__.py:5(<module>)
       66    0.000    0.000    0.003    0.000 linecache.py:15(getline)
       63    0.000    0.000    0.001    0.000 _weakrefset.py:26(__exit__)
        1    0.000    0.000    0.555    0.555 groupby.py:1(<module>)
       30    0.000    0.000    0.000    0.000 deprecation.py:23(_generate_deprecation_message)
        1    0.000    0.000    0.048    0.048 pyparsing.py:5558(pyparsing_common)
       97    0.000    0.000    0.001    0.000 ops.py:223(_gen_fill_zeros)
        1    0.000    0.000    0.025    0.025 pytables.py:4(<module>)
       10    0.000    0.000    0.013    0.001 pyparsing.py:2675(__init__)
      224    0.000    0.000    0.000    0.000 {method 'add' of 'set' objects}
      301    0.000    0.000    0.001    0.000 __init__.py:907(__iter__)
       46    0.000    0.000    0.001    0.000 sre_parse.py:84(opengroup)
      138    0.000    0.000    0.001    0.000 expr.py:147(<genexpr>)
        1    0.000    0.000    0.236    0.236 _core.py:3(<module>)
       36    0.000    0.000    0.001    0.000 pyparsing.py:2434(__init__)
       29    0.000    0.000    0.001    0.000 sre_compile.py:376(_mk_bitmap)
       29    0.000    0.000    0.001    0.000 pyparsing.py:2852(parseImpl)
    49/13    0.000    0.000    0.006    0.000 typing.py:1145(__subclasscheck__)
        1    0.000    0.000    0.000    0.000 expr.py:266(BaseExprVisitor)
       96    0.000    0.000    0.000    0.000 ops.py:162(make_invalid_op)
       88    0.000    0.000    0.004    0.000 traceback.py:283(line)
       93    0.000    0.000    0.000    0.000 pyparsing.py:1361(preParse)
       19    0.000    0.000    0.003    0.000 generic.py:9598(_make_stat_function)
      166    0.000    0.000    0.000    0.000 pyparsing.py:3336(<genexpr>)
       86    0.000    0.000    0.001    0.000 pyparsing.py:3505(<genexpr>)
      213    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:424(has_location)
       23    0.000    0.000    0.001    0.000 rcsetup.py:57(<dictcomp>)
       56    0.000    0.000    0.001    0.000 base.py:128(_make_arithmetic_op)
        6    0.000    0.000    0.169    0.028 api.py:3(<module>)
    32/16    0.000    0.000    0.001    0.000 typing.py:1164(__setattr__)
        1    0.000    0.000    0.000    0.000 __init__.py:1078(<listcomp>)
       54    0.000    0.000    0.000    0.000 accessor.py:72(_create_delegator_property)
      148    0.000    0.000    0.000    0.000 rcsetup.py:52(func)
        1    0.000    0.000    0.015    0.015 offsets.py:2(<module>)
       22    0.000    0.000    0.007    0.000 traceback.py:200(extract_stack)
        1    0.000    0.000    0.000    0.000 _color_data.py:992(<dictcomp>)
        1    0.000    0.000    0.000    0.000 __init__.py:1120(<listcomp>)
        1    0.000    0.000    0.003    0.003 panel.py:107(Panel)
        1    0.000    0.000    0.000    0.000 offsets.py:2214(Tick)
       68    0.000    0.000    0.000    0.000 _weakrefset.py:81(add)
       90    0.000    0.000    0.000    0.000 rcsetup.py:59(__call__)
        2    0.000    0.000    0.072    0.036 base.py:1(<module>)
        1    0.000    0.000    0.004    0.004 window.py:1713(Expanding)
        1    0.000    0.000    0.015    0.015 internals.py:1(<module>)
       29    0.000    0.000    0.000    0.000 sre_compile.py:378(<listcomp>)
      232    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1202(<genexpr>)
      196    0.000    0.000    0.000    0.000 {method 'pop' of 'list' objects}
        1    0.000    0.000    0.298    0.298 series.py:3(<module>)
       13    0.000    0.000    0.000    0.000 nanops.py:60(__init__)
        1    0.000    0.000    0.000    0.000 excel.py:1568(_XlsxStyler)
        1    0.000    0.000    0.000    0.000 rrule.py:1308(_genitem)
      115    0.000    0.000    0.000    0.000 pyparsing.py:2175(__str__)
       83    0.000    0.000    0.001    0.000 pyparsing.py:2390(__init__)
        1    0.000    0.000    0.382    0.382 frame.py:10(<module>)
        1    0.000    0.000    0.013    0.013 datetimes.py:2(<module>)
      106    0.000    0.000    0.000    0.000 sre_compile.py:539(isstring)
       43    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:927(exec_module)
        9    0.000    0.000    0.002    0.000 generic.py:9618(_make_stat_function_ddof)
      236    0.000    0.000    0.000    0.000 {built-in method builtins.chr}
       58    0.000    0.000    0.013    0.000 pyparsing.py:3358(<listcomp>)
       87    0.000    0.000    0.000    0.000 rcsetup.py:399(validate_string)
        1    0.000    0.000    0.006    0.006 excel.py:3(<module>)
        3    0.000    0.000    0.020    0.007 common.py:1(<module>)
       53    0.000    0.000    0.000    0.000 {built-in method _sre.compile}
        1    0.000    0.000    0.004    0.004 window.py:1428(Rolling)
       14    0.000    0.000    0.001    0.000 abc.py:135(<setcomp>)
        1    0.000    0.000    0.019    0.019 frame.py:4(<module>)
       37    0.000    0.000    0.000    0.000 sre_parse.py:294(_class_escape)
        1    0.000    0.000    0.001    0.001 rrule.py:7(<module>)
       22    0.000    0.000    0.007    0.000 pyparsing.py:1057(extract_stack)
       30    0.000    0.000    0.004    0.000 rcsetup.py:342(validate_color_for_prop_cycle)
      188    0.000    0.000    0.000    0.000 pyparsing.py:2678(<genexpr>)
        1    0.000    0.000    0.047    0.047 generic.py:2(<module>)
       20    0.000    0.000    0.001    0.000 deprecation.py:237(finalize)
        1    0.000    0.000    0.004    0.004 nanops.py:1(<module>)
      120    0.000    0.000    0.000    0.000 six.py:184(find_module)
       63    0.000    0.000    0.000    0.000 _weakrefset.py:20(__enter__)
        1    0.000    0.000    0.653    0.653 api.py:5(<module>)
      135    0.000    0.000    0.000    0.000 rcsetup.py:144(validate_float)
        1    0.000    0.000    0.227    0.227 _converter.py:1(<module>)
      170    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:825(get_filename)
        1    0.000    0.000    0.026    0.026 dates.py:134(<module>)
        1    0.000    0.000    0.012    0.012 config_init.py:11(<module>)
        1    0.000    0.000    0.000    0.000 parsers.py:2015(PythonParser)
        1    0.000    0.000    0.007    0.007 groupby.py:3409(SeriesGroupBy)
        1    0.000    0.000    0.017    0.017 ticker.py:165(<module>)
       14    0.000    0.000    0.002    0.000 ops.py:1687(_flex_method_PANEL)
        1    0.000    0.000    0.000    0.000 offsets.py:143(DateOffset)
        1    0.000    0.000    0.002    0.002 multi.py:131(MultiIndex)
       53    0.000    0.000    0.000    0.000 sre_parse.py:828(fix_flags)
      170    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:669(create_module)
       29    0.000    0.000    0.002    0.000 <frozen importlib._bootstrap_external>:1322(path_hook_for_FileFinder)
        8    0.000    0.000    0.003    0.000 base.py:4839(_make_logical_function)
    34/26    0.000    0.000    0.001    0.000 pyparsing.py:3500(__str__)
        1    0.000    0.000    0.044    0.044 reshape.py:3(<module>)
        1    0.000    0.000    0.000    0.000 ticker.py:481(ScalarFormatter)
        1    0.000    0.000    0.001    0.001 array.py:130(SparseArray)
       21    0.000    0.000    0.008    0.000 pyparsing.py:1260(setParseAction)
       39    0.000    0.000    0.001    0.000 sre_compile.py:441(_get_charset_prefix)
        1    0.000    0.000    0.006    0.006 __init__.py:7(<module>)
        7    0.000    0.000    0.000    0.000 {built-in method builtins.dir}
        6    0.000    0.000    0.002    0.000 generic.py:9673(_make_logical_function)
        4    0.000    0.000    0.001    0.000 base.py:4710(_add_numeric_methods_binary)
        3    0.000    0.000    0.005    0.002 rcsetup.py:825(validate_cycler)
       16    0.000    0.000    0.000    0.000 getlimits.py:507(__init__)
       19    0.000    0.000    0.000    0.000 inspect.py:2450(__init__)
       97    0.000    0.000    0.000    0.000 pyparsing.py:4934(<genexpr>)
        5    0.000    0.000    0.001    0.000 base.py:4634(_add_numeric_methods_disabled)
        1    0.000    0.000    0.003    0.003 strings.py:1(<module>)
        3    0.000    0.000    0.000    0.000 {pandas._libs.lib.maybe_convert_objects}
       12    0.000    0.000    0.002    0.000 generic.py:9638(_make_cum_function)
      122    0.000    0.000    0.000    0.000 {method 'items' of 'dict' objects}
       66    0.000    0.000    0.000    0.000 linecache.py:147(lazycache)
       58    0.000    0.000    0.067    0.001 re.py:231(compile)
        8    0.000    0.000    0.000    0.000 {built-in method _warnings.warn}
        1    0.000    0.000    0.000    0.000 _core.py:75(MPLPlot)
        4    0.000    0.000    0.000    0.000 {built-in method io.open}
       21    0.000    0.000    0.037    0.002 __init__.py:1(<module>)
        1    0.000    0.000    0.004    0.004 testing.py:1(<module>)
        1    0.000    0.000    0.000    0.000 colors.py:89(<dictcomp>)
       11    0.000    0.000    0.001    0.000 ops.py:741(_get_method_wrappers)
        1    0.000    0.000    0.003    0.003 ops.py:5(<module>)
      137    0.000    0.000    0.000    0.000 pyparsing.py:2083(setWhitespaceChars)
       66    0.000    0.000    0.003    0.000 linecache.py:37(getlines)
        1    0.000    0.000    0.008    0.008 transforms.py:30(<module>)
        1    0.000    0.000    0.002    0.002 category.py:32(CategoricalIndex)
       20    0.000    0.000    0.000    0.000 version.py:307(parse)
       20    0.000    0.000    0.001    0.000 _decorators.py:135(_deprecate_kwarg)
        1    0.000    0.000    0.013    0.013 categorical.py:3(<module>)
       48    0.000    0.000    0.003    0.000 config.py:726(inner)
        3    0.000    0.000    0.000    0.000 cycler.py:349(by_key)
      158    0.000    0.000    0.000    0.000 pyparsing.py:3458(<genexpr>)
    33/10    0.000    0.000    0.022    0.002 pyparsing.py:3809(leaveWhitespace)
        2    0.000    0.000    0.001    0.000 lazy.py:139(__new__)
        1    0.000    0.000    0.000    0.000 twodim_base.py:140(eye)
        2    0.000    0.000    0.005    0.003 numeric.py:1(<module>)
       38    0.000    0.000    0.000    0.000 {method 'strftime' of 'datetime.date' objects}
        3    0.000    0.000    0.000    0.000 generic.py:236(_setup_axes)
       45    0.000    0.000    0.000    0.000 pyparsing.py:1200(setName)
        1    0.000    0.000    0.011    0.011 packers.py:39(<module>)
        1    0.000    0.000    0.000    0.000 ops.py:44(Term)
        1    0.000    0.000    0.003    0.003 series.py:39(SparseSeries)
        1    0.000    0.000    0.022    0.022 window.py:7(<module>)
        1    0.000    0.000    0.001    0.001 merge.py:3(<module>)
        6    0.000    0.000    0.001    0.000 groupby.py:1404(groupby_function)
        1    0.000    0.000    0.013    0.013 panel.py:3(<module>)
        1    0.000    0.000    0.002    0.002 timedeltas.py:1(<module>)
        8    0.000    0.000    0.002    0.000 typing.py:875(__extrahook__)
      115    0.000    0.000    0.000    0.000 expr.py:157(<lambda>)
       23    0.000    0.000    0.001    0.000 rcsetup.py:47(__init__)
        1    0.000    0.000    0.000    0.000 expr.py:252(f)
       98    0.000    0.000    0.000    0.000 pyparsing.py:2181(streamline)
        1    0.000    0.000    0.009    0.009 format.py:5(<module>)
        5    0.000    0.000    0.029    0.006 ops.py:927(add_flex_arithmetic_methods)
       22    0.000    0.000    0.001    0.000 linecache.py:53(checkcache)
      6/1    0.000    0.000    0.001    0.001 copy.py:236(_deepcopy_dict)
       42    0.000    0.000    0.000    0.000 re.py:249(escape)
     23/7    0.000    0.000    0.002    0.000 pyparsing.py:3828(streamline)
       36    0.000    0.000    0.000    0.000 accessor.py:86(_create_delegator_method)
       46    0.000    0.000    0.002    0.000 sre_parse.py:96(closegroup)
        2    0.000    0.000    0.001    0.000 concat.py:3(<module>)
        2    0.000    0.000    0.001    0.000 _decorators.py:184(decorate)
        1    0.000    0.000    0.004    0.004 groupby.py:4584(DataFrameGroupBy)
        3    0.000    0.000    0.003    0.001 _decorators.py:10(deprecate)
       88    0.000    0.000    0.000    0.000 traceback.py:290(walk_stack)
       53    0.000    0.000    0.000    0.000 sre_parse.py:76(__init__)
        4    0.000    0.000    0.000    0.000 {built-in method builtins.sorted}
        1    0.000    0.000    0.001    0.001 _color_data.py:1(<module>)
        1    0.000    0.000    0.001    0.001 period.py:125(PeriodIndex)
      108    0.000    0.000    0.000    0.000 {built-in method _struct.pack}
       37    0.000    0.000    0.000    0.000 codecs.py:318(decode)
       22    0.000    0.000    0.001    0.000 compilerop.py:137(check_linecache_ipython)
       15    0.000    0.000    0.000    0.000 sre_parse.py:266(getuntil)
        1    0.000    0.000    0.003    0.003 stata.py:11(<module>)
        1    0.000    0.000    0.007    0.007 array.py:3(<module>)
        1    0.000    0.000    0.008    0.008 pyparsing.py:5028(_makeTags)
        2    0.000    0.000    0.001    0.000 lazy.py:84(__new__)
       21    0.000    0.000    0.000    0.000 pyparsing.py:2764(__str__)
       28    0.000    0.000    0.001    0.000 pyparsing.py:1079(wrapper)
       27    0.000    0.000    0.000    0.000 rcsetup.py:939(_validate_linestyle)
       37    0.000    0.000    0.000    0.000 typing.py:1019(_abc_negative_cache)
       12    0.000    0.000    0.001    0.000 ops.py:1604(_comp_method_FRAME)
        1    0.000    0.000    0.007    0.007 base.py:3(<module>)
        1    0.000    0.000    0.023    0.023 algorithms.py:4(<module>)
      112    0.000    0.000    0.000    0.000 {built-in method _struct.unpack}
        1    0.000    0.000    0.000    0.000 internals.py:3363(_rebuild_blknos_and_blklocs)
        1    0.000    0.000    0.002    0.002 window.py:1980(EWM)
       16    0.000    0.000    0.001    0.000 offsets.py:59(apply_wraps)
       15    0.000    0.000    0.001    0.000 ops.py:996(_arith_method_SERIES)
        1    0.000    0.000    0.026    0.026 interval.py:1(<module>)
      142    0.000    0.000    0.000    0.000 {method '__contains__' of 'frozenset' objects}
        1    0.000    0.000    0.002    0.002 frame.py:34(SparseDataFrame)
       52    0.000    0.000    0.000    0.000 colors.py:204(<genexpr>)
        1    0.000    0.000    0.001    0.001 categorical.py:170(Categorical)
        1    0.000    0.000    0.008    0.008 fontconfig_pattern.py:65(__init__)
       63    0.000    0.000    0.000    0.000 _weakrefset.py:16(__init__)
       16    0.000    0.000    0.000    0.000 pyparsing.py:2729(parseImpl)
        1    0.000    0.000    0.003    0.003 multi.py:3(<module>)
       93    0.000    0.000    0.000    0.000 sre_parse.py:167(__setitem__)
       12    0.000    0.000    0.005    0.000 pyparsing.py:4012(parseImpl)
       12    0.000    0.000    0.000    0.000 pyparsing.py:1589(resetCache)
        2    0.000    0.000    0.029    0.014 generic.py:8893(_add_series_or_dataframe_operations)
        2    0.000    0.000    0.000    0.000 inspect.py:2730(__init__)
       63    0.000    0.000    0.000    0.000 _weakrefset.py:52(_commit_removals)
       70    0.000    0.000    0.000    0.000 {method '__subclasses__' of 'type' objects}
        1    0.000    0.000    0.001    0.001 ops.py:2(<module>)
       37    0.000    0.000    0.000    0.000 {built-in method _codecs.utf_8_decode}
       49    0.000    0.000    0.000    0.000 pyparsing.py:219(__init__)
       37    0.000    0.000    0.000    0.000 pyparsing.py:421(__getitem__)
       11    0.000    0.000    0.000    0.000 pyparsing.py:4718(_escapeRegexRangeChars)
       48    0.000    0.000    0.000    0.000 rcsetup.py:201(validate_int)
        1    0.000    0.000    0.003    0.003 indexing.py:2(<module>)
        1    0.000    0.000    0.004    0.004 colors.py:61(<module>)
       95    0.000    0.000    0.000    0.000 pyparsing.py:213(<genexpr>)
      112    0.000    0.000    0.000    0.000 ops.py:856(<lambda>)
       58    0.000    0.000    0.000    0.000 {built-in method builtins.id}
        1    0.000    0.000    0.015    0.015 series.py:4(<module>)
       36    0.000    0.000    0.000    0.000 {method 'setter' of 'property' objects}
       16    0.000    0.000    0.001    0.000 pyparsing.py:3606(__init__)
        1    0.000    0.000    0.002    0.002 period.py:2(<module>)
        1    0.000    0.000    0.003    0.003 category.py:1(<module>)
        7    0.000    0.000    0.013    0.002 pyparsing.py:1630(parseString)
       66    0.000    0.000    0.000    0.000 traceback.py:243(__init__)
       44    0.000    0.000    0.000    0.000 pyparsing.py:3647(<genexpr>)
       17    0.000    0.000    0.000    0.000 {built-in method posix.getcwd}
        1    0.000    0.000    0.007    0.007 json.py:2(<module>)
       32    0.000    0.000    0.000    0.000 nanops.py:62(<genexpr>)
       25    0.000    0.000    0.000    0.000 common.py:1835(_get_dtype_type)
       15    0.000    0.000    0.000    0.000 rcsetup.py:288(__call__)
       12    0.000    0.000    0.001    0.000 ops.py:1654(_comp_method_PANEL)
       18    0.000    0.000    0.001    0.000 contextlib.py:129(contextmanager)
        1    0.000    0.000    0.119    0.119 api.py:1(<module>)
       43    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:908(__init__)
        1    0.000    0.000    0.000    0.000 format.py:1038(format_values_with)
       96    0.000    0.000    0.000    0.000 pyparsing.py:1376(postParse)
       31    0.000    0.000    0.000    0.000 pyparsing.py:2453(parseImpl)
       40    0.000    0.000    0.000    0.000 typing.py:1089(__eq__)
        1    0.000    0.000    0.001    0.001 datetimes.py:175(DatetimeIndex)
       62    0.000    0.000    0.000    0.000 lazy.py:96(lazy)
       25    0.000    0.000    0.000    0.000 typing.py:1033(_abc_negative_cache_version)
        1    0.000    0.000    0.000    0.000 path.py:21(Path)
       25    0.000    0.000    0.001    0.000 pyparsing.py:4150(__init__)
       10    0.000    0.000    0.022    0.002 pyparsing.py:4404(__init__)
        1    0.000    0.000    0.001    0.001 {pandas._libs.lib.clean_index_list}
        1    0.000    0.000    0.002    0.002 range.py:1(<module>)
        1    0.000    0.000    0.000    0.000 frequencies.py:2(<module>)
        2    0.000    0.000    0.000    0.000 missing.py:3(<module>)
       13    0.000    0.000    0.000    0.000 pyparsing.py:3847(__str__)
       27    0.000    0.000    0.001    0.000 pyparsing.py:4383(__init__)
       16    0.000    0.000    0.001    0.000 pyparsing.py:1970(__or__)
       13    0.000    0.000    0.000    0.000 printing.py:153(pprint_thing)
        9    0.000    0.000    0.000    0.000 rcsetup.py:906(validate_animation_writer_path)
        1    0.000    0.000    0.015    0.015 cast.py:1(<module>)
       33    0.000    0.000    0.000    0.000 rcsetup.py:667(__call__)
       94    0.000    0.000    0.000    0.000 pyparsing.py:5043(<genexpr>)
        1    0.000    0.000    0.003    0.003 datetimelike.py:4(<module>)
        1    0.000    0.000    0.006    0.006 __init__.py:9(<module>)
        1    0.000    0.000    0.002    0.002 window.py:446(Window)
       18    0.000    0.000    0.000    0.000 rcsetup.py:429(validate_fontsize)
        1    0.000    0.000    0.000    0.000 __init__.py:792(<dictcomp>)
        6    0.000    0.000    0.000    0.000 generic.py:280(set_axis)
       14    0.000    0.000    0.000    0.000 copyreg.py:96(_slotnames)
        1    0.000    0.000    0.000    0.000 pyparsing.py:334(ParseResults)
       10    0.000    0.000    0.002    0.000 pyparsing.py:4092(parseImpl)
        1    0.000    0.000    0.000    0.000 sorting.py:1(<module>)
       15    0.000    0.000    0.000    0.000 config.py:78(_get_single_key)
        9    0.000    0.000    0.000    0.000 os.py:664(__getitem__)
        5    0.000    0.000    0.000    0.000 base.py:4732(_add_numeric_methods_unary)
        1    0.000    0.000    0.002    0.002 groupby.py:563(_GroupBy)
       13    0.000    0.000    0.000    0.000 contextlib.py:59(__init__)
        1    0.000    0.000    0.001    0.001 interval.py:135(IntervalIndex)
       29    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:99(_path_isdir)
        3    0.000    0.000    0.000    0.000 pathlib.py:51(parse_parts)
       21    0.000    0.000    0.000    0.000 ops.py:1756(_arith_method_SPARSE_SERIES)
       74    0.000    0.000    0.000    0.000 pyparsing.py:2074(leaveWhitespace)
       12    0.000    0.000    0.000    0.000 pyparsing.py:4910(<lambda>)
        6    0.000    0.000    0.001    0.000 generic.py:9575(_make_min_count_stat_function)
        1    0.000    0.000    0.017    0.017 pytables.py:1(<module>)
       48    0.000    0.000    0.000    0.000 <string>:12(__new__)
       84    0.000    0.000    0.000    0.000 lazy.py:149(lazy)
       43    0.000    0.000    0.000    0.000 {built-in method _imp.exec_dynamic}
        1    0.000    0.000    0.001    0.001 melt.py:3(<module>)
        1    0.000    0.000    0.002    0.002 __init__.py:25(<module>)
        1    0.000    0.000    0.001    0.001 pivot.py:4(<module>)
        1    0.000    0.000    0.002    0.002 base.py:698(IndexOpsMixin)
       60    0.000    0.000    0.000    0.000 ops.py:891(_wrap_inplace_method)
       23    0.000    0.000    0.000    0.000 base.py:61(is_dtype)
       63    0.000    0.000    0.000    0.000 {method 'remove' of 'set' objects}
        1    0.000    0.000    0.001    0.001 timedeltas.py:99(TimedeltaIndex)
        1    0.000    0.000    0.002    0.002 sql.py:5(<module>)
       14    0.000    0.000    0.000    0.000 pyparsing.py:673(__iadd__)
       51    0.000    0.000    0.000    0.000 {built-in method sys._getframe}
        2    0.000    0.000    0.250    0.125 __init__.py:3(<module>)
       12    0.000    0.000    0.003    0.000 pyparsing.py:4156(parseImpl)
       26    0.000    0.000    0.000    0.000 calendar.py:58(__getitem__)
        5    0.000    0.000    0.000    0.000 getlimits.py:518(min)
       12    0.000    0.000    0.000    0.000 rcsetup.py:68(_listify_validator)
        1    0.000    0.000    0.002    0.002 accessors.py:3(<module>)
        1    0.000    0.000    0.003    0.003 window.py:893(_Rolling_and_Expanding)
        4    0.000    0.000    0.004    0.001 base.py:4757(_add_logical_methods)
        1    0.000    0.000    0.001    0.001 groupby.py:1400(_add_numeric_operations)
        1    0.000    0.000    0.001    0.001 zipfile.py:5(<module>)
        1    0.000    0.000    0.000    0.000 {pandas._libs.lib.to_object_array}
       59    0.000    0.000    0.000    0.000 {method 'isidentifier' of 'str' objects}
       32    0.000    0.000    0.000    0.000 {built-in method builtins.repr}
       11    0.000    0.000    0.000    0.000 sre_compile.py:393(_generate_overlap_table)
        1    0.000    0.000    0.000    0.000 datetimes.py:1(<module>)
       33    0.000    0.000    0.000    0.000 cycler.py:227(<genexpr>)
        1    0.000    0.000    0.000    0.000 cycler.py:41(<module>)
       13    0.000    0.000    0.001    0.000 nanops.py:68(__call__)
        4    0.000    0.000    0.000    0.000 __init__.py:1161(getLogger)
       24    0.000    0.000    0.000    0.000 config.py:695(config_prefix)
        2    0.000    0.000    0.000    0.000 weakref.py:102(__init__)
       29    0.000    0.000    0.000    0.000 {method 'translate' of 'bytearray' objects}
        1    0.000    0.000    0.000    0.000 colors.py:73(__init__)
       18    0.000    0.000    0.007    0.000 rcsetup.py:446(validate_font_properties)
        1    0.000    0.000    0.001    0.001 __init__.py:286(Verbose)
        1    0.000    0.000    0.001    0.001 function.py:19(<module>)
        1    0.000    0.000    0.114    0.114 fontconfig_pattern.py:7(<module>)
        4    0.000    0.000    0.000    0.000 _decorators.py:308(indent)
       52    0.000    0.000    0.000    0.000 _collections_abc.py:392(__subclasshook__)
        1    0.000    0.000    0.001    0.001 html.py:4(<module>)
        1    0.000    0.000    0.001    0.001 align.py:2(<module>)
       16    0.000    0.000    0.001    0.000 pyparsing.py:3642(__str__)
        1    0.000    0.000    0.001    0.001 generic.py:1(<module>)
        8    0.000    0.000    0.000    0.000 typing.py:1025(_abc_negative_cache)
        1    0.000    0.000    0.001    0.001 path.py:10(<module>)
       22    0.000    0.000    0.000    0.000 traceback.py:273(__getitem__)
        1    0.000    0.000    0.000    0.000 plotting.py:1(<module>)
     18/4    0.000    0.000    0.004    0.001 pyparsing.py:3803(parseImpl)
        1    0.000    0.000    6.083    6.083 interactiveshell.py:2637(safe_execfile)
     12/2    0.000    0.000    0.001    0.000 pyparsing.py:1948(makeOptionalList)
       26    0.000    0.000    0.000    0.000 pyparsing.py:516(haskeys)
        1    0.000    0.000    0.000    0.000 internals.py:4880(form_blocks)
       40    0.000    0.000    0.000    0.000 pyparsing.py:468(__bool__)
        1    0.000    0.000    0.000    0.000 pytables.py:289(JointFilterBinOp)
        2    0.000    0.000    0.000    0.000 __init__.py:175(compare_versions)
        1    0.000    0.000    0.001    0.001 range.py:28(RangeIndex)
       26    0.000    0.000    0.000    0.000 ops.py:1795(_arith_method_SPARSE_ARRAY)
        1    0.000    0.000    0.006    0.006 config.py:49(<module>)
       50    0.000    0.000    0.000    0.000 __init__.py:262(u_safe)
       26    0.000    0.000    0.000    0.000 {built-in method builtins.next}
        1    0.000    0.000    0.005    0.005 engines.py:3(<module>)
        1    0.000    0.000    0.031    0.031 __init__.py:1056(rc_params_from_file)
        1    0.000    0.000    0.001    0.001 numeric.py:292(Float64Index)
        1    0.000    0.000    0.000    0.000 offsets.py:2335(Day)
        3    0.000    0.000    0.000    0.000 socket.py:334(send)
       13    0.000    0.000    0.000    0.000 printing.py:182(as_escaped_unicode)
        6    0.000    0.000    0.000    0.000 nanops.py:95(__call__)
       15    0.000    0.000    0.000    0.000 config.py:96(_get_option)
        1    0.000    0.000    0.000    0.000 transforms.py:244(BboxBase)
       15    0.000    0.000    0.000    0.000 pyparsing.py:4170(__str__)
       20    0.000    0.000    0.001    0.000 version.py:302(__init__)
       32    0.000    0.000    0.000    0.000 {built-in method builtins.iter}
        1    0.000    0.000    0.003    0.003 linecache.py:82(updatecache)
       13    0.000    0.000    0.000    0.000 contextlib.py:85(__exit__)
       10    0.000    0.000    0.000    0.000 pyparsing.py:4004(__init__)
       24    0.000    0.000    0.000    0.000 deprecation.py:115(deprecated)
        5    0.000    0.000    0.000    0.000 {method 'translate' of 'str' objects}
        7    0.000    0.000    0.000    0.000 rrule.py:70(__init__)
        5    0.000    0.000    0.007    0.001 fontconfig_pattern.py:113(parse)
        5    0.000    0.000    0.000    0.000 textwrap.py:160(_split)
        6    0.000    0.000    0.000    0.000 copy.py:252(_keep_alive)
        5    0.000    0.000    0.000    0.000 _depr_module.py:26(__init__)
        1    0.000    0.000    0.000    0.000 transforms.py:752(Bbox)
        1    0.000    0.000    0.009    0.009 dtypes.py:1(<module>)
        4    0.000    0.000    0.000    0.000 posixpath.py:75(join)
        1    0.000    0.000    0.000    0.000 scope.py:3(<module>)
        2    0.000    0.000    0.000    0.000 __init__.py:961(_open_file_or_url)
        1    0.000    0.000    0.000    0.000 common.py:3(<module>)
        8    0.000    0.000    0.000    0.000 typing.py:1039(_abc_negative_cache_version)
       20    0.000    0.000    0.000    0.000 {built-in method _imp.is_builtin}
        1    0.000    0.000    0.001    0.001 _core.py:2911(FramePlotMethods)
       39    0.000    0.000    0.000    0.000 copy.py:190(_deepcopy_atomic)
       44    0.000    0.000    0.000    0.000 pyparsing.py:3609(<genexpr>)
        6    0.000    0.000    0.001    0.000 pyparsing.py:4934(<lambda>)
        1    0.000    0.000    0.003    0.003 _misc.py:3(<module>)
        1    0.000    0.000    0.000    0.000 internals.py:88(Block)
        6    0.000    0.000    0.000    0.000 deprecation.py:54(warn_deprecated)
        1    0.000    0.000    0.000    0.000 internals.py:3210(BlockManager)
        8    0.000    0.000    0.000    0.000 datetimelike.py:280(_join_i8_wrapper)
       11    0.000    0.000    0.000    0.000 re.py:324(_subx)
        5    0.000    0.000    0.000    0.000 pyparsing.py:4098(__str__)
        3    0.000    0.000    0.000    0.000 rcsetup.py:178(validate_axisbelow)
        3    0.000    0.000    0.000    0.000 missing.py:189(_isna_ndarraylike)
       20    0.000    0.000    0.000    0.000 _decorators.py:66(deprecate_kwarg)
        6    0.000    0.000    0.000    0.000 warnings.py:449(__enter__)
        5    0.000    0.000    0.000    0.000 warnings.py:159(_add_filter)
        1    0.000    0.000    0.001    0.001 __init__.py:783(RcParams)
        4    0.000    0.000    0.000    0.000 generic.py:9003(_doc_parms)
       12    0.000    0.000    0.000    0.000 period.py:53(_field_accessor)
        1    0.000    0.000    0.000    0.000 offsets.py:2340(Hour)
       25    0.000    0.000    0.000    0.000 __init__.py:422(<genexpr>)
        1    0.000    0.000    0.000    0.000 pickle_compat.py:3(<module>)
       22    0.000    0.000    0.000    0.000 expr.py:227(_op_maker)
        3    0.000    0.000    0.000    0.000 excel.py:245(register_writer)
        6    0.000    0.000    0.000    0.000 rcsetup.py:221(validate_fonttype)
        2    0.000    0.000    0.000    0.000 fontconfig_pattern.py:152(_property)
        1    0.000    0.000    0.001    0.001 format.py:1023(get_result_as_array)
       15    0.000    0.000    0.000    0.000 config.py:550(_get_root)
        1    0.000    0.000    0.000    0.000 {built-in method numpy.core.multiarray.putmask}
        1    0.000    0.000    0.000    0.000 tile.py:3(<module>)
        1    0.000    0.000    0.002    0.002 frame.py:334(__init__)
        1    0.000    0.000    0.000    0.000 units.py:43(<module>)
        5    0.000    0.000    0.000    0.000 fontconfig_pattern.py:132(_family)
        2    0.000    0.000    0.000    0.000 internals.py:237(mgr_locs)
       20    0.000    0.000    0.000    0.000 ops.py:1638(_arith_method_PANEL)
       32    0.000    0.000    0.000    0.000 offsets.py:2455(<genexpr>)
       25    0.000    0.000    0.000    0.000 __init__.py:420(<genexpr>)
        1    0.000    0.000    0.000    0.000 base.py:255(__new__)
      4/2    0.000    0.000    0.000    0.000 common.py:1879(_get_dtype_from_object)
       13    0.000    0.000    0.000    0.000 contextlib.py:157(helper)
        8    0.000    0.000    0.000    0.000 {method 'rfind' of 'str' objects}
        1    0.000    0.000    0.000    0.000 groupby.py:3950(NDFrameGroupBy)
        1    0.000    0.000    0.001    0.001 nanops.py:342(nanmean)
        1    0.000    0.000    6.083    6.083 abc.py:1(<module>)
       12    0.000    0.000    0.000    0.000 getlimits.py:532(max)
        4    0.000    0.000    0.000    0.000 pathlib.py:385(wrapped)
        1    0.000    0.000    0.008    0.008 eval.py:4(<module>)
        1    0.000    0.000    0.000    0.000 cycler.py:468(cycler)
       14    0.000    0.000    0.000    0.000 pyparsing.py:2773(charsAsStr)
        9    0.000    0.000    0.000    0.000 __init__.py:870(__getitem__)
        1    0.000    0.000    0.002    0.002 csv.py:4(<module>)
        5    0.000    0.000    0.000    0.000 textwrap.py:146(_munge_whitespace)
       12    0.000    0.000    0.000    0.000 rcsetup.py:300(<listcomp>)
        1    0.000    0.000    0.000    0.000 dtypes.py:86(CategoricalDtype)
       30    0.000    0.000    0.000    0.000 config.py:565(_get_deprecated_option)
       13    0.000    0.000    0.000    0.000 inference.py:447(is_sequence)
       20    0.000    0.000    0.000    0.000 version.py:312(<listcomp>)
        2    0.000    0.000    0.000    0.000 __init__.py:302(loads)
        3    0.000    0.000    0.000    0.000 functools.py:479(decorating_function)
        1    0.000    0.000    0.000    0.000 plotting.py:9(<listcomp>)
        1    0.000    0.000    0.001    0.001 __init__.py:602(_get_config_or_cache_dir)
        6    0.000    0.000    0.000    0.000 base.py:89(_make_comparison_op)
       13    0.000    0.000    0.000    0.000 version.py:331(_cmp)
       14    0.000    0.000    0.000    0.000 calendar.py:77(__getitem__)
        1    0.000    0.000    0.000    0.000 sre_parse.py:876(parse_template)
        9    0.000    0.000    0.000    0.000 os.py:742(encode)
        1    0.000    0.000    0.002    0.002 dates.py:581(DateFormatter)
        1    0.000    0.000    0.000    0.000 cycler.py:77(Cycler)
       15    0.000    0.000    0.000    0.000 cycler.py:138(keys)
        1    0.000    0.000    0.000    0.000 datetimelike.py:246(DatetimeIndexOpsMixin)
        5    0.000    0.000    0.001    0.000 base.py:4752(_add_numeric_methods)
        1    0.000    0.000    0.000    0.000 {built-in method pandas._libs.lib.is_datetime_array}
        1    0.000    0.000    0.000    0.000 lazy.py:172(<listcomp>)
        2    0.000    0.000    0.000    0.000 pathlib.py:631(_parse_args)
        7    0.000    0.000    0.002    0.000 abc.py:151(register)
        1    0.000    0.000    0.000    0.000 stata.py:959(StataReader)
        3    0.000    0.000    0.000    0.000 pytables.py:1506(__init__)
        3    0.000    0.000    0.000    0.000 rcsetup.py:468(validate_whiskers)
        1    0.000    0.000    0.000    0.000 base.py:4614(_add_comparison_methods)
        7    0.000    0.000    0.000    0.000 base.py:4862(_add_logical_methods_disabled)
        4    0.000    0.000    0.000    0.000 function_base.py:1912(__init__)
        3    0.000    0.000    0.002    0.001 textwrap.py:368(wrap)
        1    0.000    0.000    0.000    0.000 parquet.py:1(<module>)
        1    0.000    0.000    0.005    0.005 table_schema.py:5(<module>)
        1    0.000    0.000    0.001    0.001 window.py:60(_Window)
        1    0.000    0.000    0.000    0.000 {built-in method posix.mkdir}
        1    0.000    0.000    0.000    0.000 _compat.py:3(<module>)
        1    0.000    0.000    0.000    0.000 numeric.py:215(UInt64Index)
        6    0.000    0.000    0.000    0.000 category.py:798(_make_compare)
       36    0.000    0.000    0.000    0.000 config.py:725(wrap)
        1    0.000    0.000    6.083    6.083 py3compat.py:184(execfile)
        5    0.000    0.000    0.003    0.001 textwrap.py:342(wrap)
        6    0.000    0.000    0.000    0.000 warnings.py:468(__exit__)
        1    0.000    0.000    0.000    0.000 {method 'readline' of '_io.BufferedReader' objects}
        1    0.000    0.000    0.000    0.000 pytables.py:2982(Table)
        1    0.000    0.000    0.000    0.000 excel.py:788(ExcelWriter)
        1    0.000    0.000    0.000    0.000 transforms.py:1718(AffineBase)
        1    0.000    0.000    0.000    0.000 colors.py:632(LinearSegmentedColormap)
        2    0.000    0.000    0.006    0.003 pyparsing.py:4916(srange)
        1    0.000    0.000    0.001    0.001 _style.py:3(<module>)
        1    0.000    0.000    0.000    0.000 internals.py:2293(ObjectBlock)
        5    0.000    0.000    0.000    0.000 generic.py:2456(_create_indexer)
        1    0.000    0.000    0.000    0.000 base.py:174(SelectionMixin)
        7    0.000    0.000    0.000    0.000 genericpath.py:16(exists)
        6    0.000    0.000    0.000    0.000 ops.py:1126(_comp_method_SERIES)
        1    0.000    0.000    0.001    0.001 groupby.py:2768(BinGrouper)
        5    0.000    0.000    0.004    0.001 config.py:769(is_instance_factory)
        1    0.000    0.000    0.000    0.000 lazy.py:118(<listcomp>)
        5    0.000    0.000    0.000    0.000 textwrap.py:179(<listcomp>)
        1    0.000    0.000    0.000    0.000 scipy_sparse.py:5(<module>)
        4    0.000    0.000    0.001    0.000 __init__.py:734(gen_candidates)
        1    0.000    0.000    0.000    0.000 offsets.py:2355(Milli)
        2    0.000    0.000    0.000    0.000 base.py:4624(_add_numeric_methods_add_sub_disabled)
       43    0.000    0.000    0.000    0.000 {built-in method builtins.callable}
       25    0.000    0.000    0.000    0.000 {method 'end' of '_sre.SRE_Match' objects}
        1    0.000    0.000    0.000    0.000 pyparsing.py:1113(ParserElement)
        5    0.000    0.000    0.000    0.000 fontconfig_pattern.py:144(_families)
        1    0.000    0.000    0.000    0.000 _core.py:2712(SeriesPlotMethods)
        1    0.000    0.000    0.000    0.000 {pandas._libs.tslibs.offsets._get_calendar}
        7    0.000    0.000    0.000    0.000 {pandas._libs.tslibs.offsets.apply_index_wraps}
        1    0.000    0.000    0.001    0.001 accessor.py:7(<module>)
       15    0.000    0.000    0.000    0.000 config.py:226(__call__)
        1    0.000    0.000    0.000    0.000 __init__.py:389(wrapper)
        3    0.000    0.000    0.000    0.000 __init__.py:1212(_fixupParents)
        1    0.000    0.000    0.000    0.000 tokenize.py:448(open)
       22    0.000    0.000    0.000    0.000 {method 'group' of '_sre.SRE_Match' objects}
       21    0.000    0.000    0.000    0.000 {method 'groupdict' of '_sre.SRE_Match' objects}
        8    0.000    0.000    0.000    0.000 rrule.py:77(<genexpr>)
        2    0.000    0.000    0.001    0.000 pyparsing.py:4603(delimitedList)
        3    0.000    0.000    0.000    0.000 rcsetup.py:921(validate_webagg_address)
        8    0.000    0.000    0.000    0.000 __init__.py:390(_logged_cached)
        4    0.000    0.000    0.041    0.010 __init__.py:826(__init__)
        1    0.000    0.000    0.000    0.000 offsets.py:2345(Minute)
        1    0.000    0.000    0.000    0.000 offsets.py:2350(Second)
        1    0.000    0.000    0.124    0.124 index.py:2(<module>)
        2    0.000    0.000    0.000    0.000 cast.py:971(maybe_cast_to_datetime)
        4    0.000    0.000    0.000    0.000 _collections_abc.py:664(__contains__)
        2    0.000    0.000    0.000    0.000 _version.py:7(<module>)
        7    0.000    0.000    0.000    0.000 pyparsing.py:4978(tokenMap)
        6    0.000    0.000    0.001    0.000 pyparsing.py:1214(setResultsName)
        9    0.000    0.000    0.000    0.000 pyparsing.py:3994(__str__)
        1    0.000    0.000    0.001    0.001 datetimelike.py:99(TimelikeOps)
       21    0.000    0.000    0.000    0.000 datetimes.py:63(_field_accessor)
        1    0.000    0.000    0.000    0.000 offsets.py:2365(Nano)
        1    0.000    0.000    0.001    0.001 nanops.py:69(_f)
       12    0.000    0.000    0.000    0.000 typing.py:889(__extrahook__)
        2    0.000    0.000    0.002    0.001 textwrap.py:381(fill)
        7    0.000    0.000    0.000    0.000 __init__.py:1286(debug)
        7    0.000    0.000    0.000    0.000 __init__.py:1544(isEnabledFor)
        4    0.000    0.000    0.000    0.000 pyparsing.py:430(__setitem__)
        1    0.000    0.000    0.001    0.001 __init__.py:703(matplotlib_fname)
        1    0.000    0.000    0.000    0.000 _tools.py:3(<module>)
        6    0.000    0.000    0.000    0.000 period.py:77(_period_index_cmp)
       20    0.000    0.000    0.000    0.000 ops.py:945(<genexpr>)
        1    0.000    0.000    0.000    0.000 offsets.py:2360(Micro)
        1    0.000    0.000    0.000    0.000 offsets.py:521(__init__)
        3    0.000    0.000    0.000    0.000 __init__.py:1268(__init__)
       30    0.000    0.000    0.000    0.000 {method 'reverse' of 'list' objects}
        1    0.000    0.000    0.000    0.000 series.py:365(_set_axis)
       12    0.000    0.000    0.000    0.000 ops.py:1296(_bool_method_SERIES)
        1    0.000    0.000    0.000    0.000 offsets.py:1319(Week)
       20    0.000    0.000    0.000    0.000 base.py:4736(_make_evaluate_unary)
       11    0.000    0.000    0.000    0.000 version.py:51(__lt__)
        3    0.000    0.000    0.000    0.000 iostream.py:195(schedule)
        3    0.000    0.000    0.000    0.000 posixpath.py:154(dirname)
       15    0.000    0.000    0.000    0.000 {built-in method sys.intern}
        4    0.000    0.000    0.000    0.000 pyparsing.py:1863(__radd__)
        3    0.000    0.000    0.001    0.000 pyparsing.py:1899(__mul__)
        1    0.000    0.000    0.000    0.000 numeric.py:157(Int64Index)
       18    0.000    0.000    0.000    0.000 strings.py:1874(do_copy)
        5    0.000    0.000    0.000    0.000 missing.py:112(_isna_new)
        1    0.000    0.000    0.000    0.000 common.py:21(<listcomp>)
        1    0.000    0.000    0.000    0.000 numeric.py:2171(identity)
       14    0.000    0.000    0.000    0.000 {method 'get' of 'mappingproxy' objects}
        1    0.000    0.000    0.001    0.001 pickle.py:1(<module>)
        7    0.000    0.000    0.000    0.000 pyparsing.py:3164(__init__)
        3    0.000    0.000    0.000    0.000 series.py:4019(_sanitize_array)
        1    0.000    0.000    0.000    0.000 range.py:574(_add_numeric_methods_binary)
       15    0.000    0.000    0.000    0.000 strings.py:1835(_noarg_wrapper)
       15    0.000    0.000    0.000    0.000 config.py:606(_warn_if_deprecated)
        7    0.000    0.000    0.000    0.000 config.py:799(<listcomp>)
        1    0.000    0.000    0.000    0.000 tzinfo.py:1(<module>)
        5    0.000    0.000    0.000    0.000 textwrap.py:115(__init__)
        5    0.000    0.000    0.000    0.000 warnings.py:143(simplefilter)
        1    0.000    0.000    0.000    0.000 pytables.py:419(HDFStore)
        5    0.000    0.000    0.000    0.000 pyparsing.py:3269(parseImpl)
        7    0.000    0.000    0.000    0.000 pyparsing.py:4088(__init__)
        3    0.000    0.000    0.000    0.000 rcsetup.py:268(validate_toolbar)
        1    0.000    0.000    0.002    0.002 series.py:1221(__unicode__)
        1    0.000    0.000    0.000    0.000 frozen.py:9(<module>)
        1    0.000    0.000    0.000    0.000 groupby.py:2173(BaseGrouper)
        1    0.000    0.000    0.001    0.001 _decorators.py:1(<module>)
        7    0.000    0.000    0.000    0.000 config.py:802(inner)
        7    0.000    0.000    0.000    0.000 config.py:797(is_one_of_factory)
       13    0.000    0.000    0.000    0.000 contextlib.py:79(__enter__)
        1    0.000    0.000    0.000    0.000 transforms.py:2084(IdentityTransform)
        1    0.000    0.000    0.000    0.000 cycler.py:191(_from_iter)
        1    0.000    0.000    0.000    0.000 gzip.py:4(<module>)
        1    0.000    0.000    0.000    0.000 numeric.py:26(NumericIndex)
        1    0.000    0.000    0.000    0.000 timedeltas.py:182(_add_comparison_methods)
       24    0.000    0.000    0.000    0.000 config.py:761(inner)
        7    0.000    0.000    0.000    0.000 config.py:800(<listcomp>)
        4    0.000    0.000    0.000    0.000 __init__.py:1838(getLogger)
        1    0.000    0.000    0.000    0.000 parsers.py:713(TextFileReader)
        4    0.000    0.000    0.000    0.000 pyparsing.py:3982(__init__)
        1    0.000    0.000    0.000    0.000 format.py:169(__init__)
        1    0.000    0.000    0.000    0.000 category.py:794(_add_comparison_methods)
       23    0.000    0.000    0.000    0.000 function.py:31(__init__)
       12    0.000    0.000    0.000    0.000 {built-in method pandas._libs.lib.is_scalar}
        5    0.000    0.000    0.001    0.000 textwrap.py:336(_split_chunks)
        2    0.000    0.000    0.000    0.000 decoder.py:334(decode)
        2    0.000    0.000    0.000    0.000 weakref.py:288(update)
        5    0.000    0.000    0.000    0.000 _collections_abc.py:657(get)
        6    0.000    0.000    0.000    0.000 {method 'remove' of 'list' objects}
       14    0.000    0.000    0.000    0.000 {method 'lstrip' of 'str' objects}
        1    0.000    0.000    0.000    0.000 expr.py:717(Expr)
        1    0.000    0.000    0.000    0.000 transforms.py:2344(CompositeGenericTransform)
        3    0.000    0.000    0.000    0.000 cycler.py:225(__iter__)
        1    0.000    0.000    0.001    0.001 pyparsing.py:5864(pyparsing_unicode)
        4    0.000    0.000    0.000    0.000 timedeltas.py:163(_join_i8_wrapper)
        7    0.000    0.000    0.001    0.000 base.py:4914(_ensure_index)
        9    0.000    0.000    0.000    0.000 common.py:513(is_categorical_dtype)
       12    0.000    0.000    0.000    0.000 {method 'expandtabs' of 'str' objects}
        1    0.000    0.000    0.000    0.000 {method 'read' of '_io.BufferedReader' objects}
        1    0.000    0.000    0.000    0.000 pytables.py:3805(LegacyTable)
       11    0.000    0.000    0.000    0.000 _compat.py:9(_mpl_version)
        1    0.000    0.000    0.000    0.000 format.py:206(_get_footer)
        1    0.000    0.000    0.000    0.000 timedeltas.py:3(<module>)
       15    0.000    0.000    0.000    0.000 _decorators.py:60(<genexpr>)
        1    0.000    0.000    0.000    0.000 cast.py:1207(construct_1d_object_array_from_listlike)
       15    0.000    0.000    0.000    0.000 config.py:593(_translate_key)
        1    0.000    0.000    0.000    0.000 printing.py:12(adjoin)
        4    0.000    0.000    0.000    0.000 numeric.py:2491(seterr)
        6    0.000    0.000    0.000    0.000 warnings.py:428(__init__)
        1    0.000    0.000    0.000    0.000 normalize.py:4(<module>)
        1    0.000    0.000    0.000    0.000 pytables.py:2242(Fixed)
        1    0.000    0.000    0.000    0.000 transforms.py:1095(LockableBbox)
        3    0.000    0.000    0.000    0.000 rcsetup.py:312(__call__)
        1    0.000    0.000    0.000    0.000 indexing.py:88(_NDFrameIndexer)
        1    0.000    0.000    0.000    0.000 period.py:226(_add_comparison_methods)
        1    0.000    0.000    0.000    0.000 {pandas._libs.missing.isnaobj}
       15    0.000    0.000    0.000    0.000 config.py:532(_select_options)
        1    0.000    0.000    0.000    0.000 exceptions.py:3(<module>)
        1    0.000    0.000    0.001    0.001 lazy.py:1(<module>)
       19    0.000    0.000    0.000    0.000 inspect.py:2500(name)
        8    0.000    0.000    0.000    0.000 posixpath.py:41(_get_sep)
       10    0.000    0.000    0.000    0.000 {method 'encode' of 'str' objects}
        1    0.000    0.000    0.000    0.000 pytables.py:2409(GenericFixed)
        1    0.000    0.000    0.000    0.000 parsers.py:1234(ParserBase)
        1    0.000    0.000    0.000    0.000 transforms.py:1224(Transform)
        1    0.000    0.000    0.000    0.000 fontconfig_pattern.py:30(FontconfigPatternParser)
        1    0.000    0.000    0.000    0.000 series.py:166(__init__)
        1    0.000    0.000    0.000    0.000 frame.py:7547(_list_to_arrays)
        1    0.000    0.000    0.001    0.001 hashing.py:3(<module>)
        1    0.000    0.000    0.000    0.000 base.py:16(ExtensionArray)
        7    0.000    0.000    0.000    0.000 common.py:369(is_datetime64tz_dtype)
        1    0.000    0.000    0.000    0.000 inference.py:1(<module>)
        1    0.000    0.000    0.000    0.000 posixpath.py:331(normpath)
        1    0.000    0.000    0.001    0.001 pytables.py:4334(AppendableMultiFrameTable)
        6    0.000    0.000    0.000    0.000 timedeltas.py:56(_td_index_cmp)
       18    0.000    0.000    0.000    0.000 strings.py:1871(copy)
        1    0.000    0.000    0.000    0.000 <string>:5(Components)
        2    0.000    0.000    0.000    0.000 iostream.py:382(write)
        3    0.000    0.000    0.000    0.000 pathlib.py:1343(is_dir)
       19    0.000    0.000    0.000    0.000 inspect.py:2504(default)
        2    0.000    0.000    0.000    0.000 six.py:91(__get__)
        7    0.000    0.000    0.000    0.000 __init__.py:1530(getEffectiveLevel)
        3    0.000    0.000    0.000    0.000 codecs.py:308(__init__)
       17    0.000    0.000    0.000    0.000 {built-in method _warnings._filters_mutated}
        1    0.000    0.000    0.000    0.000 pytables.py:2826(SparseSeriesFixed)
        1    0.000    0.000    0.000    0.000 pytables.py:1488(IndexCol)
        1    0.000    0.000    0.000    0.000 pytables.py:1829(__init__)
        1    0.000    0.000    0.000    0.000 clipboards.py:1(<module>)
        1    0.000    0.000    0.000    0.000 window.py:772(_GroupByMixin)
        1    0.000    0.000    0.000    0.000 transforms.py:2131(BlendedGenericTransform)
        3    0.000    0.000    0.000    0.000 pyparsing.py:3230(__init__)
        3    0.000    0.000    0.000    0.000 rcsetup.py:607(validate_hinting)
        2    0.000    0.000    0.000    0.000 pyparsing.py:678(<listcomp>)
        1    0.000    0.000    0.000    0.000 internals.py:4631(SingleBlockManager)
        4    0.000    0.000    0.000    0.000 generic.py:9005(<listcomp>)
        1    0.000    0.000    0.000    0.000 indexing.py:1361(_IXIndexer)
        3    0.000    0.000    0.000    0.000 datetimelike.py:817(_add_datetimelike_methods)
        1    0.000    0.000    0.002    0.002 dtypes.py:383(DatetimeTZDtype)
        1    0.000    0.000    0.002    0.002 printing.py:3(<module>)
        1    0.000    0.000    0.000    0.000 {built-in method numpy.core.multiarray.zeros}
        7    0.000    0.000    0.000    0.000 _common.py:9(__init__)
        2    0.000    0.000    0.000    0.000 pathlib.py:998(__new__)
        2    0.000    0.000    0.000    0.000 parsers.py:538(_make_parser_function)
        1    0.000    0.000    0.000    0.000 util.py:1(<module>)
        8    0.000    0.000    0.000    0.000 rcsetup.py:284(__init__)
        3    0.000    0.000    0.000    0.000 rcsetup.py:890(validate_hist_bins)
        7    0.000    0.000    0.000    0.000 pyparsing.py:470(__iter__)
        3    0.000    0.000    0.000    0.000 nanops.py:64(check)
        5    0.000    0.000    0.000    0.000 dtypes.py:707(is_dtype)
        1    0.000    0.000    0.000    0.000 {built-in method pandas._libs.lib.is_bool}
        7    0.000    0.000    0.000    0.000 pathlib.py:691(__str__)
       19    0.000    0.000    0.000    0.000 inspect.py:2512(kind)
        2    0.000    0.000    0.000    0.000 decoder.py:345(raw_decode)
        2    0.000    0.000    0.000    0.000 util.py:73(compose)
        1    0.000    0.000    0.000    0.000 transforms.py:1621(TransformWrapper)
        3    0.000    0.000    0.000    0.000 rcsetup.py:108(validate_path_exists)
        4    0.000    0.000    0.000    0.000 fontconfig_pattern.py:138(_name)
        1    0.000    0.000    0.000    0.000 internals.py:2990(SparseBlock)
        7    0.000    0.000    0.000    0.000 offsets.py:2219(__init__)
        1    0.000    0.000    0.000    0.000 offsets.py:616(BusinessHourMixin)
        1    0.000    0.000    0.000    0.000 nanops.py:202(_get_values)
        4    0.000    0.000    0.000    0.000 function_base.py:241(iterable)
        2    0.000    0.000    0.000    0.000 pathlib.py:651(_from_parts)
        5    0.000    0.000    0.000    0.000 {built-in method _struct.calcsize}
        1    0.000    0.000    0.000    0.000 feather_format.py:1(<module>)
        1    0.000    0.000    0.000    0.000 _print_versions.py:1(<module>)
        5    0.000    0.000    0.000    0.000 pyparsing.py:3098(__str__)
        8    0.000    0.000    0.001    0.000 pyparsing.py:4936(<genexpr>)
       16    0.000    0.000    0.000    0.000 pyparsing.py:327(__getitem__)
        1    0.000    0.000    0.001    0.001 _core.py:1558(BoxPlot)
        3    0.000    0.000    0.000    0.000 {built-in method _csv.register_dialect}
        6    0.000    0.000    0.000    0.000 datetimes.py:103(_dt_index_cmp)
        1    0.000    0.000    0.000    0.000 datetimes.py:292(_add_comparison_methods)
        1    0.000    0.000    0.000    0.000 category.py:849(_add_accessors)
        3    0.000    0.000    0.000    0.000 common.py:1688(is_extension_array_dtype)
        4    0.000    0.000    0.000    0.000 __init__.py:205(iteritems)
        4    0.000    0.000    0.000    0.000 pathlib.py:282(splitroot)
        1    0.000    0.000    0.000    0.000 shutil.py:1048(get_terminal_size)
        4    0.000    0.000    0.000    0.000 __init__.py:219(_acquireLock)
        3    0.000    0.000    0.000    0.000 threading.py:1062(_wait_for_tstate_lock)
        1    0.000    0.000    0.000    0.000 _depr_module.py:4(<module>)
       11    0.000    0.000    0.000    0.000 rcsetup.py:661(__init__)
        1    0.000    0.000    0.000    0.000 _converter.py:65(register)
        1    0.000    0.000    0.001    0.001 format.py:861(format_array)
        1    0.000    0.000    0.001    0.001 frame.py:6845(_reduce)
        1    0.000    0.000    0.000    0.000 offsets.py:513(BusinessDay)
        1    0.000    0.000    0.000    0.000 offsets.py:1110(SemiMonthOffset)
       11    0.000    0.000    0.000    0.000 base.py:641(__len__)
        6    0.000    0.000    0.000    0.000 accessor.py:125(__init__)
        1    0.000    0.000    0.000    0.000 dtypes.py:11(PandasExtensionDtype)
        2    0.000    0.000    0.000    0.000 dtypes.py:401(__new__)
        5    0.000    0.000    0.000    0.000 {method 'any' of 'numpy.ndarray' objects}
        5    0.000    0.000    0.000    0.000 {built-in method numpy.core.multiarray.empty}
        1    0.000    0.000    0.000    0.000 tokenize.py:385(find_cookie)
        4    0.000    0.000    0.000    0.000 __init__.py:228(_releaseLock)
        5    0.000    0.000    0.000    0.000 {method 'keys' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 window.py:1682(RollingGroupby)
        1    0.000    0.000    0.000    0.000 transforms.py:2253(BlendedAffine2D)
        4    0.000    0.000    0.000    0.000 pyparsing.py:2042(__invert__)
        2    0.000    0.000    0.001    0.000 __init__.py:409(wrapper)
        1    0.000    0.000    0.000    0.000 __init__.py:656(_get_data_path)
        1    0.000    0.000    0.000    0.000 internals.py:3265(__init__)
        1    0.000    0.000    0.000    0.000 frame.py:7644(_homogenize)
        1    0.000    0.000    0.000    0.000 groupby.py:2911(Grouping)
        6    0.000    0.000    0.000    0.000 common.py:477(is_interval_dtype)
        2    0.000    0.000    0.000    0.000 posixpath.py:232(expanduser)
       12    0.000    0.000    0.000    0.000 {method 'clear' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 date_converters.py:1(<module>)
        1    0.000    0.000    0.000    0.000 excel.py:332(ExcelFile)
        1    0.000    0.000    0.000    0.000 colors.py:1186(PowerNorm)
       11    0.000    0.000    0.000    0.000 _color_data.py:31(<genexpr>)
        9    0.000    0.000    0.000    0.000 rcsetup.py:336(validate_color_or_auto)
        2    0.000    0.000    0.000    0.000 format.py:997(base_formatter)
        2    0.000    0.000    0.000    0.000 internals.py:116(__init__)
        1    0.000    0.000    0.000    0.000 internals.py:2545(CategoricalBlock)
        4    0.000    0.000    0.000    0.000 timedeltas.py:42(_field_accessor)
        1    0.000    0.000    0.000    0.000 offsets.py:1777(FY5253)
        1    0.000    0.000    0.000    0.000 base.py:2382(_format_with_header)
        2    0.000    0.000    0.000    0.000 common.py:1936(_validate_date_like_dtype)
        3    0.000    0.000    0.000    0.000 {built-in method _socket.inet_aton}
        1    0.000    0.000    0.000    0.000 pytables.py:3930(AppendableTable)
        1    0.000    0.000    0.000    0.000 pytables.py:4227(AppendableSeriesTable)
        1    0.000    0.000    0.000    0.000 pytables.py:1790(DataCol)
        1    0.000    0.000    0.000    0.000 transforms.py:1019(TransformedBbox)
        1    0.000    0.000    0.000    0.000 <string>:5(Boxplot)
        2    0.000    0.000    0.000    0.000 pyparsing.py:1298(addParseAction)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3127(__init__)
        3    0.000    0.000    0.000    0.000 pyparsing.py:4445(__init__)
        2    0.000    0.000    0.000    0.000 format.py:1404(<genexpr>)
        1    0.000    0.000    0.000    0.000 internals.py:5017(_stack_arrays)
        1    0.000    0.000    0.559    0.559 __init__.py:2(<module>)
        1    0.000    0.000    0.001    0.001 dtypes.py:497(PeriodDtype)
        1    0.000    0.000    0.000    0.000 {built-in method pandas._libs.algos.ensure_object}
        1    0.000    0.000    0.000    0.000 pathlib.py:814(with_name)
        1    0.000    0.000    0.000    0.000 exceptions.py:1(<module>)
        1    0.000    0.000    0.000    0.000 ticker.py:1810(MaxNLocator)
        4    0.000    0.000    0.000    0.000 pyparsing.py:3081(parseImpl)
        1    0.000    0.000    0.001    0.001 format.py:262(to_string)
        1    0.000    0.000    0.000    0.000 internals.py:4986(_multi_blockify)
        1    0.000    0.000    0.000    0.000 offsets.py:1513(LastWeekOfMonth)
        1    0.000    0.000    0.000    0.000 offsets.py:1968(FY5253Quarter)
        1    0.000    0.000    0.001    0.001 dtypes.py:614(IntervalDtype)
        1    0.000    0.000    0.000    0.000 numeric.py:630(require)
        4    0.000    0.000    0.000    0.000 numeric.py:2592(geterr)
        2    0.000    0.000    0.000    0.000 six.py:159(_resolve)
        3    0.000    0.000    0.000    0.000 threading.py:1104(is_alive)
        3    0.000    0.000    0.000    0.000 os.py:746(decode)
        1    0.000    0.000    0.002    0.002 {built-in method builtins.print}
        2    0.000    0.000    0.000    0.000 {built-in method _functools.reduce}
        1    0.000    0.000    0.000    0.000 sasreader.py:3(<module>)
        1    0.000    0.000    0.000    0.000 pytables.py:69(BinOp)
        2    0.000    0.000    0.000    0.000 expr.py:135(_is_type)
        1    0.000    0.000    0.000    0.000 excel.py:992(_OpenpyxlWriter)
       12    0.000    0.000    0.000    0.000 plotting.py:11(outer)
        1    0.000    0.000    0.000    0.000 rrule.py:1300(rruleset)
        1    0.000    0.000    0.000    0.000 ticker.py:808(LogFormatter)
      2/1    0.000    0.000    0.000    0.000 deprecation.py:248(wrapper)
        4    0.000    0.000    0.000    0.000 rcsetup.py:501(validate_ps_distiller)
        5    0.000    0.000    0.000    0.000 fontconfig_pattern.py:145(<listcomp>)
        1    0.000    0.000    0.002    0.002 series.py:1240(to_string)
        4    0.000    0.000    0.000    0.000 internals.py:3148(get_block_type)
        2    0.000    0.000    0.000    0.000 frame.py:7615(convert)
        1    0.000    0.000    0.000    0.000 range.py:68(__new__)
        4    0.000    0.000    0.000    0.000 strings.py:1849(_pat_wrapper)
        1    0.000    0.000    0.000    0.000 offsets.py:924(CustomBusinessHour)
        1    0.000    0.000    0.000    0.000 nanops.py:698(_get_counts)
        4    0.000    0.000    0.000    0.000 common.py:195(is_categorical)
        1    0.000    0.000    0.000    0.000 tzfile.py:4(<module>)
        1    0.000    0.000    0.000    0.000 pathlib.py:1241(mkdir)
        1    0.000    0.000    0.000    0.000 posixpath.py:369(abspath)
        3    0.000    0.000    0.000    0.000 _collections_abc.py:72(_check_methods)
        1    0.000    0.000    0.000    0.000 {built-in method posix.get_terminal_size}
        1    0.000    0.000    0.000    0.000 _tester.py:3(<module>)
        1    0.000    0.000    0.000    0.000 sql.py:515(SQLTable)
        1    0.000    0.000    0.000    0.000 transforms.py:1777(Affine2DBase)
        1    0.000    0.000    0.000    0.000 transforms.py:1865(Affine2D)
        6    0.000    0.000    0.000    0.000 cycler.py:371(<genexpr>)
       11    0.000    0.000    0.000    0.000 cycler.py:212(<genexpr>)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5885(Japanese)
        3    0.000    0.000    0.000    0.000 pyparsing.py:2067(suppress)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3057(__init__)
        2    0.000    0.000    0.000    0.000 __init__.py:666(get_candidate_paths)
        6    0.000    0.000    0.000    0.000 format.py:306(len)
        1    0.000    0.000    0.000    0.000 internals.py:3488(_verify_integrity)
        1    0.000    0.000    0.000    0.000 indexing.py:1519(_LocIndexer)
        1    0.000    0.000    0.000    0.000 internals.py:1717(NonConsolidatableMixIn)
        3    0.000    0.000    0.000    0.000 internals.py:3307(shape)
        1    0.000    0.000    0.001    0.001 generic.py:9599(stat_func)
        1    0.000    0.000    0.000    0.000 offsets.py:1692(YearOffset)
        1    0.000    0.000    0.000    0.000 offsets.py:878(__init__)
        1    0.000    0.000    0.000    0.000 offsets.py:1011(_CustomBusinessMonth)
        2    0.000    0.000    0.000    0.000 nanops.py:493(_nanminmax)
        4    0.000    0.000    0.000    0.000 common.py:227(is_datetimetz)
        7    0.000    0.000    0.000    0.000 config.py:789(inner)
        2    0.000    0.000    0.001    0.001 textwrap.py:356(fill)
        1    0.000    0.000    0.000    0.000 tokenize.py:355(detect_encoding)
        6    0.000    0.000    0.000    0.000 {method 'pop' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 excel.py:1442(_XlwtWriter)
        3    0.000    0.000    0.000    0.000 pytables.py:1530(set_name)
        1    0.000    0.000    0.000    0.000 ticker.py:1399(Locator)
        1    0.000    0.000    0.000    0.000 cycler.py:55(_process_keys)
        1    0.000    0.000    0.000    0.000 deprecation.py:1(<module>)
        6    0.000    0.000    0.001    0.000 rcsetup.py:329(validate_color_or_inherit)
        1    0.000    0.000    0.000    0.000 _core.py:933(LinePlot)
        2    0.000    0.000    0.000    0.000 _style.py:119(__init__)
        1    0.000    0.000    0.000    0.000 zipfile.py:1063(ZipFile)
        1    0.000    0.000    0.000    0.000 format.py:1402(_cond)
        1    0.000    0.000    0.000    0.000 internals.py:2125(DatetimeLikeBlockMixin)
        2    0.000    0.000    0.000    0.000 internals.py:3191(make_block)
        2    0.000    0.000    0.000    0.000 generic.py:124(__init__)
        4    0.000    0.000    0.000    0.000 datetimes.py:282(_join_i8_wrapper)
        1    0.000    0.000    0.000    0.000 offsets.py:1585(QuarterOffset)
        1    0.000    0.000    0.000    0.000 offsets.py:953(MonthOffset)
        3    0.000    0.000    0.000    0.000 base.py:654(_dispatch)
        1    0.000    0.000    0.002    0.002 base.py:7(<module>)
        2    0.000    0.000    0.000    0.000 dtypes.py:459(construct_from_string)
       12    0.000    0.000    0.000    0.000 algorithms.py:1253(_view_wrapper)
        2    0.000    0.000    0.000    0.000 {built-in method pandas._libs.missing.checknull}
       11    0.000    0.000    0.000    0.000 __init__.py:259(u)
        3    0.000    0.000    0.000    0.000 pathlib.py:1153(stat)
        3    0.000    0.000    0.000    0.000 __init__.py:685(__init__)
        3    0.000    0.000    0.000    0.000 _collections_abc.py:252(__subclasshook__)
        6    0.000    0.000    0.000    0.000 {method 'insert' of 'list' objects}
        8    0.000    0.000    0.000    0.000 {method 'values' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 <string>:5(ExtType)
        1    0.000    0.000    0.000    0.000 sql.py:916(SQLDatabase)
        1    0.000    0.000    0.000    0.000 transforms.py:2747(TransformedPath)
        1    0.000    0.000    0.000    0.000 rrule.py:1105(_iterinfo)
        1    0.000    0.000    0.000    0.000 ticker.py:1169(EngFormatter)
        1    0.000    0.000    0.000    0.000 ticker.py:1283(PercentFormatter)
        1    0.000    0.000    0.000    0.000 rrule.py:93(rrulebase)
        4    0.000    0.000    0.000    0.000 deprecation.py:189(finalize)
        3    0.000    0.000    0.000    0.000 rcsetup.py:644(validate_sketch)
        1    0.000    0.000    0.033    0.033 __init__.py:936(rc_params)
        1    0.000    0.000    0.000    0.000 internals.py:3922(as_array)
        1    0.000    0.000    0.000    0.000 internals.py:4639(__init__)
        1    0.000    0.000    0.000    0.000 internals.py:2778(DatetimeTZBlock)
        9    0.000    0.000    0.000    0.000 internals.py:3309(<genexpr>)
        2    0.000    0.000    0.000    0.000 generic.py:4378(__setattr__)
        1    0.000    0.000    0.000    0.000 range.py:131(_simple_new)
        5    0.000    0.000    0.000    0.000 range.py:481(__len__)
        8    0.000    0.000    0.000    0.000 range.py:578(_make_evaluate_binop)
        1    0.000    0.000    0.000    0.000 groupby.py:355(Grouper)
        1    0.000    0.000    0.000    0.000 _validators.py:4(<module>)
        1    0.000    0.000    0.000    0.000 dtypes.py:278(validate_ordered)
        3    0.000    0.000    0.000    0.000 common.py:1170(is_datetime_or_timedelta_dtype)
        4    0.000    0.000    0.000    0.000 common.py:1527(is_float_dtype)
        2    0.000    0.000    0.000    0.000 config.py:481(deprecate_option)
        1    0.000    0.000    0.000    0.000 <string>:5(RegisteredOption)
        1    0.000    0.000    0.000    0.000 terminal.py:13(<module>)
        2    0.000    0.000    0.000    0.000 {method 'sum' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 pathlib.py:674(_format_parsed_parts)
        3    0.000    0.000    0.000    0.000 functools.py:448(lru_cache)
        4    0.000    0.000    0.000    0.000 {method 'acquire' of '_thread.RLock' objects}
        1    0.000    0.000    0.000    0.000 {built-in method posix.access}
        1    0.000    0.000    0.000    0.000 stata.py:1908(StataWriter)
        1    0.000    0.000    0.000    0.000 parsers.py:1680(CParserWrapper)
        1    0.000    0.000    0.000    0.000 merge.py:484(_MergeOperation)
        1    0.000    0.000    0.000    0.000 engines.py:41(AbstractEngine)
        1    0.000    0.000    0.000    0.000 align.py:44(_filter_special_cases)
        1    0.000    0.000    0.000    0.000 ticker.py:231(TickHelper)
        1    0.000    0.000    0.000    0.000 dates.py:1031(RRuleLocator)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5844(unicode_set)
        1    0.000    0.000    0.000    0.000 pyparsing.py:1308(addCondition)
        3    0.000    0.000    0.000    0.000 rcsetup.py:484(update_savefig_format)
        1    0.000    0.000    0.000    0.000 format.py:373(DataFrameFormatter)
        1    0.000    0.000    0.000    0.000 format.py:1368(_make_fixed_width)
        1    0.000    0.000    0.001    0.001 internals.py:4869(create_block_manager_from_arrays)
        1    0.000    0.000    0.000    0.000 internals.py:1852(ExtensionBlock)
        1    0.000    0.000    0.000    0.000 groupby.py:4928(PanelGroupBy)
        1    0.000    0.000    0.000    0.000 groupby.py:5099(NDFrameSplitter)
        1    0.000    0.000    0.000    0.000 base.py:1976(is_all_dates)
        5    0.000    0.000    0.000    0.000 common.py:89(is_object_dtype)
        2    0.000    0.000    0.000    0.000 common.py:1405(needs_i8_conversion)
        3    0.000    0.000    0.000    0.000 common.py:1578(is_bool_dtype)
        1    0.000    0.000    0.000    0.000 <string>:5(DeprecatedOption)
        1    0.000    0.000    0.000    0.000 terminal.py:24(get_terminal_size)
        3    0.000    0.000    0.000    0.000 __init__.py:190(_checkLevel)
        1    0.000    0.000    0.000    0.000 sre_parse.py:885(addgroup)
        2    0.000    0.000    0.000    0.000 _bootlocale.py:23(getpreferredencoding)
        1    0.000    0.000    0.006    0.006 testing.py:5(<module>)
        1    0.000    0.000    0.000    0.000 sql.py:1342(SQLiteDatabase)
        1    0.000    0.000    0.000    0.000 expr.py:698(PandasExprVisitor)
        1    0.000    0.000    0.000    0.000 scope.py:80(Scope)
        1    0.000    0.000    0.000    0.000 ticker.py:1527(FixedLocator)
        1    0.000    0.000    0.000    0.000 colors.py:1510(LightSource)
        1    0.000    0.000    0.000    0.000 cycler.py:112(__init__)
        3    0.000    0.000    0.000    0.000 pyparsing.py:2048(__call__)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2798(Regex)
        3    0.000    0.000    0.000    0.000 rcsetup.py:631(validate_bbox)
        1    0.000    0.000    0.000    0.000 gzip.py:110(GzipFile)
        1    0.000    0.000    0.000    0.000 _style.py:107(_Options)
        3    0.000    0.000    0.000    0.000 internals.py:4718(dtype)
        1    0.000    0.000    0.000    0.000 offsets.py:465(__init__)
        6    0.000    0.000    0.000    0.000 nanops.py:834(make_nancomp)
        6    0.000    0.000    0.000    0.000 categorical.py:63(_cat_compare_op)
        1    0.000    0.000    0.000    0.000 categorical.py:2349(CategoricalAccessor)
        5    0.000    0.000    0.000    0.000 common.py:1784(_get_dtype)
        1    0.000    0.000    0.000    0.000 {pandas._libs.lib.values_from_object}
        1    0.000    0.000    0.032    0.032 __init__.py:4(<module>)
        1    0.000    0.000    0.000    0.000 {method 'transpose' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 locale.py:379(normalize)
        2    0.000    0.000    0.000    0.000 locale.py:565(getlocale)
        2    0.000    0.000    0.000    0.000 six.py:80(_import_module)
        2    0.000    0.000    0.000    0.000 sre_parse.py:452(<listcomp>)
        3    0.000    0.000    0.000    0.000 codecs.py:259(__init__)
        4    0.000    0.000    0.000    0.000 {method 'decode' of 'bytes' objects}
        2    0.000    0.000    0.000    0.000 {built-in method _locale.nl_langinfo}
        1    0.000    0.000    0.000    0.000 stata.py:2666(StataWriter117)
        1    0.000    0.000    0.000    0.000 json.py:582(Parser)
        1    0.000    0.000    0.000    0.000 pytables.py:4160(AppendableFrameTable)
        1    0.000    0.000    0.000    0.000 pytables.py:4285(GenericTable)
        1    0.000    0.000    0.000    0.000 pytables.py:379(ExprVisitor)
        1    0.000    0.000    0.000    0.000 ops.py:156(Constant)
        1    0.000    0.000    0.000    0.000 ticker.py:1591(LinearLocator)
        1    0.000    0.000    0.000    0.000 transforms.py:55(TransformNode)
        6    0.000    0.000    0.000    0.000 pyparsing.py:4449(postParse)
        3    0.000    0.000    0.000    0.000 rcsetup.py:324(<listcomp>)
        1    0.000    0.000    0.000    0.000 __init__.py:687(get_data_path)
        3    0.000    0.000    0.000    0.000 __init__.py:1876(_str_equal)
        1    0.000    0.000    0.000    0.000 _converter.py:53(get_pairs)
        3    0.000    0.000    0.000    0.000 series.py:412(dtype)
        3    0.000    0.000    0.000    0.000 series.py:4036(_try_cast)
        1    0.000    0.000    0.000    0.000 format.py:189(_chk_truncate)
        1    0.000    0.000    0.001    0.001 format.py:915(get_result)
        3    0.000    0.000    0.000    0.000 internals.py:3473(__len__)
        1    0.000    0.000    0.000    0.000 internals.py:2627(DatetimeBlock)
        1    0.000    0.000    0.000    0.000 internals.py:3351(_is_single_block)
        1    0.000    0.000    0.001    0.001 frame.py:7349(_arrays_to_mgr)
        1    0.000    0.000    0.000    0.000 frame.py:7453(_to_arrays)
        1    0.000    0.000    0.000    0.000 offsets.py:1441(WeekOfMonth)
        1    0.000    0.000    0.000    0.000 offsets.py:855(CustomBusinessDay)
        1    0.000    0.000    0.000    0.000 base.py:473(_simple_new)
        6    0.000    0.000    0.000    0.000 nanops.py:92(__init__)
        1    0.000    0.000    0.000    0.000 dtypes.py:156(_finalize)
        5    0.000    0.000    0.000    0.000 missing.py:32(isna)
        1    0.000    0.000    0.000    0.000 common.py:301(_asarray_tuplesafe)
        1    0.000    0.000    0.000    0.000 _decorators.py:212(Substitution)
        3    0.000    0.000    0.000    0.000 common.py:444(is_period_dtype)
        3    0.000    0.000    0.000    0.000 common.py:546(is_string_dtype)
        1    0.000    0.000    0.000    0.000 console.py:14(detect_console_encoding)
        1    0.000    0.000    0.001    0.001 console.py:3(<module>)
        6    0.000    0.000    0.000    0.000 __init__.py:275(strlen)
        8    0.000    0.000    0.000    0.000 basic.py:1902(_preorder_traversal)
        8    0.000    0.000    0.000    0.000 {built-in method numpy.core.umath.geterrobj}
        2    0.000    0.000    0.000    0.000 numeric.py:2887(__init__)
        2    0.000    0.000    0.000    0.000 {built-in method numpy.core.multiarray.arange}
        2    0.000    0.000    0.000    0.000 version.py:69(__ge__)
        2    0.000    0.000    0.000    0.000 locale.py:462(_parse_localename)
        1    0.000    0.000    0.000    0.000 locale.py:631(getpreferredencoding)
        7    0.000    0.000    0.000    0.000 {method 'copy' of 'dict' objects}
        2    0.000    0.000    0.000    0.000 {method 'title' of 'str' objects}
        1    0.000    0.000    0.000    0.000 testing.py:59(set_testing_mode)
        1    0.000    0.000    0.000    0.000 json.py:159(JSONTableWriter)
        1    0.000    0.000    0.000    0.000 pytables.py:4371(AppendablePanelTable)
        1    0.000    0.000    0.000    0.000 reshape.py:35(_Unstacker)
        1    0.000    0.000    0.000    0.000 rrule.py:1409(_rrulestr)
        1    0.000    0.000    0.000    0.000 dates.py:951(DateLocator)
        1    0.000    0.000    0.000    0.000 _version.py:20(get_versions)
        1    0.000    0.000    0.000    0.000 colors.py:427(Colormap)
        2    0.000    0.000    0.000    0.000 pyparsing.py:3265(__init__)
        4    0.000    0.000    0.000    0.000 rcsetup.py:249(validate_backend)
        1    0.000    0.000    0.000    0.000 _converter.py:361(PandasAutoDateFormatter)
        2    0.000    0.000    0.000    0.000 format.py:343(_get_adjustment)
        1    0.000    0.000    0.000    0.000 internals.py:5755(JoinUnit)
        1    0.000    0.000    0.000    0.000 internals.py:2150(TimeDeltaBlock)
        3    0.000    0.000    0.000    0.000 generic.py:255(<dictcomp>)
        1    0.000    0.000    0.000    0.000 frozen.py:17(FrozenList)
        6    0.000    0.000    0.000    0.000 offsets.py:2207(_tick_comp)
        1    0.000    0.000    0.000    0.000 easter.py:5(<module>)
        1    0.000    0.000    0.000    0.000 groupby.py:5073(FrameSplitter)
        1    0.000    0.000    0.000    0.000 base.py:2394(<listcomp>)
        4    0.000    0.000    0.000    0.000 nanops.py:72(<genexpr>)
        1    0.000    0.000    0.000    0.000 base.py:8(_DtypeOpsMixin)
        4    0.000    0.000    0.000    0.000 common.py:122(is_sparse)
        3    0.000    0.000    0.000    0.000 common.py:811(is_integer_dtype)
        1    0.000    0.000    0.000    0.000 tzinfo.py:18(memorized_timedelta)
        1    0.000    0.000    0.000    0.000 chainmap.py:1(<module>)
        5    0.000    0.000    0.000    0.000 _methods.py:42(_any)
        2    0.000    0.000    0.000    0.000 numeric.py:2891(__enter__)
        2    0.000    0.000    0.000    0.000 iostream.py:307(_is_master_process)
        2    0.000    0.000    0.000    0.000 traitlets.py:526(get)
        3    0.000    0.000    0.000    0.000 abc.py:9(abstractmethod)
        2    0.000    0.000    0.000    0.000 {built-in method _locale.setlocale}
        1    0.000    0.000    0.000    0.000 pytables.py:2789(SeriesFixed)
        1    0.000    0.000    0.000    0.000 ops.py:178(Op)
        1    0.000    0.000    0.000    0.000 transforms.py:2474(CompositeAffine2D)
        1    0.000    0.000    0.000    0.000 ticker.py:2541(AutoMinorLocator)
        1    0.000    0.000    0.000    0.000 cycler.py:529(_cycler)
        1    0.000    0.000    0.000    0.000 units.py:87(ConversionInterface)
        1    0.000    0.000    0.000    0.000 colors.py:870(Normalize)
        4    0.000    0.000    0.000    0.000 pyparsing.py:4536(postParse)
        3    0.000    0.000    0.000    0.000 rcsetup.py:153(validate_float_or_None)
        4    0.000    0.000    0.000    0.000 pyparsing.py:325(__init__)
        2    0.000    0.000    0.000    0.000 pyparsing.py:467(__len__)
        1    0.000    0.000    0.000    0.000 __init__.py:308(__init__)
        1    0.000    0.000    0.000    0.000 __init__.py:578(_get_xdg_config_dir)
        1    0.000    0.000    0.001    0.001 __init__.py:625(get_configdir)
        1    0.000    0.000    0.000    0.000 _core.py:1164(BarPlot)
        3    0.000    0.000    0.000    0.000 series.py:401(name)
        1    0.000    0.000    0.000    0.000 format.py:1396(_trim_zeros)
        1    0.000    0.000    0.000    0.000 generic.py:8873(_add_series_only_operations)
        1    0.000    0.000    0.000    0.000 offsets.py:2162(Easter)
        1    0.000    0.000    0.000    0.000 offsets.py:837(BusinessHour)
        1    0.000    0.000    0.000    0.000 offsets.py:1092(CustomBusinessMonthEnd)
        1    0.000    0.000    0.000    0.000 groupby.py:5019(DataSplitter)
        2    0.000    0.000    0.000    0.000 base.py:2067(__getitem__)
        1    0.000    0.000    0.000    0.000 accessor.py:13(DirNamesMixin)
        1    0.000    0.000    0.000    0.000 base.py:85(PandasObject)
        1    0.000    0.000    0.001    0.001 nanops.py:103(f)
        3    0.000    0.000    0.000    0.000 dtypes.py:584(is_dtype)
        1    0.000    0.000    0.000    0.000 common.py:123(_default_index)
        3    0.000    0.000    0.000    0.000 cast.py:853(maybe_castable)
        3    0.000    0.000    0.000    0.000 common.py:407(is_timedelta64_dtype)
        2    0.000    0.000    0.000    0.000 inference.py:251(is_list_like)
        4    0.000    0.000    0.000    0.000 {built-in method numpy.core.umath.seterrobj}
        4    0.000    0.000    0.000    0.000 numeric.py:433(asarray)
        2    0.000    0.000    0.000    0.000 numeric.py:701(<genexpr>)
        1    0.000    0.000    0.000    0.000 syspathcontext.py:48(__enter__)
        1    0.000    0.000    0.000    0.000 pathlib.py:664(_from_parsed_parts)
        2    0.000    0.000    0.000    0.000 traitlets.py:545(__get__)
        1    0.000    0.000    0.000    0.000 tokenize.py:344(_get_normal_name)
        1    0.000    0.000    0.000    0.000 re.py:314(_compile_repl)
        1    0.000    0.000    0.000    0.000 posixpath.py:64(isabs)
        3    0.000    0.000    0.000    0.000 {method 'update' of 'set' objects}
        3    0.000    0.000    0.000    0.000 {method 'acquire' of '_thread.lock' objects}
        1    0.000    0.000    0.000    0.000 _version.py:22(get_versions)
        1    0.000    0.000    0.000    0.000 html.py:147(_HtmlFrameParser)
        1    0.000    0.000    0.000    0.000 pytables.py:3912(LegacyFrameTable)
        1    0.000    0.000    0.000    0.000 pytables.py:4270(AppendableMultiSeriesTable)
        1    0.000    0.000    0.000    0.000 merge.py:1254(_AsOfMerge)
        3    0.000    0.000    0.000    0.000 util.py:68(_compose2)
        1    0.000    0.000    0.000    0.000 window.py:807(_Rolling)
        1    0.000    0.000    0.000    0.000 window.py:1942(ExpandingGroupby)
        1    0.000    0.000    0.000    0.000 transforms.py:2654(BboxTransformToMaxOnly)
        1    0.000    0.000    0.000    0.000 ticker.py:1144(LogitFormatter)
        1    0.000    0.000    0.000    0.000 dates.py:1136(AutoDateLocator)
        4    0.000    0.000    0.000    0.000 rrule.py:80(_invalidates_cache)
        1    0.000    0.000    0.000    0.000 colors.py:306(ColorConverter)
        1    0.000    0.000    0.000    0.000 colors.py:94(<dictcomp>)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4067(__str__)
        3    0.000    0.000    0.000    0.000 rcsetup.py:596(validate_svg_fonttype)
        1    0.000    0.000    0.000    0.000 __init__.py:867(Grouper)
        1    0.000    0.000    0.000    0.000 __init__.py:1769(Locked)
        1    0.000    0.000    0.000    0.000 _core.py:804(PlanePlot)
        1    0.000    0.000    0.000    0.000 _core.py:1492(PiePlot)
        1    0.000    0.000    0.000    0.000 _converter.py:403(MilliSecondLocator)
        1    0.000    0.000    0.000    0.000 csv.py:81(DictReader)
        1    0.000    0.000    0.000    0.000 indexing.py:1915(_iLocIndexer)
        1    0.000    0.000    0.000    0.000 generic.py:4436(f)
        1    0.000    0.000    0.001    0.001 frame.py:6856(f)
        1    0.000    0.000    0.000    0.000 frame.py:7604(_convert_object_array)
        3    0.000    0.000    0.000    0.000 generic.py:258(<dictcomp>)
        2    0.000    0.000    0.000    0.000 range.py:84(_ensure_int)
        2    0.000    0.000    0.000    0.000 base.py:672(values)
        1    0.000    0.000    0.000    0.000 base.py:2367(format)
        2    0.000    0.000    0.000    0.000 dtypes.py:266(construct_from_string)
        1    0.000    0.000    0.000    0.000 common.py:250(groupby)
        4    0.000    0.000    0.000    0.000 config.py:222(__init__)
        2    0.000    0.000    0.000    0.000 printing.py:52(<listcomp>)
        2    0.000    0.000    0.000    0.000 iostream.py:320(_schedule_flush)
        1    0.000    0.000    0.000    0.000 syspathcontext.py:55(__exit__)
        3    0.000    0.000    0.000    0.000 pathlib.py:1008(_init)
        1    0.000    0.000    0.000    0.000 threading.py:74(RLock)
        1    0.000    0.000    0.000    0.000 sre_parse.py:784(_parse_flags)
        2    0.000    0.000    0.000    0.000 {built-in method builtins.delattr}
        2    0.000    0.000    0.000    0.000 {method 'search' of '_sre.SRE_Pattern' objects}
        1    0.000    0.000    6.083    6.083 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 json.py:431(JsonReader)
        1    0.000    0.000    0.000    0.000 pytables.py:2850(SparseFrameFixed)
        1    0.000    0.000    0.000    0.000 pytables.py:2883(BlockManagerFixed)
        1    0.000    0.000    0.000    0.000 pytables.py:30(Term)
        3    0.000    0.000    0.000    0.000 expr.py:208(disallow)
        1    0.000    0.000    0.000    0.000 ops.py:539(FuncNode)
        1    0.000    0.000    0.000    0.000 ticker.py:205(_DummyAxis)
        1    0.000    0.000    0.000    0.000 ticker.py:338(FixedFormatter)
        1    0.000    0.000    0.000    0.000 ticker.py:1498(IndexLocator)
        1    0.000    0.000    0.000    0.000 ticker.py:1669(Base)
        1    0.000    0.000    0.000    0.000 ticker.py:2057(LogLocator)
        1    0.000    0.000    0.000    0.000 dates.py:846(rrulewrapper)
        1    0.000    0.000    0.000    0.000 dates.py:1574(MicrosecondLocator)
        1    0.000    0.000    0.000    0.000 dates.py:1766(DateConverter)
        1    0.000    0.000    0.000    0.000 rrule.py:304(rrule)
        1    0.000    0.000    0.000    0.000 colors.py:1068(SymLogNorm)
        4    0.000    0.000    0.000    0.000 pyparsing.py:5833(__init__)
        1    0.000    0.000    0.000    0.000 pyparsing.py:1477(_UnboundedCache)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3322(ParseExpression)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3434(And)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4115(Optional)
        3    0.000    0.000    0.000    0.000 rcsetup.py:168(validate_string_or_None)
        3    0.000    0.000    0.000    0.000 rcsetup.py:209(validate_int_or_None)
        3    0.000    0.000    0.000    0.000 rcsetup.py:422(validate_fontsize_None)
        1    0.000    0.000    0.000    0.000 pyparsing.py:215(ParseBaseException)
        2    0.000    0.000    0.000    0.000 pyparsing.py:654(__getattr__)
        1    0.000    0.000    0.001    0.001 __init__.py:932(copy)
        1    0.000    0.000    0.000    0.000 __init__.py:956(is_url)
        1    0.000    0.000    0.000    0.000 __init__.py:629(Stack)
        1    0.000    0.000    0.000    0.000 _core.py:1288(BarhPlot)
        1    0.000    0.000    0.000    0.000 _core.py:1310(HistPlot)
        1    0.000    0.000    0.000    0.000 csv.py:166(Sniffer)
        1    0.000    0.000    0.000    0.000 zipfile.py:760(ZipExtFile)
        1    0.000    0.000    0.000    0.000 format.py:245(_get_formatted_index)
        3    0.000    0.000    0.000    0.000 format.py:309(justify)
        1    0.000    0.000    0.000    0.000 format.py:312(adjoin)
        1    0.000    0.000    0.000    0.000 format.py:1049(<listcomp>)
        1    0.000    0.000    0.000    0.000 internals.py:3784(_consolidate_check)
        2    0.000    0.000    0.000    0.000 internals.py:5020(_asarray_compat)
        6    0.000    0.000    0.000    0.000 internals.py:233(mgr_locs)
        1    0.000    0.000    0.000    0.000 internals.py:1694(ScalarBlock)
        1    0.000    0.000    0.000    0.000 generic.py:4423(_protect_consolidate)
        1    0.000    0.000    0.000    0.000 generic.py:364(_get_axis_number)
        1    0.000    0.000    0.000    0.000 datetimelike.py:57(DatelikeOps)
        1    0.000    0.000    0.000    0.000 offsets.py:1419(_WeekOfMonthMixin)
        1    0.000    0.000    0.000    0.000 offsets.py:478(BusinessMixin)
        4    0.000    0.000    0.000    0.000 missing.py:458(_interp_wrapper)
        1    0.000    0.000    0.000    0.000 nanops.py:808(_ensure_numeric)
        1    0.000    0.000    0.000    0.000 base.py:98(ExtensionDtype)
        2    0.000    0.000    0.000    0.000 missing.py:259(notna)
        2    0.000    0.000    0.000    0.000 common.py:332(is_datetime64_dtype)
        1    0.000    0.000    0.000    0.000 {pandas._libs.lib.infer_dtype}
        1    0.000    0.000    0.000    0.000 printing.py:56(<listcomp>)
        3    0.000    0.000    0.000    0.000 printing.py:47(justify)
        3    0.000    0.000    0.000    0.000 printing.py:59(_join_unicode)
        1    0.000    0.000    0.000    0.000 terminal.py:52(is_terminal)
        1    0.000    0.000    0.000    0.000 tzinfo.py:156(DstTzInfo)
        1    0.000    0.000    0.000    0.000 __init__.py:124(lrange)
        1    0.000    0.000    0.000    0.000 __init__.py:192(UTC)
        2    0.000    0.000    0.000    0.000 numeric.py:2896(__exit__)
        3    0.000    0.000    0.000    0.000 iostream.py:93(_event_pipe)
        1    0.000    0.000    0.000    0.000 builtin_trap.py:46(__exit__)
        1    0.000    0.000    0.000    0.000 tokenize.py:379(read_or_stop)
        1    0.000    0.000    0.000    0.000 {method 'copy' of 'mappingproxy' objects}
        2    0.000    0.000    0.000    0.000 {method 'setdefault' of 'dict' objects}
        4    0.000    0.000    0.000    0.000 {method 'release' of '_thread.RLock' objects}
        2    0.000    0.000    0.000    0.000 {built-in method posix.getpid}
        1    0.000    0.000    0.000    0.000 {built-in method _codecs.lookup}
        3    0.000    0.000    0.000    0.000 {built-in method _stat.S_ISDIR}
        1    0.000    0.000    0.000    0.000 {method 'seek' of '_io.BufferedReader' objects}
        1    0.000    0.000    0.000    0.000 testing.py:2560(RNGContext)
        1    0.000    0.000    0.000    0.000 parquet.py:174(FastParquetImpl)
        3    0.000    0.000    0.000    0.000 api.py:69(outer)
        1    0.000    0.000    0.000    0.000 html.py:425(_BeautifulSoupHtml5LibFrameParser)
        1    0.000    0.000    0.000    0.000 html.py:532(_LxmlFrameParser)
        1    0.000    0.000    0.000    0.000 pytables.py:2967(FrameFixed)
        1    0.000    0.000    0.000    0.000 pytables.py:2972(PanelFixed)
        1    0.000    0.000    0.000    0.000 pytables.py:3784(WORMTable)
        1    0.000    0.000    0.000    0.000 pytables.py:4688(Selection)
        1    0.000    0.000    0.000    0.000 excel.py:1711(_XlsxWriter)
        3    0.000    0.000    0.000    0.000 pytables.py:1545(set_pos)
        1    0.000    0.000    0.000    0.000 parsers.py:3338(FixedWidthReader)
        1    0.000    0.000    0.000    0.000 ops.py:478(UnaryOp)
        1    0.000    0.000    0.000    0.000 concat.py:229(_Concatenator)
        1    0.000    0.000    0.000    0.000 ticker.py:252(Formatter)
        1    0.000    0.000    0.000    0.000 ticker.py:327(NullFormatter)
        1    0.000    0.000    0.000    0.000 ticker.py:1711(MultipleLocator)
        1    0.000    0.000    0.000    0.000 ticker.py:2256(SymmetricalLogLocator)
        1    0.000    0.000    0.000    0.000 ticker.py:2604(OldAutoLocator)
        1    0.000    0.000    0.000    0.000 dates.py:338(bytespdate2num)
        1    0.000    0.000    0.000    0.000 dates.py:747(AutoDateFormatter)
        1    0.000    0.000    0.000    0.000 units.py:52(AxisInfo)
        1    0.000    0.000    0.000    0.000 units.py:136(__init__)
        3    0.000    0.000    0.000    0.000 cycler.py:145(change_key)
        1    0.000    0.000    0.008    0.008 pyparsing.py:5057(makeHTMLTags)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2398(__init__)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2461(Keyword)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2567(CloseMatch)
        3    0.000    0.000    0.000    0.000 pyparsing.py:3130(<genexpr>)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3217(__init__)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3279(WordStart)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3589(MatchFirst)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3781(ParseElementEnhance)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4294(Forward)
        1    0.000    0.000    0.000    0.000 rcsetup.py:46(ValidateInStrings)
        3    0.000    0.000    0.000    0.000 rcsetup.py:190(validate_dpi)
        3    0.000    0.000    0.000    0.000 rcsetup.py:256(validate_qt4)
        3    0.000    0.000    0.000    0.000 rcsetup.py:413(validate_aspect)
        1    0.000    0.000    0.000    0.000 __init__.py:1424(is_interactive)
        1    0.000    0.000    0.000    0.000 _core.py:837(ScatterPlot)
        1    0.000    0.000    0.000    0.000 _core.py:1092(AreaPlot)
        1    0.000    0.000    0.000    0.000 _core.py:1436(KdePlot)
        1    0.000    0.000    0.000    0.000 _converter.py:158(TimeConverter)
        1    0.000    0.000    0.000    0.000 _converter.py:231(PeriodConverter)
        1    0.000    0.000    0.000    0.000 _converter.py:291(DatetimeConverter)
        1    0.000    0.000    0.000    0.000 accessors.py:114(DatetimeProperties)
        1    0.000    0.000    0.000    0.000 accessors.py:189(TimedeltaProperties)
        1    0.000    0.000    0.000    0.000 zipfile.py:318(ZipInfo)
        1    0.000    0.000    0.000    0.000 zipfile.py:534(_ZipDecrypter)
        1    0.000    0.000    0.001    0.001 format.py:257(_get_formatted_values)
        2    0.000    0.000    0.000    0.000 format.py:303(__init__)
        1    0.000    0.000    0.000    0.000 format.py:901(__init__)
        1    0.000    0.000    0.000    0.000 format.py:972(__init__)
        3    0.000    0.000    0.000    0.000 format.py:1078(<genexpr>)
        1    0.000    0.000    0.001    0.001 format.py:1099(_format_strings)
        3    0.000    0.000    0.000    0.000 format.py:1376(<genexpr>)
        1    0.000    0.000    0.000    0.000 format.py:1391(<listcomp>)
        1    0.000    0.000    0.000    0.000 internals.py:3785(<listcomp>)
        4    0.000    0.000    0.000    0.000 internals.py:4684(_block)
        1    0.000    0.000    0.000    0.000 indexing.py:1460(_LocationIndexer)
        1    0.000    0.000    0.000    0.000 internals.py:356(ftype)
        1    0.000    0.000    0.000    0.000 internals.py:3266(<listcomp>)
        1    0.000    0.000    0.000    0.000 generic.py:4563(values)
        1    0.000    0.000    0.000    0.000 indexing.py:29(get_indexers_list)
        1    0.000    0.000    0.000    0.000 frame.py:7621(<listcomp>)
        1    0.000    0.000    0.000    0.000 frozen.py:84(FrozenNDArray)
        1    0.000    0.000    0.000    0.000 offsets.py:1099(CustomBusinessMonthBegin)
        1    0.000    0.000    0.000    0.000 offsets.py:1201(SemiMonthEnd)
        2    0.000    0.000    0.000    0.000 base.py:635(_reset_identity)
        1    0.000    0.000    0.002    0.002 base.py:52(__str__)
        1    0.000    0.000    0.000    0.000 base.py:37(StringMixin)
        1    0.000    0.000    0.000    0.000 nanops.py:58(disallow)
        1    0.000    0.000    0.000    0.000 dtypes.py:137(__init__)
        2    0.000    0.000    0.000    0.000 _decorators.py:183(rewrite_axis_style_signature)
        1    0.000    0.000    0.000    0.000 common.py:907(is_unsigned_integer_dtype)
        1    0.000    0.000    0.000    0.000 common.py:995(is_int_or_datetime_dtype)
        5    0.000    0.000    0.000    0.000 config.py:747(is_type_factory)
        1    0.000    0.000    0.000    0.000 __init__.py:371(_FixedOffset)
        1    0.000    0.000    0.000    0.000 {method 'copy' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 {built-in method numpy.core.multiarray.datetime_data}
        1    0.000    0.000    0.000    0.000 pathlib.py:777(name)
        1    0.000    0.000    0.000    0.000 {built-in method sys.getdefaultencoding}
        1    0.000    0.000    0.000    0.000 {method 'fileno' of '_io.TextIOWrapper' objects}
        1    0.000    0.000    0.000    0.000 testing.py:2053(optional_args)
        1    0.000    0.000    0.000    0.000 testing.py:2688(SubclassedSparseDataFrame)
        1    0.000    0.000    0.000    0.000 stata.py:595(StataValueLabel)
        1    0.000    0.000    0.000    0.000 stata.py:2479(StataStrLWriter)
        1    0.000    0.000    0.000    0.000 json.py:817(FrameParser)
        1    0.000    0.000    0.000    0.000 sql.py:1230(SQLiteTable)
        1    0.000    0.000    0.000    0.000 parquet.py:43(BaseImpl)
        1    0.000    0.000    0.000    0.000 pytables.py:226(FilterBinOp)
        1    0.000    0.000    0.000    0.000 json.py:75(Writer)
        1    0.000    0.000    0.000    0.000 pytables.py:1391(TableIterator)
        1    0.000    0.000    0.000    0.000 pytables.py:1769(GenericIndexCol)
        1    0.000    0.000    0.000    0.000 pytables.py:2212(DataIndexableCol)
        1    0.000    0.000    0.000    0.000 pytables.py:2760(LegacyFixed)
        1    0.000    0.000    0.000    0.000 ops.py:315(BinOp)
        1    0.000    0.000    0.000    0.000 ops.py:523(MathCall)
        1    0.000    0.000    0.000    0.000 _depr_module.py:10(_DeprecatedModule)
        1    0.000    0.000    0.000    0.000 engines.py:101(NumExprEngine)
        1    0.000    0.000    0.000    0.000 engines.py:133(PythonEngine)
        1    0.000    0.000    0.000    0.000 transforms.py:2563(BboxTransform)
        1    0.000    0.000    0.000    0.000 transforms.py:2611(BboxTransformTo)
        1    0.000    0.000    0.000    0.000 transforms.py:2675(BboxTransformFrom)
        1    0.000    0.000    0.000    0.000 transforms.py:2715(ScaledTranslation)
        1    0.000    0.000    0.000    0.000 ticker.py:391(FormatStrFormatter)
        1    0.000    0.000    0.000    0.000 ticker.py:1767(_Edge_integer)
        1    0.000    0.000    0.000    0.000 ticker.py:2429(LogitLocator)
        1    0.000    0.000    0.000    0.000 dates.py:723(IndexDateFormatter)
        1    0.000    0.000    0.000    0.000 dates.py:1370(YearLocator)
        1    0.000    0.000    0.000    0.000 colors.py:781(ListedColormap)
        1    0.000    0.000    0.000    0.000 colors.py:1252(BoundaryNorm)
        1    0.000    0.000    0.000    0.000 colors.py:72(_ColorMapping)
        3    0.000    0.000    0.000    0.000 pyparsing.py:3132(<genexpr>)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3112(White)
        1    0.000    0.000    0.001    0.001 pyparsing.py:3419(setResultsName)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3510(Or)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4488(__init__)
        3    0.000    0.000    0.000    0.000 rcsetup.py:103(validate_any)
        3    0.000    0.000    0.000    0.000 rcsetup.py:262(validate_qt5)
        2    0.000    0.000    0.000    0.000 pyparsing.py:676(<lambda>)
        1    0.000    0.000    0.000    0.000 __init__.py:292(<dictcomp>)
        1    0.000    0.000    0.000    0.000 __init__.py:313(set_level)
        1    0.000    0.000    0.000    0.000 __init__.py:1264(rc_context)
        1    0.000    0.000    0.000    0.000 __init__.py:70(_StrongRef)
        1    0.000    0.000    0.000    0.000 __init__.py:88(CallbackRegistry)
        1    0.000    0.000    0.000    0.000 __init__.py:2020(_OrderedSet)
        1    0.000    0.000    0.000    0.000 gzip.py:69(_PaddedFile)
        1    0.000    0.000    0.000    0.000 gzip.py:377(_GzipReader)
        1    0.000    0.000    0.000    0.000 _converter.py:187(TimeFormatter)
        1    0.000    0.000    0.000    0.000 _converter.py:1065(TimeSeries_DateFormatter)
        1    0.000    0.000    0.000    0.000 csv.py:24(Dialect)
        1    0.000    0.000    0.000    0.000 zipfile.py:989(_ZipWriteFile)
        3    0.000    0.000    0.000    0.000 series.py:111(_coerce_method)
        1    0.000    0.000    0.000    0.000 format.py:351(TableFormatter)
        2    0.000    0.000    0.000    0.000 format.py:1385(just)
        1    0.000    0.000    0.000    0.000 format.py:1411(<listcomp>)
        1    0.000    0.000    0.000    0.000 format.py:1421(EngFormatter)
        1    0.000    0.000    0.000    0.000 common.py:453(MMapWrapper)
        2    0.000    0.000    0.000    0.000 internals.py:3490(<genexpr>)
        1    0.000    0.000    0.000    0.000 internals.py:4748(formatting_values)
        1    0.000    0.000    0.000    0.000 indexing.py:2126(_ScalarAccessIndexer)
        2    0.000    0.000    0.000    0.000 internals.py:127(_check_ndim)
        1    0.000    0.000    0.000    0.000 internals.py:209(formatting_values)
        4    0.000    0.000    0.000    0.000 internals.py:352(dtype)
        1    0.000    0.000    0.000    0.000 internals.py:2037(FloatBlock)
        1    0.000    0.000    0.000    0.000 internals.py:2088(ComplexBlock)
        1    0.000    0.000    0.000    0.000 internals.py:2267(BoolBlock)
        2    0.000    0.000    0.000    0.000 internals.py:3311(ndim)
        4    0.000    0.000    0.000    0.000 internals.py:3384(_get_items)
        1    0.000    0.000    0.000    0.000 generic.py:4362(__getattr__)
        4    0.000    0.000    0.000    0.000 interval.py:1394(_setop)
        1    0.000    0.000    0.000    0.000 sorting.py:269(_KeyMapper)
        1    0.000    0.000    0.000    0.000 offsets.py:1646(BQuarterEnd)
        1    0.000    0.000    0.000    0.000 offsets.py:451(SingleConstructorOffset)
        1    0.000    0.000    0.000    0.000 offsets.py:1260(SemiMonthBegin)
        1    0.000    0.000    0.000    0.000 groupby.py:531(GroupByPlot)
        1    0.000    0.000    0.000    0.000 base.py:651(GroupByMixin)
        1    0.000    0.000    0.000    0.000 nanops.py:266(_wrap_results)
        1    0.000    0.000    0.000    0.000 _decorators.py:344(docstring_wrapper)
        1    0.000    0.000    0.000    0.000 algorithms.py:1070(SelectN)
        1    0.000    0.000    0.000    0.000 common.py:858(is_signed_integer_dtype)
        1    0.000    0.000    0.000    0.000 config.py:178(__init__)
        1    0.000    0.000    0.000    0.000 inference.py:119(is_iterator)
        1    0.000    0.000    0.000    0.000 tzinfo.py:76(StaticTzInfo)
        1    0.000    0.000    0.000    0.000 tzfile.py:13(_byte_string)
        1    0.000    0.000    0.000    0.000 __init__.py:211(itervalues)
        2    0.000    0.000    0.000    0.000 _methods.py:34(_sum)
        2    0.000    0.000    0.000    0.000 {method 'fill' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 syspathcontext.py:45(__init__)
        1    0.000    0.000    0.000    0.000 {built-in method utcfromtimestamp}
        1    0.000    0.000    0.000    0.000 platform.py:1147(_sys_version)
        1    0.000    0.000    0.000    0.000 platform.py:1254(python_implementation)
        4    0.000    0.000    0.000    0.000 {method 'ljust' of 'str' objects}
        1    0.000    0.000    0.000    0.000 {built-in method builtins.sum}
        1    0.000    0.000    0.000    0.000 packers.py:759(Unpacker)
        1    0.000    0.000    0.000    0.000 gbq.py:1(<module>)
        1    0.000    0.000    0.000    0.000 testing.py:2376(_AssertRaisesContextmanager)
        1    0.000    0.000    0.000    0.000 testing.py:2652(SubclassedSeries)
        1    0.000    0.000    0.000    0.000 testing.py:2700(SubclassedCategorical)
        1    0.000    0.000    0.000    0.000 json.py:773(SeriesParser)
        1    0.000    0.000    0.000    0.000 parquet.py:73(PyArrowImpl)
        1    0.000    0.000    0.000    0.000 pytables.py:2813(SparseFixed)
        1    0.000    0.000    0.000    0.000 pytables.py:3923(LegacyPanelTable)
        1    0.000    0.000    0.000    0.000 pytables.py:20(Scope)
        1    0.000    0.000    0.000    0.000 pytables.py:298(ConditionBinOp)
        1    0.000    0.000    0.000    0.000 pytables.py:482(Expr)
        1    0.000    0.000    0.000    0.000 expr.py:709(PythonExprVisitor)
        1    0.000    0.000    0.000    0.000 json.py:116(SeriesWriter)
        1    0.000    0.000    0.000    0.000 json.py:134(FrameWriter)
        1    0.000    0.000    0.000    0.000 pytables.py:1871(set_metadata)
        1    0.000    0.000    0.000    0.000 pytables.py:2413(<dictcomp>)
        1    0.000    0.000    0.000    0.000 parsers.py:3436(FixedWidthFieldParser)
        1    0.000    0.000    0.000    0.000 ops.py:445(Div)
        1    0.000    0.000    0.000    0.000 ops.py:32(UndefinedVariableError)
        1    0.000    0.000    0.000    0.000 panel.py:1552(WidePanel)
        1    0.000    0.000    0.000    0.000 ticker.py:305(IndexFormatter)
        1    0.000    0.000    0.000    0.000 ticker.py:371(FuncFormatter)
        1    0.000    0.000    0.000    0.000 ticker.py:431(OldScalarFormatter)
        1    0.000    0.000    0.000    0.000 ticker.py:1059(LogFormatterMathtext)
        1    0.000    0.000    0.000    0.000 ticker.py:1124(LogFormatterSciNotation)
        1    0.000    0.000    0.000    0.000 ticker.py:1571(NullLocator)
        1    0.000    0.000    0.000    0.000 dates.py:322(strpdate2num)
        1    0.000    0.000    0.000    0.000 dates.py:1434(MonthLocator)
        1    0.000    0.000    0.000    0.000 dates.py:1459(WeekdayLocator)
        1    0.000    0.000    0.000    0.000 dates.py:1487(DayLocator)
        1    0.000    0.000    0.000    0.000 dates.py:1514(HourLocator)
        1    0.000    0.000    0.000    0.000 dates.py:1554(SecondLocator)
        1    0.000    0.000    0.000    0.000 rrule.py:66(weekday)
        1    0.000    0.000    0.000    0.000 units.py:132(Registry)
        1    0.000    0.000    0.000    0.000 colors.py:997(LogNorm)
        1    0.000    0.000    0.000    0.000 colors.py:1338(NoNorm)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5904(Thai)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2405(NoMatch)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2420(Literal)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2526(CaselessLiteral)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2549(CaselessKeyword)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2628(Word)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2904(QuotedString)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3041(CharsNotIn)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3196(LineStart)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3226(LineEnd)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3250(__init__)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3657(Each)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3858(FollowedBy)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3887(PrecededBy)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3960(NotAny)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4003(_MultipleMatch)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4041(OneOrMore)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4076(ZeroOrMore)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4107(_NullToken)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4387(Combine)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4543(OnlyOnce)
        1    0.000    0.000    0.000    0.000 rcsetup.py:283(validate_nseq_float)
        1    0.000    0.000    0.000    0.000 rcsetup.py:309(__init__)
        1    0.000    0.000    0.000    0.000 rcsetup.py:308(validate_nseq_int)
        1    0.000    0.000    0.000    0.000 rcsetup.py:657(ValidateInterval)
        1    0.000    0.000    0.000    0.000 pyparsing.py:196(<genexpr>)
        1    0.000    0.000    0.000    0.000 pyparsing.py:316(RecursiveGrammarException)
        1    0.000    0.000    0.000    0.000 pyparsing.py:324(_ParseResultsWithOffset)
        1    0.000    0.000    0.000    0.000 __init__.py:265(_parse_commandline)
        1    0.000    0.000    0.000    0.000 __init__.py:225(silent_list)
        1    0.000    0.000    0.000    0.000 __init__.py:1274(<dictcomp>)
        1    0.000    0.000    0.000    0.000 _core.py:901(HexBinPlot)
        1    0.000    0.000    0.000    0.000 _core.py:2703(BasePlotMethods)
        1    0.000    0.000    0.000    0.000 _converter.py:981(TimeSeries_DateLocator)
        1    0.000    0.000    0.000    0.000 _converter.py:1135(TimeSeries_TimedeltaFormatter)
        1    0.000    0.000    0.000    0.000 series.py:470(_formatting_values)
        2    0.000    0.000    0.000    0.000 series.py:562(__len__)
        1    0.000    0.000    0.000    0.000 accessors.py:24(Properties)
        1    0.000    0.000    0.000    0.000 series.py:391(_set_subtyp)
        1    0.000    0.000    0.000    0.000 format.py:113(CategoricalFormatter)
        1    0.000    0.000    0.000    0.000 format.py:167(SeriesFormatter)
        1    0.000    0.000    0.000    0.000 format.py:317(EastAsianTextAdjustment)
        1    0.000    0.000    0.000    0.000 format.py:982(_value_formatter)
        1    0.000    0.000    0.000    0.000 format.py:1403(<listcomp>)
        1    0.000    0.000    0.000    0.000 common.py:431(BytesZipFile)
        1    0.000    0.000    0.000    0.000 internals.py:4101(_consolidate_inplace)
        1    0.000    0.000    0.000    0.000 indexing.py:2162(_AtIndexer)
        1    0.000    0.000    0.000    0.000 indexing.py:2233(_iAtIndexer)
        1    0.000    0.000    0.000    0.000 internals.py:213(get_values)
        1    0.000    0.000    0.000    0.000 internals.py:2107(IntBlock)
        1    0.000    0.000    0.000    0.000 generic.py:4433(_consolidate_inplace)
        1    0.000    0.000    0.000    0.000 frame.py:7047(_get_agg_axis)
        1    0.000    0.000    0.000    0.000 multi.py:50(MultiIndexUIntEngine)
        1    0.000    0.000    0.000    0.000 offsets.py:1669(QuarterEnd)
        1    0.000    0.000    0.000    0.000 offsets.py:1744(BYearEnd)
        1    0.000    0.000    0.000    0.000 offsets.py:1760(YearEnd)
        1    0.000    0.000    0.000    0.000 offsets.py:987(MonthEnd)
        1    0.000    0.000    0.000    0.000 base.py:615(is_)
        1    0.000    0.000    0.000    0.000 base.py:2445(equals)
        1    0.000    0.000    0.000    0.000 accessor.py:43(PandasDelegate)
        1    0.000    0.000    0.000    0.000 base.py:131(NoNewAttributesMixin)
        1    0.000    0.000    0.000    0.000 nanops.py:90(bottleneck_switch)
        1    0.000    0.000    0.000    0.000 nanops.py:256(_na_ok_dtype)
        1    0.000    0.000    0.000    0.000 __init__.py:23(UnsortedIndexError)
        1    0.000    0.000    0.000    0.000 __init__.py:33(ParserError)
        1    0.000    0.000    0.000    0.000 __init__.py:164(AbstractMethodError)
        1    0.000    0.000    0.000    0.000 common.py:1043(is_datetime64_any_dtype)
        1    0.000    0.000    0.000    0.000 config.py:220(CallableDynamicDoc)
        1    0.000    0.000    0.000    0.000 printing.py:33(<listcomp>)
        1    0.000    0.000    0.000    0.000 inference.py:388(is_named_tuple)
        1    0.000    0.000    0.000    0.000 api.py:72(TimeGrouper)
        2    0.000    0.000    0.000    0.000 lazy.py:91(LazyList)
        1    0.000    0.000    0.000    0.000 lazy.py:71(LazyList)
        2    0.000    0.000    0.000    0.000 lazy.py:144(LazySet)
        1    0.000    0.000    0.000    0.000 __init__.py:387(add_metaclass)
        1    0.000    0.000    0.000    0.000 {method 'astype' of 'numpy.ndarray' objects}
        3    0.000    0.000    0.000    0.000 {method 'ravel' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 {method 'view' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 builtin_trap.py:39(__enter__)
        3    0.000    0.000    0.000    0.000 threading.py:506(is_set)
        1    0.000    0.000    0.000    0.000 sre_parse.py:98(checkgroup)
        1    0.000    0.000    0.000    0.000 {method 'startswith' of 'bytes' objects}
        2    0.000    0.000    0.000    0.000 {method 'rjust' of 'str' objects}
        3    0.000    0.000    0.000    0.000 {method 'append' of 'collections.deque' objects}
        1    0.000    0.000    0.000    0.000 pickle_compat.py:129(Unpickler)
        1    0.000    0.000    0.000    0.000 packers.py:743(Packer)
        1    0.000    0.000    0.000    0.000 packers.py:777(Iterator)
        1    0.000    0.000    0.000    0.000 __init__.py:9(ExtType)
        1    0.000    0.000    0.000    0.000 exceptions.py:1(UnpackException)
        1    0.000    0.000    0.000    0.000 exceptions.py:9(OutOfData)
        1    0.000    0.000    0.000    0.000 exceptions.py:13(UnpackValueError)
        1    0.000    0.000    0.000    0.000 exceptions.py:17(ExtraData)
        1    0.000    0.000    0.000    0.000 testing.py:2047(TestSubDict)
        1    0.000    0.000    0.000    0.000 testing.py:2278(SimpleMock)
        1    0.000    0.000    0.000    0.000 testing.py:2664(SubclassedDataFrame)
        1    0.000    0.000    0.000    0.000 testing.py:2676(SubclassedSparseSeries)
        1    0.000    0.000    0.000    0.000 stata.py:471(PossiblePrecisionLoss)
        1    0.000    0.000    0.000    0.000 stata.py:481(ValueLabelTypeMismatch)
        1    0.000    0.000    0.000    0.000 stata.py:840(StataParser)
        1    0.000    0.000    0.000    0.000 sql.py:30(SQLAlchemyRequired)
        1    0.000    0.000    0.000    0.000 sql.py:34(DatabaseError)
        1    0.000    0.000    0.000    0.000 sql.py:902(PandasSQL)
        1    0.000    0.000    0.000    0.000 pytables.py:59(Constant)
        1    0.000    0.000    0.000    0.000 pytables.py:355(UnaryOp)
        1    0.000    0.000    0.000    0.000 pytables.py:571(TermValue)
        1    0.000    0.000    0.000    0.000 expr.py:250(add_ops)
        1    0.000    0.000    0.000    0.000 pytables.py:117(PossibleDataLossError)
        1    0.000    0.000    0.000    0.000 pytables.py:121(ClosedFileError)
        1    0.000    0.000    0.000    0.000 pytables.py:125(IncompatibilityWarning)
        1    0.000    0.000    0.000    0.000 pytables.py:136(AttributeConflictWarning)
        1    0.000    0.000    0.000    0.000 pytables.py:146(DuplicateWarning)
        1    0.000    0.000    0.000    0.000 pytables.py:1856(set_data)
        1    0.000    0.000    0.000    0.000 pytables.py:2234(GenericDataIndexableCol)
        1    0.000    0.000    0.000    0.000 pytables.py:2779(LegacyFrameFixed)
        1    0.000    0.000    0.000    0.000 merge.py:1151(_OrderedMerge)
        1    0.000    0.000    0.000    0.000 engines.py:19(NumExprClobberingError)
        1    0.000    0.000    0.000    0.000 common.py:23(NameResolutionError)
        1    0.000    0.000    0.000    0.000 panel.py:1562(LongPanel)
        1    0.000    0.000    0.000    0.000 transforms.py:2821(TransformedPatchPath)
        1    0.000    0.000    0.000    0.000 ticker.py:410(StrMethodFormatter)
        1    0.000    0.000    0.000    0.000 ticker.py:1043(LogFormatterExponent)
        1    0.000    0.000    0.000    0.000 ticker.py:2521(AutoLocator)
        1    0.000    0.000    0.000    0.000 dates.py:1534(MinuteLocator)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5832(_lazyclassproperty)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5867(Latin1)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5872(Greek)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5882(Chinese)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5888(Kanji)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5891(Hiragana)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5894(Katakana)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5897(Korean)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5907(Arabic)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5910(Hebrew)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2386(Token)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2394(Empty)
        1    0.000    0.000    0.000    0.000 pyparsing.py:2787(Char)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3163(_PositionToken)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3170(GoToColumn)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3246(StringStart)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3261(StringEnd)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3299(WordEnd)
        1    0.000    0.000    0.000    0.000 pyparsing.py:3450(_ErrorStop)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4179(SkipTo)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4379(TokenConverter)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4431(Group)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4452(Dict)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4517(Suppress)
        1    0.000    0.000    0.000    0.000 pyparsing.py:271(ParseException)
        1    0.000    0.000    0.000    0.000 pyparsing.py:297(ParseSyntaxException)
        1    0.000    0.000    0.000    0.000 __init__.py:491(checkdep_ps_distiller)
        1    0.000    0.000    0.000    0.000 __init__.py:315(Bunch)
        1    0.000    0.000    0.000    0.000 __init__.py:505(GetRealpathAndStat)
        1    0.000    0.000    0.000    0.000 __init__.py:609(maxdict)
        1    0.000    0.000    0.000    0.000 __init__.py:1784(TimeoutError)
        1    0.000    0.000    0.000    0.000 deprecation.py:6(MatplotlibDeprecationWarning)
        1    0.000    0.000    0.000    0.000 _converter.py:377(PandasAutoDateLocator)
        1    0.000    0.000    0.000    0.000 accessors.py:266(PeriodProperties)
        1    0.000    0.000    0.000    0.000 accessors.py:291(CombinedDatetimelikeProperties)
        1    0.000    0.000    0.000    0.000 csv.py:55(excel)
        1    0.000    0.000    0.000    0.000 csv.py:70(unix_dialect)
        1    0.000    0.000    0.000    0.000 csv.py:131(DictWriter)
        1    0.000    0.000    0.000    0.000 zipfile.py:43(BadZipFile)
        1    0.000    0.000    0.000    0.000 zipfile.py:595(LZMACompressor)
        1    0.000    0.000    0.000    0.000 zipfile.py:618(LZMADecompressor)
        1    0.000    0.000    0.000    0.000 zipfile.py:740(_Tellable)
        1    0.000    0.000    0.000    0.000 zipfile.py:1819(PyZipFile)
        1    0.000    0.000    0.000    0.000 format.py:301(TextAdjustment)
        1    0.000    0.000    0.000    0.000 format.py:899(GenericArrayFormatter)
        1    0.000    0.000    0.000    0.000 format.py:967(FloatArrayFormatter)
        1    0.000    0.000    0.000    0.000 format.py:1115(Datetime64Formatter)
        1    0.000    0.000    0.000    0.000 format.py:1141(IntervalArrayFormatter)
        1    0.000    0.000    0.000    0.000 format.py:1299(Datetime64TZFormatter)
        1    0.000    0.000    0.000    0.000 format.py:1314(Timedelta64Formatter)
        1    0.000    0.000    0.000    0.000 common.py:61(BaseIterator)
        1    0.000    0.000    0.000    0.000 common.py:496(UTF8Recoder)
        2    0.000    0.000    0.000    0.000 internals.py:3776(is_consolidated)
        1    0.000    0.000    0.000    0.000 internals.py:4085(consolidate)
        2    0.000    0.000    0.000    0.000 internals.py:4990(<lambda>)
        1    0.000    0.000    0.000    0.000 internals.py:5026(_shape_compat)
        1    0.000    0.000    0.000    0.000 internals.py:2021(NumericBlock)
        1    0.000    0.000    0.000    0.000 internals.py:2027(FloatOrComplexBlock)
        1    0.000    0.000    0.000    0.000 indexing.py:45(_IndexSlice)
        1    0.000    0.000    0.000    0.000 indexing.py:84(IndexingError)
        1    0.000    0.000    0.000    0.000 frame.py:320(_constructor)
        1    0.000    0.000    0.000    0.000 multi.py:89(MultiIndexPyIntEngine)
        1    0.000    0.000    0.000    0.000 offsets.py:1660(BQuarterBegin)
        1    0.000    0.000    0.000    0.000 offsets.py:1681(QuarterBegin)
        1    0.000    0.000    0.000    0.000 offsets.py:1752(BYearBegin)
        1    0.000    0.000    0.000    0.000 offsets.py:1767(YearBegin)
        1    0.000    0.000    0.000    0.000 offsets.py:460(_CustomMixin)
        1    0.000    0.000    0.000    0.000 offsets.py:993(MonthBegin)
        1    0.000    0.000    0.000    0.000 offsets.py:999(BusinessMonthEnd)
        1    0.000    0.000    0.000    0.000 offsets.py:1005(BusinessMonthBegin)
        1    0.000    0.000    0.000    0.000 groupby.py:5067(SeriesSplitter)
        1    0.000    0.000    0.000    0.000 base.py:163(InvalidIndexError)
        1    0.000    0.000    0.000    0.000 base.py:162(GroupByError)
        1    0.000    0.000    0.000    0.000 base.py:166(DataError)
        1    0.000    0.000    0.000    0.000 nanops.py:48(set_use_bottleneck)
        1    0.000    0.000    0.000    0.000 nanops.py:260(_view_if_needed)
        1    0.000    0.000    0.000    0.000 function.py:38(__call__)
        1    0.000    0.000    0.000    0.000 function.py:29(CompatValidator)
        1    0.000    0.000    0.000    0.000 dtypes.py:376(DatetimeTZDtypeType)
        1    0.000    0.000    0.000    0.000 dtypes.py:490(PeriodDtypeType)
        1    0.000    0.000    0.000    0.000 dtypes.py:607(IntervalDtypeType)
        1    0.000    0.000    0.000    0.000 generic.py:64(_ABCGeneric)
        1    0.000    0.000    0.000    0.000 __init__.py:10(PerformanceWarning)
        1    0.000    0.000    0.000    0.000 __init__.py:16(UnsupportedFunctionCall)
        1    0.000    0.000    0.000    0.000 __init__.py:39(DtypeWarning)
        1    0.000    0.000    0.000    0.000 __init__.py:97(EmptyDataError)
        1    0.000    0.000    0.000    0.000 __init__.py:104(ParserWarning)
        1    0.000    0.000    0.000    0.000 __init__.py:145(MergeError)
        1    0.000    0.000    0.000    0.000 __init__.py:152(NullFrequencyError)
        1    0.000    0.000    0.000    0.000 __init__.py:160(AccessorRegistrationWarning)
        1    0.000    0.000    0.000    0.000 common.py:23(SettingWithCopyError)
        1    0.000    0.000    0.000    0.000 _decorators.py:273(Appender)
        1    0.000    0.000    0.000    0.000 algorithms.py:1096(SelectNSeries)
        1    0.000    0.000    0.000    0.000 algorithms.py:1157(SelectNFrame)
        1    0.000    0.000    0.000    0.000 common.py:1490(is_string_like_dtype)
        2    0.000    0.000    0.000    0.000 {built-in method pandas._libs.lib.is_integer}
        1    0.000    0.000    0.000    0.000 config.py:69(OptionError)
        1    0.000    0.000    0.000    0.000 config.py:175(DictWrapper)
        1    0.000    0.000    0.000    0.000 config.py:377(option_context)
        1    0.000    0.000    0.000    0.000 __init__.py:288(_CountryTimezoneDict)
        1    0.000    0.000    0.000    0.000 exceptions.py:11(UnknownTimeZoneError)
        1    0.000    0.000    0.000    0.000 exceptions.py:42(NonExistentTimeError)
        1    0.000    0.000    0.000    0.000 lazy.py:16(LazyDict)
        1    0.000    0.000    0.000    0.000 lazy.py:121(LazySet)
        1    0.000    0.000    0.000    0.000 tzinfo.py:66(BaseTzInfo)
        1    0.000    0.000    0.000    0.000 chainmap.py:7(DeepChainMap)
        1    0.000    0.000    0.000    0.000 {method 'reshape' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 calendar.py:53(<lambda>)
        1    0.000    0.000    0.000    0.000 interactiveshell.py:690(get_ipython)
        1    0.000    0.000    0.000    0.000 {method 'toordinal' of 'datetime.date' objects}
        1    0.000    0.000    0.000    0.000 {method 'items' of 'collections.OrderedDict' objects}
        1    0.000    0.000    0.000    0.000 {method 'discard' of 'set' objects}
        1    0.000    0.000    0.000    0.000 {method 'write' of '_io.StringIO' objects}
        1    0.000    0.000    0.000    0.000 exceptions.py:5(BufferFull)
        1    0.000    0.000    0.000    0.000 exceptions.py:27(PackException)
        1    0.000    0.000    0.000    0.000 exceptions.py:31(PackValueError)
        1    0.000    0.000    0.000    0.000 _version.py:1(<module>)
        1    0.000    0.000    0.000    0.000 stata.py:492(InvalidColumnName)
        1    0.000    0.000    0.000    0.000 pytables.py:346(JointConditionBinOp)
        1    0.000    0.000    0.000    0.000 pytables.py:2770(LegacySeriesFixed)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5879(Cyrillic)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5900(CJK)
        1    0.000    0.000    0.000    0.000 pyparsing.py:5913(Devanagari)
        1    0.000    0.000    0.000    0.000 pyparsing.py:1500(_FifoCache)
        1    0.000    0.000    0.000    0.000 pyparsing.py:4375(_ForwardNoRecurse)
        1    0.000    0.000    0.000    0.000 pyparsing.py:292(ParseFatalException)
        1    0.000    0.000    0.000    0.000 __init__.py:525(checkdep_usetex)
        1    0.000    0.000    0.000    0.000 __init__.py:250(IgnoredKeywordWarning)
        1    0.000    0.000    0.000    0.000 series.py:405(name)
        1    0.000    0.000    0.000    0.000 csv.py:65(excel_tab)
        1    0.000    0.000    0.000    0.000 zipfile.py:47(LargeZipFile)
        1    0.000    0.000    0.000    0.000 zipfile.py:714(_SharedFile)
        1    0.000    0.000    0.000    0.000 format.py:1107(IntArrayFormatter)
        1    0.000    0.000    0.000    0.000 format.py:1152(PeriodArrayFormatter)
        1    0.000    0.000    0.000    0.000 format.py:1167(CategoricalArrayFormatter)
        1    0.000    0.000    0.000    0.000 internals.py:203(internal_values)
        1    0.000    0.000    0.000    0.000 internals.py:348(shape)
        1    0.000    0.000    0.000    0.000 range.py:158(_validate_dtype)
        1    0.000    0.000    0.000    0.000 accessor.py:113(CachedAccessor)
        1    0.000    0.000    0.000    0.000 base.py:170(SpecificationError)
        1    0.000    0.000    0.000    0.000 nanops.py:179(_get_fill_value)
        1    0.000    0.000    0.000    0.000 dtypes.py:79(CategoricalDtypeType)
        1    0.000    0.000    0.000    0.000 common.py:27(SettingWithCopyWarning)
        1    0.000    0.000    0.000    0.000 common.py:154(_all_none)
        1    0.000    0.000    0.000    0.000 __init__.py:346(_CountryNameDict)
        1    0.000    0.000    0.000    0.000 exceptions.py:27(InvalidTimeError)
        1    0.000    0.000    0.000    0.000 exceptions.py:31(AmbiguousTimeError)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 sre_parse.py:101(checklookbehindgroup)
        1    0.000    0.000    0.000    0.000 {method 'upper' of 'str' objects}
        1    0.000    0.000    0.000    0.000 {built-in method _stat.S_ISREG}
        1    0.000    0.000    0.000    0.000 {method 'getvalue' of '_io.StringIO' objects}
 

The profiling information includes the following information:

  • ncalls: number of calls to function
  • tottime: total time spent in the given function (excluding time in calls to sub-functions)
  • percall: time per call
  • cumtime: cumulative time spent in this and all subfunctions

We can see that most of the time in this example is spent inside of core NumPy functions and methods.

The %prun command does a similar job for single Python expressions (like function calls).

In [10]:
%prun trapez(2, 6, 100000)
         100006 function calls in 0.235 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.122    0.122    0.235    0.235 <ipython-input-4-82c378c72124>:4(trapez)
   100002    0.113    0.000    0.113    0.000 <ipython-input-4-82c378c72124>:1(f)
        1    0.000    0.000    0.235    0.235 {built-in method builtins.exec}
        1    0.000    0.000    0.235    0.235 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
 

For even more fine-grained profiling information, we can use a line profiler to see how long it takes each line of a function to run.

In [11]:
!pprofile ../examples/bisection.py
/bin/sh: 1: pprofile: not found

This output makes it clear that the biggest cost is in the repeated calling of the function $f$ for which the root is being found. If we could improve the speed of this function, it would be the easiest single way of improving the performance of the code.

Snakeviz

SnakeViz is a browser based graphical viewer for the output of Python’s cProfile module. Though it is ostensibly a command-line tool, SnakeViz includes IPython line and cell magics for going straight from code to a visualization.

In [12]:
%load_ext snakeviz

Let's load the ABC example from the source, so that we can run it in the notebook.

In [13]:
# %load ../examples/abc.py
import numpy as np
import pandas as pd

def abc(y, N, epsilon):

    trace = []

    while len(trace) < N:

        # Simulate from priors
        mu = np.random.normal(0, 10)
        sigma = np.random.uniform(0, 20)

        x = np.random.normal(mu, sigma, 50)

        #if (np.linalg.norm(y - x) < epsilon):
        if ((abs(x.mean() - y.mean()) < epsilon[0]) &
            (abs(x.std() - y.std()) < epsilon[1])):
            trace.append([mu, sigma])

    return pd.DataFrame(trace, columns=['mu', 'sigma'])

if __name__ == '__main__':
    y = np.random.normal(4, 2, 50)
    N = 20
    epsilon = [0.2, 0.8]
    print(abc(y, N, epsilon).mean(0))
mu       4.031276
sigma    2.042091
dtype: float64
In [14]:
%snakeviz abc(y=np.random.normal(4, 2, 50), N=20, epsilon=[0.2, 0.8]).mean(0)
 
*** Profile stats marshalled to file '/tmp/tmp41fk4j6q'. 

The default SnakeViz visualization displays profiles as a sunburst that represent function calls as a series of nested arcs, expanding outward. Thus, the root function is represented as a circle at the center, with the functions it calls displayed as arcs wrapping around it, and so forth. The amount of time spent inside a function is represented by the angular width of the arc. Hence, an arc that wraps most of the way around the circle represents a function that is taking up most of the time of its calling function, while a narrow arc represents a function that is using relatively little time.

Functions don’t just spend time calling other functions, they also have their own internal time. SnakeViz shows this by putting a special child on each node that represents internal time. Only functions that call other functions will have this, functions with no calls are entirely internal time.

SnakeViz is useful for a global overview of the runtime of your project because it might uncover functions that do not seem (at first) to be worth optimizing because of their short runtime, but are in fact called from other functions so that their total runtime is significant.

snakeviz

Exercise: Profiling EM

Use one or more of the above methods to profile your expectation maximization code, and locate any bottlenecks.

In [15]:
# Write your answer here

Speeding up Python by being Pythonic

When you have decided that your code is unacceptably slow, and have gone through the process of profiling to see if and where your program is experiencing a bottleneck, it can be easy to jump ahead and try speeding it up using external tools. There are several packages that will certainly improve Python's performance (and we will introduce some of them later), but the first place to look for better performance is in refactoring your implementation of whichever algorithm you happen to be using.

Effective Python programming involves applying particular Python idioms effectively; these are idiosyncratic expressions that may only exist in Python (if you are coming from another languate), but when used appropriately they can make your code more readable, faster, or both. You have seen some of these already -- for example, the list comprehension as a means for succinctly implementing a for loop.

Comprehensions

In [16]:
def do_math(x):
    return 3 + x**3
In [17]:
%%timeit
squares = np.empty(1000000)
for i in range(1000000):
    squares[i] = do_math(i)
963 ms ± 246 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
In [18]:
%timeit squares = [do_math(i) for i in range(1000000)]
820 ms ± 176 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Here, not only is the list comprehension easier to write and read, it is also slightly faster.

Generators

When you are dealing with a large number of elements that you do not need all at once, you can also consider another Python expression we have already seen, a generator. For example, if we enclose the comprehension in parentheses instead of square brackets, we get a generator expression object:

In [19]:
(i**2 for i in range(int(1e20)))
Out[19]:
<generator object <genexpr> at 0x7c1b342d2a98>

Now, rather than storing 100,000,000,000,000,000,000,000 elements in memory, we can produce values as needed:

In [20]:
squares = (i**2 for i in range(int(1e10)))
next(squares)
Out[20]:
0
In [21]:
next(squares)
Out[21]:
1

Built-in functions

Before you go about coding your own functions, make sure that it isn't already provided as a built-in function. These are typically highly optimized, and written in C! Here is a list of built-in functions.

String concatenation

Just as you should avoid growing lists or arrays by concatenation or appending, iterating over strings and concatenating them manually is very inefficient. For example, let's say we want to concatente a list of strings into a single string:

In [22]:
words = ["Six",
"days",
"in",
"to",
"what",
"should",
"be",
"a",
"greatest",
"two",
"months",
"of",
"my",
"life",
"and",
"it’s",
"turned",
"in",
"to",
"a",
"nightmare"]

One might be tempted to code the following:

In [23]:
%%timeit
sentence = ""
for word in words:
    sentence += " " + word
4.11 µs ± 625 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [24]:
sentence = ""
for word in words:
    sentence += " " + word
sentence
Out[24]:
' Six days in to what should be a greatest two months of my life and it’s turned in to a nightmare'

However, this is inefficient; since strings is immutable in Python, every + operation involves creating a new string and copying the old content. Instead, we can use the string method join, which is not only faster, but more flexible. Here, we would like to separate the words by spaces, which is easily done:

In [25]:
' '.join(words)
Out[25]:
'Six days in to what should be a greatest two months of my life and it’s turned in to a nightmare'
In [26]:
%timeit ' '.join(words)
921 ns ± 178 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

Avoid loops

As we have seen, for loops in Python are slow. Wherever possible, avoid looping by using alternative strategies or vectorized operations. For example, say we wanted to return the common elements between two arrays. We might naively loop over both lists, comparing them elementwise to return their intersection:

In [27]:
np.random.seed(1)
list1 = np.random.choice(np.arange(20), replace=False, size=10)
list2 = np.random.choice(np.arange(20), replace=False, size=10)

def common_elements(a, b):
    for i in a:
        for j in b:
            if i==j:
                yield i
In [28]:
%timeit list(common_elements(list1, list2))
32.4 µs ± 1.93 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

However, this involves two Python for loops and a conditional statement. Instead, we can use set operations on the built-in set type provided by Python:

In [29]:
%timeit set(list1) & set(list2)
7.03 µs ± 1.32 µs per loop (mean ± std. dev. of 7 runs, 100000 loops each)

Use NumPy

Often, considerable performance gains can be achieved by replacing Python data structures and functions with corresponding NumPy versions. It provides a high-performance multidimensional array object, and tools for working with these arrays.

This example, borrowed from NumPy creator Travis Oliphant, solves Laplace's equation over a 2-d rectangular grid using a simple iterative method. The code finds a two-dimensional function, u, where ∇2 u = 0, given some fixed boundary conditions.

In [30]:
dx = 0.1
dy = 0.1
dx2 = dx*dx
dy2 = dy*dy

def py_update(u):
    nx, ny = u.shape
    for i in range(1,nx-1):
        for j in range(1, ny-1):
            u[i,j] = ((u[i+1, j] + u[i-1, j]) * dy2 +
                      (u[i, j+1] + u[i, j-1]) * dx2) / (2*(dx2+dy2))
In [31]:
def calc(N, Niter=100, func=py_update, args=()):
    u = np.zeros([N, N])
    u[0] = 1
    for i in range(Niter):
        func(u,*args)
    return u

This code takes a very long time to run in order to converge to the correct solution. For a 100x100 grid, visually-indistinguishable convergence occurs after about 8000 iterations.

In [32]:
%timeit calc(10)
20.9 ms ± 1.86 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Using NumPy, we can speed this code up significantly by using slicing and vectorized (automatic looping) calculations that replace the explicit loops in the Python-only solution.

In [33]:
def num_update(u):
    u[1:-1,1:-1] = ((u[2:,1:-1]+u[:-2,1:-1])*dy2 + 
                    (u[1:-1,2:] + u[1:-1,:-2])*dx2) / (2*(dx2+dy2))
In [34]:
%timeit calc(10, func=num_update)
1.49 ms ± 89.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Such speed-ups are not uncommon when using NumPy to replace Python loops where the inner loop is doing simple math on basic data-types.

Memoize repeated function calls

The fibonacci sequence benchmark cited in the table above on the Julia website uses the following implementation:

In [35]:
def fib(n):
    if n < 2:
        return n
    return fib(n-1) + fib(n-2)

This is a simple recursive function, and as such, grows quickly with the size of n passed to it.

In [36]:
fib(5)
Out[36]:
5
In [37]:
fib(20)
Out[37]:
6765
In [38]:
fib(40)
Out[38]:
102334155

Notice how long the last call took. We can use the timeit magic to quantify the speed of this naive implementation.

In [39]:
%timeit fib(20)
4.18 ms ± 412 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

The first thing you should notice is that, being a recursive function, the same values will tend to be calculated over and over again. What if, instead of re-calculating values that have been calculated before, we instead store calculations so that they can be retrieved when needed the next time. Almost always, looking up a value stored in an array or hash table is going to be faster than performing an expensive calculation.

This optimization technique is called memoization.

Here is a memoization function that is implemented as a decorator.

In [40]:
def memoize(f):
    memo = {}
    def func(arg):
        if arg not in memo:            
            memo[arg] = f(arg)
        return memo[arg]
    return func

We can simply annotate our function with this decorator, and run it the same way as the original function.

In [41]:
@memoize
def fib_memo(n):
    if n < 2:
        return n
    return fib_memo(n - 1) + fib_memo(n - 2)
In [42]:
fib_memo(20)
Out[42]:
6765
In [44]:
%timeit fib_memo(20)
171 ns ± 8.78 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

This is a massive speedup.

The built-in functools library includes an efficient memoization function called lru_cache (LRU: Least Recently Used).

In [45]:
from functools import lru_cache

@lru_cache(maxsize=None)
def fib_memo(n):
    if n < 2:
        return n
    return fib_memo(n - 1) + fib_memo(n - 2)

The only constraint on lru_cache is that the arguments to the function must be hashable.

Fast array expression evaluation with numexpr

numexpr allows array expressions to be evaluated far faster that what can be achieved in Python using Numpy arrays. numexpr parses a string expression and optimizes and compiles the code on the fly, using a virtual machine that includes a Just-in-time (JIT) compiler. In addition, numexpr offers direct support for parallel multi-threaded computations, since Python's global interpreter lock is bypassed.

Python's global interpreter lock (GIL) ensures that only one thread runs in the interpreter at once. This simplifies many of the low-level activities, such as memory management, and allows for co-operative multi-tasking. But, since the currently-running thread holds onto the interpreter, it makes multi-core parallelization difficult.

Part the reason Python can be slow for array calculations is that it creates temporary arrays to store intermediate results from array element calculations, which wastes memory and cache. numexpr handles such calculations in manageable chunks, which accellerates computation.

The speedup over NumPy by using numexpr can be as high as 20x, but is typically in the range of 2-4x.

Example: Computing a polynomial

In [46]:
import numpy as np

x = np.linspace(-1, 1, 1000000)

0.25*x**3 + 0.75*x**2 - 1.5*x - 2
Out[46]:
array([ 0.0000000e+00, -4.5000045e-06, -9.0000090e-06, ...,
       -2.5000030e+00, -2.5000015e+00, -2.5000000e+00])
In [47]:
%timeit 0.25*x**3 + 0.75*x**2 - 1.5*x - 2
82.1 ms ± 2.94 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [48]:
import numexpr as ne

ne.set_num_threads(1)
Out[48]:
4
In [49]:
ne.evaluate('0.25*x**3 + 0.75*x**2 - 1.5*x - 2')
Out[49]:
array([ 0.0000000e+00, -4.5000045e-06, -9.0000090e-06, ...,
       -2.5000030e+00, -2.5000015e+00, -2.5000000e+00])
In [50]:
%timeit ne.evaluate('0.25*x**3 + 0.75*x**2 - 1.5*x - 2')
7.34 ms ± 644 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

numexpr actually expands the polynomial terms so that it does not need to use a transcendental function.

We can achieve further gains in performance by multithreading the calculations:

In [51]:
ne.set_num_threads(4)
Out[51]:
1
In [52]:
%timeit ne.evaluate('0.25*x**3 + 0.75*x**2 - 1.5*x - 2')
5.05 ms ± 1.89 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)

Since the performance of processors has outpaced that of memory in the past several decades, the CPU spends a lot of time waiting for memory to give it computations; this is the processor-memory performance gap.

performance gap (graph courtesy http://www.techdesignforums.com)

CPU caches are often used to make up for this difference. CPU caches are more effective when the data are optimally located in memory to take advantage of cache performance. numexpr does this by moving contiguous blocks of data from memory to the CPU cache, reusing them as much as possible within the cache to more quickly give the CPU access to the data.

Limitations

numexpr only implements element-wise operations. So, a * b becomes:

for i in range(N):
    c[i] = a[i] * b[i]

Similarly, it cannot index other parts of arrays in the same expression:

for i in range(N):
    c[i] = a[i-1] + a[i] * b[i]

Cython

Python developers typically solve performance constraints by building Python extensions by wrapping code written in other languages (for example, SciPy contains more lines of C/C++/Fortran than Python). However, programming with the Python/C API is not straightforward for most users.

Cython is a language that allows Python programmers to write fast code without having to write C/C++/Fortran directly. It looks much like Python code, but with type declarations. Cython code is translated it to C (or C++ or others), which is then compiled to create a Python extension that we can import and use.

Using Cython, we can achieve speedups of several orders of magnitude, often faster than hand-coded C code. In addtion, Cython is compatible with core scientific programming tools like NumPy and IPython.

Cython has built-in support for multicore processing.

Cython is used to varying degrees by other packages in the Python scientific stack, such as pandas, sympy, scikit-learn and SciPy.

Example: Numerical integration

Recall from above the function trapez for performing numerical integration using the trapezoidal rule. As a benchmark, let's time the execution of the pure-Python version of the trapezoidal rule:

In [53]:
%timeit trapez(1, 5, 1000)
440 µs ± 16.9 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Perhaps the easiest way to use Cython, is via the IPython cythonmagic, which allows us to run Cython interactively:

In [54]:
%load_ext cython
In [55]:
%%cython

def f(x):
    return 2*x*x + 3*x + 1

def trapez2(a, b, n):
    h = (b-a)/float(n) 
    sumy = 0
    x=a
    for i in range(n):
        x += h
        sumy += f(x)
    sumy += 0.5*(f(a) + f(b))
    return sumy*h

The Cython magic is doing a lot of work for you: it compiles the code into an extension module, and loads it into the notebook. This allows us to ignore all of the compilation details of building Cython extensions.

If we run trapez2, we can see a reasonable speedup simply by compiling it, unchanged, using Cython.

In [56]:
%timeit trapez2(1, 5, 1000)
180 µs ± 10 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Under the hood, several things are happening in order to deliver this improved performance. The Cython source code is translated into C source code by cython. Then, this C source is compiled, using the appropriate compiler, flags and associated library files (if any), into a Python extension. This extension is then loaded by IPython into the current session.

cython flow

C extensions can also be compiled manually, using a setup file. Here is an example for an extension called dist within a package called probability:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

import numpy as np

setup(
    cmdclass = {'build_ext': build_ext},
    ext_modules = [Extension("dist", ["probability/src/dist.pyx"], include_dirs=[np.get_include()])]
)

It mainly uses machinery from a core Python package distutils that manages the build process.

To get a closer look at where Cython is improving our unchanged Python code, we can add an --annotate flag to the %%cython magic declaration:

In [57]:
%%cython --annotate

def f(x):
    return 2*x*x + 3*x + 1
      
def trapez2(a, b, n):
    h = (b-a)/float(n) 
    sumy = 0
    x=a
    for i in range(n):
        x += h
        sumy += f(x)
    sumy += 0.5*(f(a) + f(b))
    return sumy*h
Out[57]:
Cython: _cython_magic_7c2f66cc05b706df58359bfb0c2282d5.pyx

Generated by Cython 0.29.2

Yellow lines hint at Python interaction.
Click on a line that starts with a "+" to see the C code that Cython generated for it.

 01: 
+02: def f(x):
/* Python wrapper */
static PyObject *__pyx_pw_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_1f(PyObject *__pyx_self, PyObject *__pyx_v_x); /*proto*/
static PyMethodDef __pyx_mdef_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_1f = {"f", (PyCFunction)__pyx_pw_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_1f, METH_O, 0};
static PyObject *__pyx_pw_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_1f(PyObject *__pyx_self, PyObject *__pyx_v_x) {
  PyObject *__pyx_r = 0;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("f (wrapper)", 0);
  __pyx_r = __pyx_pf_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_f(__pyx_self, ((PyObject *)__pyx_v_x));

  /* function exit code */
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}

static PyObject *__pyx_pf_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_f(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_x) {
  PyObject *__pyx_r = NULL;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("f", 0);
/* … */
  /* function exit code */
  __pyx_L1_error:;
  __Pyx_XDECREF(__pyx_t_1);
  __Pyx_XDECREF(__pyx_t_2);
  __Pyx_XDECREF(__pyx_t_3);
  __Pyx_AddTraceback("_cython_magic_7c2f66cc05b706df58359bfb0c2282d5.f", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __pyx_r = NULL;
  __pyx_L0:;
  __Pyx_XGIVEREF(__pyx_r);
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}
/* … */
  __pyx_tuple_ = PyTuple_Pack(1, __pyx_n_s_x); if (unlikely(!__pyx_tuple_)) __PYX_ERR(0, 2, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_tuple_);
  __Pyx_GIVEREF(__pyx_tuple_);
/* … */
  __pyx_t_1 = PyCFunction_NewEx(&__pyx_mdef_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_1f, NULL, __pyx_n_s_cython_magic_7c2f66cc05b706df58); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  if (PyDict_SetItem(__pyx_d, __pyx_n_s_f, __pyx_t_1) < 0) __PYX_ERR(0, 2, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
  __pyx_codeobj__2 = (PyObject*)__Pyx_PyCode_New(1, 0, 1, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple_, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_home_fonnesbeck_cache_ipython_c, __pyx_n_s_f, 2, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__2)) __PYX_ERR(0, 2, __pyx_L1_error)
+03:     return 2*x*x + 3*x + 1
  __Pyx_XDECREF(__pyx_r);
  __pyx_t_1 = PyNumber_Multiply(__pyx_int_2, __pyx_v_x); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 3, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  __pyx_t_2 = PyNumber_Multiply(__pyx_t_1, __pyx_v_x); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 3, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_2);
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
  __pyx_t_1 = PyNumber_Multiply(__pyx_int_3, __pyx_v_x); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 3, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  __pyx_t_3 = PyNumber_Add(__pyx_t_2, __pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 3, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_3);
  __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
  __pyx_t_1 = __Pyx_PyInt_AddObjC(__pyx_t_3, __pyx_int_1, 1, 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 3, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
  __pyx_r = __pyx_t_1;
  __pyx_t_1 = 0;
  goto __pyx_L0;
 04: 
+05: def trapez2(a, b, n):
/* Python wrapper */
static PyObject *__pyx_pw_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_3trapez2(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
static PyMethodDef __pyx_mdef_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_3trapez2 = {"trapez2", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_3trapez2, METH_VARARGS|METH_KEYWORDS, 0};
static PyObject *__pyx_pw_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_3trapez2(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
  PyObject *__pyx_v_a = 0;
  PyObject *__pyx_v_b = 0;
  PyObject *__pyx_v_n = 0;
  PyObject *__pyx_r = 0;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("trapez2 (wrapper)", 0);
  {
    static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_a,&__pyx_n_s_b,&__pyx_n_s_n,0};
    PyObject* values[3] = {0,0,0};
    if (unlikely(__pyx_kwds)) {
      Py_ssize_t kw_args;
      const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
      switch (pos_args) {
        case  3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
        CYTHON_FALLTHROUGH;
        case  2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
        CYTHON_FALLTHROUGH;
        case  1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
        CYTHON_FALLTHROUGH;
        case  0: break;
        default: goto __pyx_L5_argtuple_error;
      }
      kw_args = PyDict_Size(__pyx_kwds);
      switch (pos_args) {
        case  0:
        if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_a)) != 0)) kw_args--;
        else goto __pyx_L5_argtuple_error;
        CYTHON_FALLTHROUGH;
        case  1:
        if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_b)) != 0)) kw_args--;
        else {
          __Pyx_RaiseArgtupleInvalid("trapez2", 1, 3, 3, 1); __PYX_ERR(0, 5, __pyx_L3_error)
        }
        CYTHON_FALLTHROUGH;
        case  2:
        if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_n)) != 0)) kw_args--;
        else {
          __Pyx_RaiseArgtupleInvalid("trapez2", 1, 3, 3, 2); __PYX_ERR(0, 5, __pyx_L3_error)
        }
      }
      if (unlikely(kw_args > 0)) {
        if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "trapez2") < 0)) __PYX_ERR(0, 5, __pyx_L3_error)
      }
    } else if (PyTuple_GET_SIZE(__pyx_args) != 3) {
      goto __pyx_L5_argtuple_error;
    } else {
      values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
      values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
      values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
    }
    __pyx_v_a = values[0];
    __pyx_v_b = values[1];
    __pyx_v_n = values[2];
  }
  goto __pyx_L4_argument_unpacking_done;
  __pyx_L5_argtuple_error:;
  __Pyx_RaiseArgtupleInvalid("trapez2", 1, 3, 3, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 5, __pyx_L3_error)
  __pyx_L3_error:;
  __Pyx_AddTraceback("_cython_magic_7c2f66cc05b706df58359bfb0c2282d5.trapez2", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __Pyx_RefNannyFinishContext();
  return NULL;
  __pyx_L4_argument_unpacking_done:;
  __pyx_r = __pyx_pf_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_2trapez2(__pyx_self, __pyx_v_a, __pyx_v_b, __pyx_v_n);

  /* function exit code */
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}

static PyObject *__pyx_pf_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_2trapez2(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_a, PyObject *__pyx_v_b, PyObject *__pyx_v_n) {
  PyObject *__pyx_v_h = NULL;
  PyObject *__pyx_v_sumy = NULL;
  PyObject *__pyx_v_x = NULL;
  CYTHON_UNUSED PyObject *__pyx_v_i = NULL;
  PyObject *__pyx_r = NULL;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("trapez2", 0);
/* … */
  /* function exit code */
  __pyx_L1_error:;
  __Pyx_XDECREF(__pyx_t_1);
  __Pyx_XDECREF(__pyx_t_2);
  __Pyx_XDECREF(__pyx_t_3);
  __Pyx_XDECREF(__pyx_t_6);
  __Pyx_AddTraceback("_cython_magic_7c2f66cc05b706df58359bfb0c2282d5.trapez2", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __pyx_r = NULL;
  __pyx_L0:;
  __Pyx_XDECREF(__pyx_v_h);
  __Pyx_XDECREF(__pyx_v_sumy);
  __Pyx_XDECREF(__pyx_v_x);
  __Pyx_XDECREF(__pyx_v_i);
  __Pyx_XGIVEREF(__pyx_r);
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}
/* … */
  __pyx_tuple__3 = PyTuple_Pack(7, __pyx_n_s_a, __pyx_n_s_b, __pyx_n_s_n, __pyx_n_s_h, __pyx_n_s_sumy, __pyx_n_s_x, __pyx_n_s_i); if (unlikely(!__pyx_tuple__3)) __PYX_ERR(0, 5, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_tuple__3);
  __Pyx_GIVEREF(__pyx_tuple__3);
/* … */
  __pyx_t_1 = PyCFunction_NewEx(&__pyx_mdef_46_cython_magic_7c2f66cc05b706df58359bfb0c2282d5_3trapez2, NULL, __pyx_n_s_cython_magic_7c2f66cc05b706df58); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 5, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  if (PyDict_SetItem(__pyx_d, __pyx_n_s_trapez2, __pyx_t_1) < 0) __PYX_ERR(0, 5, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
+06:     h = (b-a)/float(n)
  __pyx_t_1 = PyNumber_Subtract(__pyx_v_b, __pyx_v_a); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 6, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  __pyx_t_2 = __Pyx_PyNumber_Float(__pyx_v_n); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 6, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_2);
  __pyx_t_3 = __Pyx_PyNumber_Divide(__pyx_t_1, __pyx_t_2); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 6, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_3);
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
  __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
  __pyx_v_h = __pyx_t_3;
  __pyx_t_3 = 0;
+07:     sumy = 0
  __Pyx_INCREF(__pyx_int_0);
  __pyx_v_sumy = __pyx_int_0;
+08:     x=a
  __Pyx_INCREF(__pyx_v_a);
  __pyx_v_x = __pyx_v_a;
+09:     for i in range(n):
  __pyx_t_3 = __Pyx_PyObject_CallOneArg(__pyx_builtin_range, __pyx_v_n); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 9, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_3);
  if (likely(PyList_CheckExact(__pyx_t_3)) || PyTuple_CheckExact(__pyx_t_3)) {
    __pyx_t_2 = __pyx_t_3; __Pyx_INCREF(__pyx_t_2); __pyx_t_4 = 0;
    __pyx_t_5 = NULL;
  } else {
    __pyx_t_4 = -1; __pyx_t_2 = PyObject_GetIter(__pyx_t_3); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 9, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_2);
    __pyx_t_5 = Py_TYPE(__pyx_t_2)->tp_iternext; if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 9, __pyx_L1_error)
  }
  __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
  for (;;) {
    if (likely(!__pyx_t_5)) {
      if (likely(PyList_CheckExact(__pyx_t_2))) {
        if (__pyx_t_4 >= PyList_GET_SIZE(__pyx_t_2)) break;
        #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
        __pyx_t_3 = PyList_GET_ITEM(__pyx_t_2, __pyx_t_4); __Pyx_INCREF(__pyx_t_3); __pyx_t_4++; if (unlikely(0 < 0)) __PYX_ERR(0, 9, __pyx_L1_error)
        #else
        __pyx_t_3 = PySequence_ITEM(__pyx_t_2, __pyx_t_4); __pyx_t_4++; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 9, __pyx_L1_error)
        __Pyx_GOTREF(__pyx_t_3);
        #endif
      } else {
        if (__pyx_t_4 >= PyTuple_GET_SIZE(__pyx_t_2)) break;
        #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS
        __pyx_t_3 = PyTuple_GET_ITEM(__pyx_t_2, __pyx_t_4); __Pyx_INCREF(__pyx_t_3); __pyx_t_4++; if (unlikely(0 < 0)) __PYX_ERR(0, 9, __pyx_L1_error)
        #else
        __pyx_t_3 = PySequence_ITEM(__pyx_t_2, __pyx_t_4); __pyx_t_4++; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 9, __pyx_L1_error)
        __Pyx_GOTREF(__pyx_t_3);
        #endif
      }
    } else {
      __pyx_t_3 = __pyx_t_5(__pyx_t_2);
      if (unlikely(!__pyx_t_3)) {
        PyObject* exc_type = PyErr_Occurred();
        if (exc_type) {
          if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear();
          else __PYX_ERR(0, 9, __pyx_L1_error)
        }
        break;
      }
      __Pyx_GOTREF(__pyx_t_3);
    }
    __Pyx_XDECREF_SET(__pyx_v_i, __pyx_t_3);
    __pyx_t_3 = 0;
/* … */
  }
  __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
+10:         x += h
    __pyx_t_3 = PyNumber_InPlaceAdd(__pyx_v_x, __pyx_v_h); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 10, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_3);
    __Pyx_DECREF_SET(__pyx_v_x, __pyx_t_3);
    __pyx_t_3 = 0;
+11:         sumy += f(x)
    __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_f); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 11, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_1);
    __pyx_t_6 = NULL;
    if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_1))) {
      __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_1);
      if (likely(__pyx_t_6)) {
        PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1);
        __Pyx_INCREF(__pyx_t_6);
        __Pyx_INCREF(function);
        __Pyx_DECREF_SET(__pyx_t_1, function);
      }
    }
    __pyx_t_3 = (__pyx_t_6) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_6, __pyx_v_x) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_v_x);
    __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0;
    if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 11, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_3);
    __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
    __pyx_t_1 = PyNumber_InPlaceAdd(__pyx_v_sumy, __pyx_t_3); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 11, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_1);
    __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
    __Pyx_DECREF_SET(__pyx_v_sumy, __pyx_t_1);
    __pyx_t_1 = 0;
+12:     sumy += 0.5*(f(a) + f(b))
  __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_f); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 12, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  __pyx_t_3 = NULL;
  if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_1))) {
    __pyx_t_3 = PyMethod_GET_SELF(__pyx_t_1);
    if (likely(__pyx_t_3)) {
      PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1);
      __Pyx_INCREF(__pyx_t_3);
      __Pyx_INCREF(function);
      __Pyx_DECREF_SET(__pyx_t_1, function);
    }
  }
  __pyx_t_2 = (__pyx_t_3) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_3, __pyx_v_a) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_v_a);
  __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0;
  if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 12, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_2);
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
  __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_f); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 12, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_3);
  __pyx_t_6 = NULL;
  if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) {
    __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_3);
    if (likely(__pyx_t_6)) {
      PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3);
      __Pyx_INCREF(__pyx_t_6);
      __Pyx_INCREF(function);
      __Pyx_DECREF_SET(__pyx_t_3, function);
    }
  }
  __pyx_t_1 = (__pyx_t_6) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_6, __pyx_v_b) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_b);
  __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0;
  if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 12, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
  __pyx_t_3 = PyNumber_Add(__pyx_t_2, __pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 12, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_3);
  __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
  __pyx_t_1 = PyNumber_Multiply(__pyx_float_0_5, __pyx_t_3); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 12, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
  __pyx_t_3 = PyNumber_InPlaceAdd(__pyx_v_sumy, __pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 12, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_3);
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
  __Pyx_DECREF_SET(__pyx_v_sumy, __pyx_t_3);
  __pyx_t_3 = 0;
+13:     return sumy*h
  __Pyx_XDECREF(__pyx_r);
  __pyx_t_3 = PyNumber_Multiply(__pyx_v_sumy, __pyx_v_h); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 13, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_3);
  __pyx_r = __pyx_t_3;
  __pyx_t_3 = 0;
  goto __pyx_L0;

In the above, the line color indicates the "typedness" of the extension, where yellower lines are closer to Python, and therefore require calls to the Python C API, while whiter lines indicate code that is closer to pure C, hence requiring few, if any, Python API calls.

If you click on a line, it unravels to show you the C code that results from the call to cython.

The goal in speeding up code with Cython is to turn as many lines to white as we can. The easiest way to do this is to add type declarations to the Python code:

In [58]:
%%cython --annotate

# Add type to argument
def ff(double x):
    return 2*x*x + 3*x + 1

# Add types to arguments
def trapez3(double a, double b, int n):
    # Declare types of variables
    cdef double h, x, sumy
    cdef int i
    h = (b-a)/float(n) 
    sumy = 0
    x=a
    for i in range(n):
        x += h
        sumy += ff(x)
    sumy += 0.5*(ff(a) + ff(b))
    return sumy*h
Out[58]:
Cython: _cython_magic_edb24a3ae69e1a29821d036c6f188980.pyx

Generated by Cython 0.29.2

Yellow lines hint at Python interaction.
Click on a line that starts with a "+" to see the C code that Cython generated for it.

 01: 
 02: # Add type to argument
+03: def ff(double x):
/* Python wrapper */
static PyObject *__pyx_pw_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_1ff(PyObject *__pyx_self, PyObject *__pyx_arg_x); /*proto*/
static PyMethodDef __pyx_mdef_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_1ff = {"ff", (PyCFunction)__pyx_pw_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_1ff, METH_O, 0};
static PyObject *__pyx_pw_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_1ff(PyObject *__pyx_self, PyObject *__pyx_arg_x) {
  double __pyx_v_x;
  PyObject *__pyx_r = 0;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("ff (wrapper)", 0);
  assert(__pyx_arg_x); {
    __pyx_v_x = __pyx_PyFloat_AsDouble(__pyx_arg_x); if (unlikely((__pyx_v_x == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 3, __pyx_L3_error)
  }
  goto __pyx_L4_argument_unpacking_done;
  __pyx_L3_error:;
  __Pyx_AddTraceback("_cython_magic_edb24a3ae69e1a29821d036c6f188980.ff", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __Pyx_RefNannyFinishContext();
  return NULL;
  __pyx_L4_argument_unpacking_done:;
  __pyx_r = __pyx_pf_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_ff(__pyx_self, ((double)__pyx_v_x));

  /* function exit code */
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}

static PyObject *__pyx_pf_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_ff(CYTHON_UNUSED PyObject *__pyx_self, double __pyx_v_x) {
  PyObject *__pyx_r = NULL;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("ff", 0);
/* … */
  /* function exit code */
  __pyx_L1_error:;
  __Pyx_XDECREF(__pyx_t_1);
  __Pyx_AddTraceback("_cython_magic_edb24a3ae69e1a29821d036c6f188980.ff", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __pyx_r = NULL;
  __pyx_L0:;
  __Pyx_XGIVEREF(__pyx_r);
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}
/* … */
  __pyx_tuple_ = PyTuple_Pack(2, __pyx_n_s_x, __pyx_n_s_x); if (unlikely(!__pyx_tuple_)) __PYX_ERR(0, 3, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_tuple_);
  __Pyx_GIVEREF(__pyx_tuple_);
/* … */
  __pyx_t_1 = PyCFunction_NewEx(&__pyx_mdef_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_1ff, NULL, __pyx_n_s_cython_magic_edb24a3ae69e1a2982); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 3, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  if (PyDict_SetItem(__pyx_d, __pyx_n_s_ff, __pyx_t_1) < 0) __PYX_ERR(0, 3, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
  __pyx_codeobj__2 = (PyObject*)__Pyx_PyCode_New(1, 0, 2, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple_, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_home_fonnesbeck_cache_ipython_c, __pyx_n_s_ff, 3, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__2)) __PYX_ERR(0, 3, __pyx_L1_error)
+04:     return 2*x*x + 3*x + 1
  __Pyx_XDECREF(__pyx_r);
  __pyx_t_1 = PyFloat_FromDouble(((((2.0 * __pyx_v_x) * __pyx_v_x) + (3.0 * __pyx_v_x)) + 1.0)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 4, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  __pyx_r = __pyx_t_1;
  __pyx_t_1 = 0;
  goto __pyx_L0;
 05: 
 06: # Add types to arguments
+07: def trapez3(double a, double b, int n):
/* Python wrapper */
static PyObject *__pyx_pw_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_3trapez3(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
static PyMethodDef __pyx_mdef_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_3trapez3 = {"trapez3", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_3trapez3, METH_VARARGS|METH_KEYWORDS, 0};
static PyObject *__pyx_pw_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_3trapez3(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
  double __pyx_v_a;
  double __pyx_v_b;
  int __pyx_v_n;
  PyObject *__pyx_r = 0;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("trapez3 (wrapper)", 0);
  {
    static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_a,&__pyx_n_s_b,&__pyx_n_s_n,0};
    PyObject* values[3] = {0,0,0};
    if (unlikely(__pyx_kwds)) {
      Py_ssize_t kw_args;
      const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
      switch (pos_args) {
        case  3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
        CYTHON_FALLTHROUGH;
        case  2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
        CYTHON_FALLTHROUGH;
        case  1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
        CYTHON_FALLTHROUGH;
        case  0: break;
        default: goto __pyx_L5_argtuple_error;
      }
      kw_args = PyDict_Size(__pyx_kwds);
      switch (pos_args) {
        case  0:
        if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_a)) != 0)) kw_args--;
        else goto __pyx_L5_argtuple_error;
        CYTHON_FALLTHROUGH;
        case  1:
        if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_b)) != 0)) kw_args--;
        else {
          __Pyx_RaiseArgtupleInvalid("trapez3", 1, 3, 3, 1); __PYX_ERR(0, 7, __pyx_L3_error)
        }
        CYTHON_FALLTHROUGH;
        case  2:
        if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_n)) != 0)) kw_args--;
        else {
          __Pyx_RaiseArgtupleInvalid("trapez3", 1, 3, 3, 2); __PYX_ERR(0, 7, __pyx_L3_error)
        }
      }
      if (unlikely(kw_args > 0)) {
        if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "trapez3") < 0)) __PYX_ERR(0, 7, __pyx_L3_error)
      }
    } else if (PyTuple_GET_SIZE(__pyx_args) != 3) {
      goto __pyx_L5_argtuple_error;
    } else {
      values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
      values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
      values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
    }
    __pyx_v_a = __pyx_PyFloat_AsDouble(values[0]); if (unlikely((__pyx_v_a == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 7, __pyx_L3_error)
    __pyx_v_b = __pyx_PyFloat_AsDouble(values[1]); if (unlikely((__pyx_v_b == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 7, __pyx_L3_error)
    __pyx_v_n = __Pyx_PyInt_As_int(values[2]); if (unlikely((__pyx_v_n == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 7, __pyx_L3_error)
  }
  goto __pyx_L4_argument_unpacking_done;
  __pyx_L5_argtuple_error:;
  __Pyx_RaiseArgtupleInvalid("trapez3", 1, 3, 3, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 7, __pyx_L3_error)
  __pyx_L3_error:;
  __Pyx_AddTraceback("_cython_magic_edb24a3ae69e1a29821d036c6f188980.trapez3", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __Pyx_RefNannyFinishContext();
  return NULL;
  __pyx_L4_argument_unpacking_done:;
  __pyx_r = __pyx_pf_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_2trapez3(__pyx_self, __pyx_v_a, __pyx_v_b, __pyx_v_n);

  /* function exit code */
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}

static PyObject *__pyx_pf_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_2trapez3(CYTHON_UNUSED PyObject *__pyx_self, double __pyx_v_a, double __pyx_v_b, int __pyx_v_n) {
  double __pyx_v_h;
  double __pyx_v_x;
  double __pyx_v_sumy;
  CYTHON_UNUSED int __pyx_v_i;
  PyObject *__pyx_r = NULL;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("trapez3", 0);
/* … */
  /* function exit code */
  __pyx_L1_error:;
  __Pyx_XDECREF(__pyx_t_5);
  __Pyx_XDECREF(__pyx_t_6);
  __Pyx_XDECREF(__pyx_t_7);
  __Pyx_XDECREF(__pyx_t_8);
  __Pyx_XDECREF(__pyx_t_9);
  __Pyx_XDECREF(__pyx_t_10);
  __Pyx_AddTraceback("_cython_magic_edb24a3ae69e1a29821d036c6f188980.trapez3", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __pyx_r = NULL;
  __pyx_L0:;
  __Pyx_XGIVEREF(__pyx_r);
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}
/* … */
  __pyx_tuple__3 = PyTuple_Pack(7, __pyx_n_s_a, __pyx_n_s_b, __pyx_n_s_n, __pyx_n_s_h, __pyx_n_s_x, __pyx_n_s_sumy, __pyx_n_s_i); if (unlikely(!__pyx_tuple__3)) __PYX_ERR(0, 7, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_tuple__3);
  __Pyx_GIVEREF(__pyx_tuple__3);
/* … */
  __pyx_t_1 = PyCFunction_NewEx(&__pyx_mdef_46_cython_magic_edb24a3ae69e1a29821d036c6f188980_3trapez3, NULL, __pyx_n_s_cython_magic_edb24a3ae69e1a2982); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 7, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  if (PyDict_SetItem(__pyx_d, __pyx_n_s_trapez3, __pyx_t_1) < 0) __PYX_ERR(0, 7, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
 08:     # Declare types of variables
 09:     cdef double h, x, sumy
 10:     cdef int i
+11:     h = (b-a)/float(n)
  __pyx_t_1 = (__pyx_v_b - __pyx_v_a);
  if (unlikely(((double)__pyx_v_n) == 0)) {
    PyErr_SetString(PyExc_ZeroDivisionError, "float division");
    __PYX_ERR(0, 11, __pyx_L1_error)
  }
  __pyx_v_h = (__pyx_t_1 / ((double)__pyx_v_n));
+12:     sumy = 0
  __pyx_v_sumy = 0.0;
+13:     x=a
  __pyx_v_x = __pyx_v_a;
+14:     for i in range(n):
  __pyx_t_2 = __pyx_v_n;
  __pyx_t_3 = __pyx_t_2;
  for (__pyx_t_4 = 0; __pyx_t_4 < __pyx_t_3; __pyx_t_4+=1) {
    __pyx_v_i = __pyx_t_4;
+15:         x += h
    __pyx_v_x = (__pyx_v_x + __pyx_v_h);
+16:         sumy += ff(x)
    __pyx_t_5 = PyFloat_FromDouble(__pyx_v_sumy); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 16, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_5);
    __Pyx_GetModuleGlobalName(__pyx_t_7, __pyx_n_s_ff); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 16, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_7);
    __pyx_t_8 = PyFloat_FromDouble(__pyx_v_x); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 16, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_8);
    __pyx_t_9 = NULL;
    if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_7))) {
      __pyx_t_9 = PyMethod_GET_SELF(__pyx_t_7);
      if (likely(__pyx_t_9)) {
        PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_7);
        __Pyx_INCREF(__pyx_t_9);
        __Pyx_INCREF(function);
        __Pyx_DECREF_SET(__pyx_t_7, function);
      }
    }
    __pyx_t_6 = (__pyx_t_9) ? __Pyx_PyObject_Call2Args(__pyx_t_7, __pyx_t_9, __pyx_t_8) : __Pyx_PyObject_CallOneArg(__pyx_t_7, __pyx_t_8);
    __Pyx_XDECREF(__pyx_t_9); __pyx_t_9 = 0;
    __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0;
    if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 16, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_6);
    __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0;
    __pyx_t_7 = PyNumber_InPlaceAdd(__pyx_t_5, __pyx_t_6); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 16, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_7);
    __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
    __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
    __pyx_t_1 = __pyx_PyFloat_AsDouble(__pyx_t_7); if (unlikely((__pyx_t_1 == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 16, __pyx_L1_error)
    __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0;
    __pyx_v_sumy = __pyx_t_1;
  }
+17:     sumy += 0.5*(ff(a) + ff(b))
  __pyx_t_7 = PyFloat_FromDouble(__pyx_v_sumy); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 17, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_7);
  __Pyx_GetModuleGlobalName(__pyx_t_5, __pyx_n_s_ff); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 17, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_5);
  __pyx_t_8 = PyFloat_FromDouble(__pyx_v_a); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 17, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_8);
  __pyx_t_9 = NULL;
  if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_5))) {
    __pyx_t_9 = PyMethod_GET_SELF(__pyx_t_5);
    if (likely(__pyx_t_9)) {
      PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_5);
      __Pyx_INCREF(__pyx_t_9);
      __Pyx_INCREF(function);
      __Pyx_DECREF_SET(__pyx_t_5, function);
    }
  }
  __pyx_t_6 = (__pyx_t_9) ? __Pyx_PyObject_Call2Args(__pyx_t_5, __pyx_t_9, __pyx_t_8) : __Pyx_PyObject_CallOneArg(__pyx_t_5, __pyx_t_8);
  __Pyx_XDECREF(__pyx_t_9); __pyx_t_9 = 0;
  __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0;
  if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 17, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_6);
  __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
  __Pyx_GetModuleGlobalName(__pyx_t_8, __pyx_n_s_ff); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 17, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_8);
  __pyx_t_9 = PyFloat_FromDouble(__pyx_v_b); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 17, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_9);
  __pyx_t_10 = NULL;
  if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_8))) {
    __pyx_t_10 = PyMethod_GET_SELF(__pyx_t_8);
    if (likely(__pyx_t_10)) {
      PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_8);
      __Pyx_INCREF(__pyx_t_10);
      __Pyx_INCREF(function);
      __Pyx_DECREF_SET(__pyx_t_8, function);
    }
  }
  __pyx_t_5 = (__pyx_t_10) ? __Pyx_PyObject_Call2Args(__pyx_t_8, __pyx_t_10, __pyx_t_9) : __Pyx_PyObject_CallOneArg(__pyx_t_8, __pyx_t_9);
  __Pyx_XDECREF(__pyx_t_10); __pyx_t_10 = 0;
  __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0;
  if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 17, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_5);
  __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0;
  __pyx_t_8 = PyNumber_Add(__pyx_t_6, __pyx_t_5); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 17, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_8);
  __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
  __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
  __pyx_t_5 = PyNumber_Multiply(__pyx_float_0_5, __pyx_t_8); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 17, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_5);
  __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0;
  __pyx_t_8 = PyNumber_InPlaceAdd(__pyx_t_7, __pyx_t_5); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 17, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_8);
  __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0;
  __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
  __pyx_t_1 = __pyx_PyFloat_AsDouble(__pyx_t_8); if (unlikely((__pyx_t_1 == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 17, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0;
  __pyx_v_sumy = __pyx_t_1;
+18:     return sumy*h
  __Pyx_XDECREF(__pyx_r);
  __pyx_t_8 = PyFloat_FromDouble((__pyx_v_sumy * __pyx_v_h)); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 18, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_8);
  __pyx_r = __pyx_t_8;
  __pyx_t_8 = 0;
  goto __pyx_L0;
In [59]:
%timeit trapez3(1, 5, 1000)
51.1 µs ± 1.13 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

This gives us a considerable speedup. The next thing we might try is to inline the polynomial function. By inlining, we mean that we ask the compiler to perform an inline expansion of said function; that is, it will insert a copy of the function itself wherever the function is called, instead of calling the function wherever it is defined.

We do three things to the specification of ff:

  • change def to cdef
  • add a return type to the function
  • add an inline keyword
In [60]:
%%cython --annotate

cdef inline double ff(double x):
    return 2*x*x + 3*x + 1

cpdef trapez4(double a, double b, int n):
    cdef double h, x, sumy
    cdef int i
    h = (b-a)/n 
    sumy = 0
    x=a
    for i in range(n):
        x += h
        sumy += ff(x)
    sumy += 0.5*(ff(a) + ff(b))
    return sumy*h
Out[60]:
Cython: _cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a.pyx

Generated by Cython 0.29.2

Yellow lines hint at Python interaction.
Click on a line that starts with a "+" to see the C code that Cython generated for it.

 01: 
+02: cdef inline double ff(double x):
static CYTHON_INLINE double __pyx_f_46_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a_ff(double __pyx_v_x) {
  double __pyx_r;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("ff", 0);
/* … */
  /* function exit code */
  __pyx_L0:;
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}
+03:     return 2*x*x + 3*x + 1
  __pyx_r = ((((2.0 * __pyx_v_x) * __pyx_v_x) + (3.0 * __pyx_v_x)) + 1.0);
  goto __pyx_L0;
 04: 
+05: cpdef trapez4(double a, double b, int n):
static PyObject *__pyx_pw_46_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a_1trapez4(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
static PyObject *__pyx_f_46_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a_trapez4(double __pyx_v_a, double __pyx_v_b, int __pyx_v_n, CYTHON_UNUSED int __pyx_skip_dispatch) {
  double __pyx_v_h;
  double __pyx_v_x;
  double __pyx_v_sumy;
  CYTHON_UNUSED int __pyx_v_i;
  PyObject *__pyx_r = NULL;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("trapez4", 0);
/* … */
  /* function exit code */
  __pyx_L1_error:;
  __Pyx_XDECREF(__pyx_t_5);
  __Pyx_AddTraceback("_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a.trapez4", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __pyx_r = 0;
  __pyx_L0:;
  __Pyx_XGIVEREF(__pyx_r);
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}

/* Python wrapper */
static PyObject *__pyx_pw_46_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a_1trapez4(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
static PyObject *__pyx_pw_46_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a_1trapez4(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
  double __pyx_v_a;
  double __pyx_v_b;
  int __pyx_v_n;
  PyObject *__pyx_r = 0;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("trapez4 (wrapper)", 0);
  {
    static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_a,&__pyx_n_s_b,&__pyx_n_s_n,0};
    PyObject* values[3] = {0,0,0};
    if (unlikely(__pyx_kwds)) {
      Py_ssize_t kw_args;
      const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
      switch (pos_args) {
        case  3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
        CYTHON_FALLTHROUGH;
        case  2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
        CYTHON_FALLTHROUGH;
        case  1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
        CYTHON_FALLTHROUGH;
        case  0: break;
        default: goto __pyx_L5_argtuple_error;
      }
      kw_args = PyDict_Size(__pyx_kwds);
      switch (pos_args) {
        case  0:
        if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_a)) != 0)) kw_args--;
        else goto __pyx_L5_argtuple_error;
        CYTHON_FALLTHROUGH;
        case  1:
        if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_b)) != 0)) kw_args--;
        else {
          __Pyx_RaiseArgtupleInvalid("trapez4", 1, 3, 3, 1); __PYX_ERR(0, 5, __pyx_L3_error)
        }
        CYTHON_FALLTHROUGH;
        case  2:
        if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_n)) != 0)) kw_args--;
        else {
          __Pyx_RaiseArgtupleInvalid("trapez4", 1, 3, 3, 2); __PYX_ERR(0, 5, __pyx_L3_error)
        }
      }
      if (unlikely(kw_args > 0)) {
        if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "trapez4") < 0)) __PYX_ERR(0, 5, __pyx_L3_error)
      }
    } else if (PyTuple_GET_SIZE(__pyx_args) != 3) {
      goto __pyx_L5_argtuple_error;
    } else {
      values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
      values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
      values[2] = PyTuple_GET_ITEM(__pyx_args, 2);
    }
    __pyx_v_a = __pyx_PyFloat_AsDouble(values[0]); if (unlikely((__pyx_v_a == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 5, __pyx_L3_error)
    __pyx_v_b = __pyx_PyFloat_AsDouble(values[1]); if (unlikely((__pyx_v_b == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 5, __pyx_L3_error)
    __pyx_v_n = __Pyx_PyInt_As_int(values[2]); if (unlikely((__pyx_v_n == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 5, __pyx_L3_error)
  }
  goto __pyx_L4_argument_unpacking_done;
  __pyx_L5_argtuple_error:;
  __Pyx_RaiseArgtupleInvalid("trapez4", 1, 3, 3, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 5, __pyx_L3_error)
  __pyx_L3_error:;
  __Pyx_AddTraceback("_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a.trapez4", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __Pyx_RefNannyFinishContext();
  return NULL;
  __pyx_L4_argument_unpacking_done:;
  __pyx_r = __pyx_pf_46_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a_trapez4(__pyx_self, __pyx_v_a, __pyx_v_b, __pyx_v_n);

  /* function exit code */
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}

static PyObject *__pyx_pf_46_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a_trapez4(CYTHON_UNUSED PyObject *__pyx_self, double __pyx_v_a, double __pyx_v_b, int __pyx_v_n) {
  PyObject *__pyx_r = NULL;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("trapez4", 0);
  __Pyx_XDECREF(__pyx_r);
  __pyx_t_1 = __pyx_f_46_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a_trapez4(__pyx_v_a, __pyx_v_b, __pyx_v_n, 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 5, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  __pyx_r = __pyx_t_1;
  __pyx_t_1 = 0;
  goto __pyx_L0;

  /* function exit code */
  __pyx_L1_error:;
  __Pyx_XDECREF(__pyx_t_1);
  __Pyx_AddTraceback("_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a.trapez4", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __pyx_r = NULL;
  __pyx_L0:;
  __Pyx_XGIVEREF(__pyx_r);
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}
 06:     cdef double h, x, sumy
 07:     cdef int i
+08:     h = (b-a)/n
  __pyx_t_1 = (__pyx_v_b - __pyx_v_a);
  if (unlikely(__pyx_v_n == 0)) {
    PyErr_SetString(PyExc_ZeroDivisionError, "float division");
    __PYX_ERR(0, 8, __pyx_L1_error)
  }
  __pyx_v_h = (__pyx_t_1 / ((double)__pyx_v_n));
+09:     sumy = 0
  __pyx_v_sumy = 0.0;
+10:     x=a
  __pyx_v_x = __pyx_v_a;
+11:     for i in range(n):
  __pyx_t_2 = __pyx_v_n;
  __pyx_t_3 = __pyx_t_2;
  for (__pyx_t_4 = 0; __pyx_t_4 < __pyx_t_3; __pyx_t_4+=1) {
    __pyx_v_i = __pyx_t_4;
+12:         x += h
    __pyx_v_x = (__pyx_v_x + __pyx_v_h);
+13:         sumy += ff(x)
    __pyx_v_sumy = (__pyx_v_sumy + __pyx_f_46_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a_ff(__pyx_v_x));
  }
+14:     sumy += 0.5*(ff(a) + ff(b))
  __pyx_v_sumy = (__pyx_v_sumy + (0.5 * (__pyx_f_46_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a_ff(__pyx_v_a) + __pyx_f_46_cython_magic_bc2a61d8ea0fe6a9b418b8ea7c73bd3a_ff(__pyx_v_b))));
+15:     return sumy*h
  __Pyx_XDECREF(__pyx_r);
  __pyx_t_5 = PyFloat_FromDouble((__pyx_v_sumy * __pyx_v_h)); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 15, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_5);
  __pyx_r = __pyx_t_5;
  __pyx_t_5 = 0;
  goto __pyx_L0;

The cdef keyword declares a C object. Everything that follows it is therefore specified in terms of C; we are essentially writing C, but using a subset of Python's syntax rules. So, when we are creating a function cdef ff it is a C function, and is not available to you in Python.

cpdef is a hybrid declaration that creates both a C interface and a Python interface to the function.

Let's see how this performs.

In [61]:
%timeit trapez4(1, 5, 1000)
2.78 µs ± 128 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

Woof! That's a big speedup, and there's not much yellow left in the annotated code.

If you would like a very simple way of injecting types into your code with Cython, without modifying any of the code itelf, you can use the @cython.locals decorator. Note that you don't get as fast of a speedup as we have just achieved.

In [62]:
%%cython
import cython

@cython.locals(x=cython.double)
def f(x):
    return 2*x*x + 3*x + 1
     
@cython.locals(a=cython.double, b=cython.double, n=cython.int,
               h=cython.double, sumy=cython.double, i=cython.int,
               x=cython.double, func=cython.double)
def trapez5(a, b, n):
    h = (b-a)/float(n) 
    sumy = 0
    x=a
    
    for i in range(n):
        x += h
        sumy += f(x)
    sumy += 0.5*(f(a) + f(b))
    return sumy*h
In [63]:
%timeit trapez5(1, 5, 1000)
56.3 µs ± 2.71 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

If you can stand to look at it, you can peek at all the C code that is generated by Cython just to optimize this short function.

In [64]:
# %load ../examples/trapezoid.c

Due to conveneince, running Cython from IPython is a preferred way of using the language. However, if we have some legacy C/C++ code that we wish to use in Python, we can do that by writing a wrapper and calling cython from the terminal.

Here is the C code:

In [68]:
# %load ../examples/fact.h
int fact(int n) {
    if(n <= 1)
    {
        return 1;
    }
    return n * fact(n-1);
}

And here is the Cython wrapper. Cython code is stored in files with a .pyx extension.

In [70]:
# %load ../examples/fact.pyx
cdef extern from "fact.h":
    int _fact "fact"(int)

def fact(int n):
    return _fact(n)
In [71]:
!cython ../examples/fact.pyx
/home/fonnesbeck/anaconda3/envs/dev/lib/python3.6/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/fonnesbeck/repos/Bios8366/examples/fact.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)

Now we can compile the extension.

In [72]:
import os, sys
os.chdir('../examples')

if sys.platform == "darwin":
    !gcc -Wall -fno-strict-aliasing -static -undefined dynamic_lookup \
    -bundle -arch x86_64 \
    -I$HOME/anaconda3/include/python3.6m \
    -o fact.so fact.c
else:
    !gcc -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing \
    -I$HOME/anaconda3/include/python3.7m \
    -o fact.so fact.c

os.chdir('../notebooks')
In [73]:
!cp ../examples/fact.so .
In [74]:
import fact

fact.fact(5)
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-74-ba9e3c8d226c> in <module>
----> 1 import fact
      2 
      3 fact.fact(5)

ImportError: /home/fonnesbeck/repos/Bios8366/notebooks/fact.so: undefined symbol: PyInterpreterState_GetID

Using lists and arrays in Cython

The above example used only scalar variables. When we have vector-valued data, we need to declare the appropriate types. Here's a simple example, using a function that calculates the Euclidean distance between two arrays:

In [75]:
def euclidean(x, y):
    x = np.array(x)
    y = np.array(y)
    return np.sqrt(((x - y) ** 2).sum())
In [76]:
%timeit euclidean(np.random.randn(10), np.random.randn(10))
13.3 µs ± 671 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In order to get a speedup under Cython, we need to iterate over the elements of each passed array, and aggregate them manually.

In [77]:
%%cython --annotate

import cython
cimport numpy as np
from libc.math cimport sqrt

@cython.boundscheck(False)
@cython.wraparound(False)
def euclidean2(np.ndarray[np.float64_t, ndim=1] x, 
               np.ndarray[np.float64_t, ndim=1] y):
    cdef: 
        double diff
        int i
    diff = 0
    for i in range(x.shape[0]):
        diff += (x[i] - y[i])**2
    return sqrt(diff)
Out[77]:
Cython: _cython_magic_0a9b7f37e79624f414a3fce164851071.pyx

Generated by Cython 0.29.2

Yellow lines hint at Python interaction.
Click on a line that starts with a "+" to see the C code that Cython generated for it.

 01: 
+02: import cython
  __pyx_t_1 = __Pyx_PyDict_NewPresized(0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  if (PyDict_SetItem(__pyx_d, __pyx_n_s_test, __pyx_t_1) < 0) __PYX_ERR(0, 2, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
 03: cimport numpy as np
 04: from libc.math cimport sqrt
 05: 
 06: @cython.boundscheck(False)
 07: @cython.wraparound(False)
+08: def euclidean2(np.ndarray[np.float64_t, ndim=1] x,
/* Python wrapper */
static PyObject *__pyx_pw_46_cython_magic_0a9b7f37e79624f414a3fce164851071_1euclidean2(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
static PyMethodDef __pyx_mdef_46_cython_magic_0a9b7f37e79624f414a3fce164851071_1euclidean2 = {"euclidean2", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_46_cython_magic_0a9b7f37e79624f414a3fce164851071_1euclidean2, METH_VARARGS|METH_KEYWORDS, 0};
static PyObject *__pyx_pw_46_cython_magic_0a9b7f37e79624f414a3fce164851071_1euclidean2(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
  PyArrayObject *__pyx_v_x = 0;
  PyArrayObject *__pyx_v_y = 0;
  PyObject *__pyx_r = 0;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("euclidean2 (wrapper)", 0);
  {
    static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_x,&__pyx_n_s_y,0};
    PyObject* values[2] = {0,0};
    if (unlikely(__pyx_kwds)) {
      Py_ssize_t kw_args;
      const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
      switch (pos_args) {
        case  2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
        CYTHON_FALLTHROUGH;
        case  1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
        CYTHON_FALLTHROUGH;
        case  0: break;
        default: goto __pyx_L5_argtuple_error;
      }
      kw_args = PyDict_Size(__pyx_kwds);
      switch (pos_args) {
        case  0:
        if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_x)) != 0)) kw_args--;
        else goto __pyx_L5_argtuple_error;
        CYTHON_FALLTHROUGH;
        case  1:
        if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_y)) != 0)) kw_args--;
        else {
          __Pyx_RaiseArgtupleInvalid("euclidean2", 1, 2, 2, 1); __PYX_ERR(0, 8, __pyx_L3_error)
        }
      }
      if (unlikely(kw_args > 0)) {
        if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "euclidean2") < 0)) __PYX_ERR(0, 8, __pyx_L3_error)
      }
    } else if (PyTuple_GET_SIZE(__pyx_args) != 2) {
      goto __pyx_L5_argtuple_error;
    } else {
      values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
      values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
    }
    __pyx_v_x = ((PyArrayObject *)values[0]);
    __pyx_v_y = ((PyArrayObject *)values[1]);
  }
  goto __pyx_L4_argument_unpacking_done;
  __pyx_L5_argtuple_error:;
  __Pyx_RaiseArgtupleInvalid("euclidean2", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 8, __pyx_L3_error)
  __pyx_L3_error:;
  __Pyx_AddTraceback("_cython_magic_0a9b7f37e79624f414a3fce164851071.euclidean2", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __Pyx_RefNannyFinishContext();
  return NULL;
  __pyx_L4_argument_unpacking_done:;
  if (unlikely(!__Pyx_ArgTypeTest(((PyObject *)__pyx_v_x), __pyx_ptype_5numpy_ndarray, 1, "x", 0))) __PYX_ERR(0, 8, __pyx_L1_error)
  if (unlikely(!__Pyx_ArgTypeTest(((PyObject *)__pyx_v_y), __pyx_ptype_5numpy_ndarray, 1, "y", 0))) __PYX_ERR(0, 9, __pyx_L1_error)
  __pyx_r = __pyx_pf_46_cython_magic_0a9b7f37e79624f414a3fce164851071_euclidean2(__pyx_self, __pyx_v_x, __pyx_v_y);

  /* function exit code */
  goto __pyx_L0;
  __pyx_L1_error:;
  __pyx_r = NULL;
  __pyx_L0:;
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}

static PyObject *__pyx_pf_46_cython_magic_0a9b7f37e79624f414a3fce164851071_euclidean2(CYTHON_UNUSED PyObject *__pyx_self, PyArrayObject *__pyx_v_x, PyArrayObject *__pyx_v_y) {
  double __pyx_v_diff;
  int __pyx_v_i;
  __Pyx_LocalBuf_ND __pyx_pybuffernd_x;
  __Pyx_Buffer __pyx_pybuffer_x;
  __Pyx_LocalBuf_ND __pyx_pybuffernd_y;
  __Pyx_Buffer __pyx_pybuffer_y;
  PyObject *__pyx_r = NULL;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("euclidean2", 0);
  __pyx_pybuffer_x.pybuffer.buf = NULL;
  __pyx_pybuffer_x.refcount = 0;
  __pyx_pybuffernd_x.data = NULL;
  __pyx_pybuffernd_x.rcbuffer = &__pyx_pybuffer_x;
  __pyx_pybuffer_y.pybuffer.buf = NULL;
  __pyx_pybuffer_y.refcount = 0;
  __pyx_pybuffernd_y.data = NULL;
  __pyx_pybuffernd_y.rcbuffer = &__pyx_pybuffer_y;
  {
    __Pyx_BufFmt_StackElem __pyx_stack[1];
    if (unlikely(__Pyx_GetBufferAndValidate(&__pyx_pybuffernd_x.rcbuffer->pybuffer, (PyObject*)__pyx_v_x, &__Pyx_TypeInfo_nn___pyx_t_5numpy_float64_t, PyBUF_FORMAT| PyBUF_STRIDES, 1, 0, __pyx_stack) == -1)) __PYX_ERR(0, 8, __pyx_L1_error)
  }
  __pyx_pybuffernd_x.diminfo[0].strides = __pyx_pybuffernd_x.rcbuffer->pybuffer.strides[0]; __pyx_pybuffernd_x.diminfo[0].shape = __pyx_pybuffernd_x.rcbuffer->pybuffer.shape[0];
  {
    __Pyx_BufFmt_StackElem __pyx_stack[1];
    if (unlikely(__Pyx_GetBufferAndValidate(&__pyx_pybuffernd_y.rcbuffer->pybuffer, (PyObject*)__pyx_v_y, &__Pyx_TypeInfo_nn___pyx_t_5numpy_float64_t, PyBUF_FORMAT| PyBUF_STRIDES, 1, 0, __pyx_stack) == -1)) __PYX_ERR(0, 8, __pyx_L1_error)
  }
  __pyx_pybuffernd_y.diminfo[0].strides = __pyx_pybuffernd_y.rcbuffer->pybuffer.strides[0]; __pyx_pybuffernd_y.diminfo[0].shape = __pyx_pybuffernd_y.rcbuffer->pybuffer.shape[0];
/* … */
  /* function exit code */
  __pyx_L1_error:;
  __Pyx_XDECREF(__pyx_t_6);
  { PyObject *__pyx_type, *__pyx_value, *__pyx_tb;
    __Pyx_PyThreadState_declare
    __Pyx_PyThreadState_assign
    __Pyx_ErrFetch(&__pyx_type, &__pyx_value, &__pyx_tb);
    __Pyx_SafeReleaseBuffer(&__pyx_pybuffernd_x.rcbuffer->pybuffer);
    __Pyx_SafeReleaseBuffer(&__pyx_pybuffernd_y.rcbuffer->pybuffer);
  __Pyx_ErrRestore(__pyx_type, __pyx_value, __pyx_tb);}
  __Pyx_AddTraceback("_cython_magic_0a9b7f37e79624f414a3fce164851071.euclidean2", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __pyx_r = NULL;
  goto __pyx_L2;
  __pyx_L0:;
  __Pyx_SafeReleaseBuffer(&__pyx_pybuffernd_x.rcbuffer->pybuffer);
  __Pyx_SafeReleaseBuffer(&__pyx_pybuffernd_y.rcbuffer->pybuffer);
  __pyx_L2:;
  __Pyx_XGIVEREF(__pyx_r);
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}
/* … */
  __pyx_tuple__8 = PyTuple_Pack(4, __pyx_n_s_x, __pyx_n_s_y, __pyx_n_s_diff, __pyx_n_s_i); if (unlikely(!__pyx_tuple__8)) __PYX_ERR(0, 8, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_tuple__8);
  __Pyx_GIVEREF(__pyx_tuple__8);
/* … */
  __pyx_t_1 = PyCFunction_NewEx(&__pyx_mdef_46_cython_magic_0a9b7f37e79624f414a3fce164851071_1euclidean2, NULL, __pyx_n_s_cython_magic_0a9b7f37e79624f414); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 8, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  if (PyDict_SetItem(__pyx_d, __pyx_n_s_euclidean2, __pyx_t_1) < 0) __PYX_ERR(0, 8, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
 09:                np.ndarray[np.float64_t, ndim=1] y):
 10:     cdef:
 11:         double diff
 12:         int i
+13:     diff = 0
  __pyx_v_diff = 0.0;
+14:     for i in range(x.shape[0]):
  __pyx_t_1 = (__pyx_v_x->dimensions[0]);
  __pyx_t_2 = __pyx_t_1;
  for (__pyx_t_3 = 0; __pyx_t_3 < __pyx_t_2; __pyx_t_3+=1) {
    __pyx_v_i = __pyx_t_3;
+15:         diff += (x[i] - y[i])**2
    __pyx_t_4 = __pyx_v_i;
    __pyx_t_5 = __pyx_v_i;
    __pyx_v_diff = (__pyx_v_diff + pow(((*__Pyx_BufPtrStrided1d(__pyx_t_5numpy_float64_t *, __pyx_pybuffernd_x.rcbuffer->pybuffer.buf, __pyx_t_4, __pyx_pybuffernd_x.diminfo[0].strides)) - (*__Pyx_BufPtrStrided1d(__pyx_t_5numpy_float64_t *, __pyx_pybuffernd_y.rcbuffer->pybuffer.buf, __pyx_t_5, __pyx_pybuffernd_y.diminfo[0].strides))), 2.0));
  }
+16:     return sqrt(diff)
  __Pyx_XDECREF(__pyx_r);
  __pyx_t_6 = PyFloat_FromDouble(sqrt(__pyx_v_diff)); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 16, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_6);
  __pyx_r = __pyx_t_6;
  __pyx_t_6 = 0;
  goto __pyx_L0;
In [78]:
%timeit euclidean2(np.random.randn(10), np.random.randn(10))
4.69 µs ± 287 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

The decorators for trapez5 are compiler directives that alter the behavior of Cython code. Setting boundscheck to False removes boundary checking for indexing operations, forcing us to ensure that we do not try to index arrays using index vlaues that are out of bounds. When we set wraparound to False, Cython will not support negative indexes, as is the case with Python. While these directives may increase the speed of our code, it can be dangerous; if we do not ensure that we index our arrays properly, it may cause segmentation faults or data corruption.

The full set of compiler directives are described in the Cython docs.

Here is the same code using lists instead of NumPy arrays:

In [79]:
%%cython --annotate

from libc.math cimport sqrt

def euclidean3(list x, list y):
    cdef: 
        double diff
        int i
    diff = 0
    for i in range(len(x)):
        diff += (x[i] - y[i])**2
    return sqrt(diff)
Out[79]:
Cython: _cython_magic_7e5c94ff564e970b46eec387ab4af637.pyx

Generated by Cython 0.29.2

Yellow lines hint at Python interaction.
Click on a line that starts with a "+" to see the C code that Cython generated for it.

 01: 
 02: from libc.math cimport sqrt
 03: 
+04: def euclidean3(list x, list y):
/* Python wrapper */
static PyObject *__pyx_pw_46_cython_magic_7e5c94ff564e970b46eec387ab4af637_1euclidean3(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/
static PyMethodDef __pyx_mdef_46_cython_magic_7e5c94ff564e970b46eec387ab4af637_1euclidean3 = {"euclidean3", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_46_cython_magic_7e5c94ff564e970b46eec387ab4af637_1euclidean3, METH_VARARGS|METH_KEYWORDS, 0};
static PyObject *__pyx_pw_46_cython_magic_7e5c94ff564e970b46eec387ab4af637_1euclidean3(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
  PyObject *__pyx_v_x = 0;
  PyObject *__pyx_v_y = 0;
  PyObject *__pyx_r = 0;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("euclidean3 (wrapper)", 0);
  {
    static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_x,&__pyx_n_s_y,0};
    PyObject* values[2] = {0,0};
    if (unlikely(__pyx_kwds)) {
      Py_ssize_t kw_args;
      const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args);
      switch (pos_args) {
        case  2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
        CYTHON_FALLTHROUGH;
        case  1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
        CYTHON_FALLTHROUGH;
        case  0: break;
        default: goto __pyx_L5_argtuple_error;
      }
      kw_args = PyDict_Size(__pyx_kwds);
      switch (pos_args) {
        case  0:
        if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_x)) != 0)) kw_args--;
        else goto __pyx_L5_argtuple_error;
        CYTHON_FALLTHROUGH;
        case  1:
        if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_y)) != 0)) kw_args--;
        else {
          __Pyx_RaiseArgtupleInvalid("euclidean3", 1, 2, 2, 1); __PYX_ERR(0, 4, __pyx_L3_error)
        }
      }
      if (unlikely(kw_args > 0)) {
        if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "euclidean3") < 0)) __PYX_ERR(0, 4, __pyx_L3_error)
      }
    } else if (PyTuple_GET_SIZE(__pyx_args) != 2) {
      goto __pyx_L5_argtuple_error;
    } else {
      values[0] = PyTuple_GET_ITEM(__pyx_args, 0);
      values[1] = PyTuple_GET_ITEM(__pyx_args, 1);
    }
    __pyx_v_x = ((PyObject*)values[0]);
    __pyx_v_y = ((PyObject*)values[1]);
  }
  goto __pyx_L4_argument_unpacking_done;
  __pyx_L5_argtuple_error:;
  __Pyx_RaiseArgtupleInvalid("euclidean3", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 4, __pyx_L3_error)
  __pyx_L3_error:;
  __Pyx_AddTraceback("_cython_magic_7e5c94ff564e970b46eec387ab4af637.euclidean3", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __Pyx_RefNannyFinishContext();
  return NULL;
  __pyx_L4_argument_unpacking_done:;
  if (unlikely(!__Pyx_ArgTypeTest(((PyObject *)__pyx_v_x), (&PyList_Type), 1, "x", 1))) __PYX_ERR(0, 4, __pyx_L1_error)
  if (unlikely(!__Pyx_ArgTypeTest(((PyObject *)__pyx_v_y), (&PyList_Type), 1, "y", 1))) __PYX_ERR(0, 4, __pyx_L1_error)
  __pyx_r = __pyx_pf_46_cython_magic_7e5c94ff564e970b46eec387ab4af637_euclidean3(__pyx_self, __pyx_v_x, __pyx_v_y);

  /* function exit code */
  goto __pyx_L0;
  __pyx_L1_error:;
  __pyx_r = NULL;
  __pyx_L0:;
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}

static PyObject *__pyx_pf_46_cython_magic_7e5c94ff564e970b46eec387ab4af637_euclidean3(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_x, PyObject *__pyx_v_y) {
  double __pyx_v_diff;
  int __pyx_v_i;
  PyObject *__pyx_r = NULL;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("euclidean3", 0);
/* … */
  /* function exit code */
  __pyx_L1_error:;
  __Pyx_XDECREF(__pyx_t_4);
  __Pyx_XDECREF(__pyx_t_5);
  __Pyx_XDECREF(__pyx_t_6);
  __Pyx_XDECREF(__pyx_t_7);
  __Pyx_AddTraceback("_cython_magic_7e5c94ff564e970b46eec387ab4af637.euclidean3", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __pyx_r = NULL;
  __pyx_L0:;
  __Pyx_XGIVEREF(__pyx_r);
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}
/* … */
  __pyx_tuple_ = PyTuple_Pack(4, __pyx_n_s_x, __pyx_n_s_y, __pyx_n_s_diff, __pyx_n_s_i); if (unlikely(!__pyx_tuple_)) __PYX_ERR(0, 4, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_tuple_);
  __Pyx_GIVEREF(__pyx_tuple_);
/* … */
  __pyx_t_1 = PyCFunction_NewEx(&__pyx_mdef_46_cython_magic_7e5c94ff564e970b46eec387ab4af637_1euclidean3, NULL, __pyx_n_s_cython_magic_7e5c94ff564e970b46); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 4, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_1);
  if (PyDict_SetItem(__pyx_d, __pyx_n_s_euclidean3, __pyx_t_1) < 0) __PYX_ERR(0, 4, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
 05:     cdef:
 06:         double diff
 07:         int i
+08:     diff = 0
  __pyx_v_diff = 0.0;
+09:     for i in range(len(x)):
  if (unlikely(__pyx_v_x == Py_None)) {
    PyErr_SetString(PyExc_TypeError, "object of type 'NoneType' has no len()");
    __PYX_ERR(0, 9, __pyx_L1_error)
  }
  __pyx_t_1 = PyList_GET_SIZE(__pyx_v_x); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 9, __pyx_L1_error)
  __pyx_t_2 = __pyx_t_1;
  for (__pyx_t_3 = 0; __pyx_t_3 < __pyx_t_2; __pyx_t_3+=1) {
    __pyx_v_i = __pyx_t_3;
+10:         diff += (x[i] - y[i])**2
    __pyx_t_4 = PyFloat_FromDouble(__pyx_v_diff); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 10, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_4);
    if (unlikely(__pyx_v_x == Py_None)) {
      PyErr_SetString(PyExc_TypeError, "'NoneType' object is not subscriptable");
      __PYX_ERR(0, 10, __pyx_L1_error)
    }
    __pyx_t_5 = __Pyx_GetItemInt_List(__pyx_v_x, __pyx_v_i, int, 1, __Pyx_PyInt_From_int, 1, 1, 1); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 10, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_5);
    if (unlikely(__pyx_v_y == Py_None)) {
      PyErr_SetString(PyExc_TypeError, "'NoneType' object is not subscriptable");
      __PYX_ERR(0, 10, __pyx_L1_error)
    }
    __pyx_t_6 = __Pyx_GetItemInt_List(__pyx_v_y, __pyx_v_i, int, 1, __Pyx_PyInt_From_int, 1, 1, 1); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 10, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_6);
    __pyx_t_7 = PyNumber_Subtract(__pyx_t_5, __pyx_t_6); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 10, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_7);
    __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
    __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
    __pyx_t_6 = PyNumber_Power(__pyx_t_7, __pyx_int_2, Py_None); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 10, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_6);
    __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0;
    __pyx_t_7 = PyNumber_InPlaceAdd(__pyx_t_4, __pyx_t_6); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 10, __pyx_L1_error)
    __Pyx_GOTREF(__pyx_t_7);
    __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0;
    __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
    __pyx_t_8 = __pyx_PyFloat_AsDouble(__pyx_t_7); if (unlikely((__pyx_t_8 == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 10, __pyx_L1_error)
    __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0;
    __pyx_v_diff = __pyx_t_8;
  }
+11:     return sqrt(diff)
  __Pyx_XDECREF(__pyx_r);
  __pyx_t_7 = PyFloat_FromDouble(sqrt(__pyx_v_diff)); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 11, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_7);
  __pyx_r = __pyx_t_7;
  __pyx_t_7 = 0;
  goto __pyx_L0;
In [80]:
%timeit euclidean3(np.random.randn(10).tolist(), np.random.randn(10).tolist())
6.87 µs ± 2.63 µs per loop (mean ± std. dev. of 7 runs, 100000 loops each)

pyximport

If we have some Cython source code, we can use pyximport to directly import it as if it were a Python module.

In [81]:
import pyximport

# Move source into current directory
!cp ../examples/trapezoid.pyx .

# Allow it to use Python's import mechanism
pyximport.install()

from trapezoid import trapez as trapez_pyx
trapez_pyx(1, 10, 10) 
/home/fonnesbeck/anaconda3/envs/dev/lib/python3.6/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/fonnesbeck/repos/Bios8366/notebooks/trapezoid.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)
Out[81]:
1033.8299560546875

In other words, it treats .pyx files as if they were .py files. This includes detecting changes in the source file,, if any, and recompiling it as necessary before importing.

Benchmark example: Gibbs sampling

Let's see if we can use Cython to speed up MCMC.

Gibbs sampler for function:

$$f(x,y) = x x^2 \exp(-xy^2 - y^2 + 2y - 4x)$$

using conditional distributions:

$$x|y \sim Gamma(3, y^2 +4)$$$$y|x \sim Normal(\frac{1}{1+x}, \frac{1}{2(1+x)})$$

Here is the pure Python implementation:

In [82]:
from numpy import zeros, random, sqrt
gamma = random.gamma
normal = random.normal

def pygibbs(N=20000, thin=200):
    mat = zeros((N,2))
    x,y = mat[0]
    for i in range(N):
        for j in range(thin):
            x = gamma(3, y**2 + 4)
            y = normal(1./(x+1), 1./sqrt(2*(x+1)))
        mat[i] = x,y

    return mat
In [83]:
%timeit pygibbs(1000, 10)
86.7 ms ± 9.1 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Unchanged, compiling this code with Cython results in a slight improvement in speed.

In [84]:
%%cython

from numpy import zeros, random, sqrt
gamma = random.gamma
normal = random.normal

def cygibbs(N=20000, thin=200):
    mat = zeros((N,2))
    x,y = mat[0]
    for i in range(N):
        for j in range(thin):
            x = gamma(3, y**2 + 4)
            y = normal(1./(x+1), 1./sqrt(2*(x+1)))
        mat[i] = x,y

    return mat
In [85]:
%timeit cygibbs(1000, 10)
76.3 ms ± 2.92 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Now, for some type declarations:

In [86]:
%%cython

from numpy import zeros, random, sqrt
from numpy cimport *
gamma = random.gamma
normal = random.normal

def cygibbs2(int N=20000, int thin=200):
    cdef: 
        ndarray[float64_t, ndim=2] mat = zeros((N,2))
        float64_t x,y = 0
        int i,j
    for i in range(N):
        for j in range(thin):
            x = gamma(3, y**2 + 4)
            y = normal(1./(x+1), 1./sqrt(2*(x+1)))
        mat[i] = x,y

    return mat
In [87]:
%timeit cygibbs2(1000, 10)
80.7 ms ± 6.31 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

A full-blown "Cythonization" involves using GSL's random number generators, and giving Cython a few more instructions:

If you are using Homebrew on OS X, you can install GSL using "brew"

!brew install gsl
In [89]:
%%cython -lm -lgsl -lgslcblas

cimport cython
import numpy as np
from numpy cimport *

cdef extern from "math.h":
    double sqrt(double) 
  
cdef extern from "gsl/gsl_rng.h":
    ctypedef struct gsl_rng_type
    ctypedef struct gsl_rng

    gsl_rng_type *gsl_rng_mt19937
    gsl_rng *gsl_rng_alloc(gsl_rng_type * T) nogil
  
cdef extern from "gsl/gsl_randist.h":
    double gamma "gsl_ran_gamma"(gsl_rng * r,double,double)
    double gaussian "gsl_ran_gaussian"(gsl_rng * r,double)
  
cdef gsl_rng *r = gsl_rng_alloc(gsl_rng_mt19937)

@cython.wraparound(False)
@cython.boundscheck(False)
def gibbs(int N=20000,int thin=500):
    cdef: 
        double x=0
        double y=0
        int i, j
        ndarray[float64_t, ndim=2] samples

    samples = np.empty((N,thin))
    for i from 0 <= i < N:
        for j from 0 <= j < thin:
            x = gamma(r,3,1.0/(y*y+4))
            y = gaussian(r,1.0/sqrt(x+1))
        samples[i,0] = x
        samples[i,1] = y
    return samples
In [90]:
%timeit gibbs(1000, 10)
1.52 ms ± 95.3 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Numba

Cython precompiles parts of Python code before running. Another approach is Just-in-Time (JIT) compilation. Numba is a compiler that runs Python code through an LLVM compiler to produce optimized bytecode for fast execution. Numba does not require a C/C++ compiler on your machine.

Numba's lone API is a decorator.

The @jit decorator runs the decorated function through bytecode analysis and the function arguments through a type inference engine, and generates an intermediate representation of your code, which is then passed to LLVM for compilation to bytecode.

In [91]:
from numba import jit, autojit
In [92]:
def nfibonacci(size):
    F = np.empty(size, 'int')
    a, b = 0, 1
    for i in range(size):
        F[i] = a
        a, b = b, a + b
    return F
In [93]:
%timeit nfibonacci(50)
13.2 µs ± 2.35 µs per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [94]:
@jit
def nfibonacci(size):
    F = np.empty(size, 'int')
    a, b = 0, 1
    for i in range(size):
        F[i] = a
        a, b = b, a + b
    return F
In [95]:
%timeit nfibonacci(50)
The slowest run took 9.44 times longer than the fastest. This could mean that an intermediate result is being cached.
19.7 µs ± 22.4 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

Numba is able to compile separate specializations depending on the input types.

If you want fine-grained control over types chosen by the compiler, you can tell Numba the function signature (types) to expect.

In [96]:
from numba import int32

@jit(int32[:](int32))
def nfibonacci(size):
    F = np.empty(size, 'int')
    a, b = 0, 1
    for i in range(size):
        F[i] = a
        a, b = b, a + b
    return F
In [97]:
%timeit nfibonacci(50)
4.61 µs ± 290 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

Compilation is deferred until the first function execution. Numba will infer the argument types at call time, and generate optimized code based on this information.

In [98]:
def pairwise_python(X):
    M = X.shape[0]
    N = X.shape[1]
    D = np.empty((M, M), dtype=np.float)
    for i in range(M):
        for j in range(M):
            d = 0.0
            for k in range(N):
                tmp = X[i, k] - X[j, k]
                d += tmp * tmp
            D[i, j] = np.sqrt(d)
    return D

X = np.random.random((1000, 3))

%timeit pairwise_python(X)
6.49 s ± 605 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
In [99]:
@jit
def npairwise_python(X):
    M = X.shape[0]
    N = X.shape[1]
    D = np.empty((M, M), dtype=np.float)
    for i in range(M):
        for j in range(M):
            d = 0.0
            for k in range(N):
                tmp = X[i, k] - X[j, k]
                d += tmp * tmp
            D[i, j] = np.sqrt(d)
    return D

%timeit npairwise_python(X)
4.22 ms ± 161 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

Numba-compiled functions can call other compiled functions. In some situations, the optimizer may even inline the function in the machine code code.

In [100]:
def square(x):
    return x ** 2

def hypot(x, y):
    return np.sqrt(square(x) + square(y))
In [101]:
%timeit hypot(10, 8)
2.88 µs ± 261 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [102]:
@jit
def nsquare(x):
    return x ** 2

@jit
def nhypot(x, y):
    return np.sqrt(nsquare(x) + nsquare(y))
In [103]:
%timeit nhypot(10, 8)
The slowest run took 8.08 times longer than the fastest. This could mean that an intermediate result is being cached.
3.17 µs ± 3.78 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

Numba can compile most NumPy functions, as well as generators.

Numba cannot compile things like lists, sets, dictionaries (tuples are compiled), comprehensions, and string operations, so there will be no speedup for these.

As with all performance tools, the best strategy is not to apply the @jit decorator all over your code, but to use Python's profiling tools to identify "hotspots" in your program, and selectively apply @jit.

Exercise: Fixed-point algorithm speedup

Use the method of your choice to speed up the execution of the fixed-point algorithm that was implemented in the optimization lecture (you will find it in the examples directory).

In [104]:
# Write your answer here