Preliminaries

Import the package and choose a movie directory to work with.

If you don't have a movie, just give a directory for an example movie. We'll fill it with fake image files below.

from path import path # Used in examples
import runtrackpy

movdir = path('bigtracks-demo-movie').abspath()
print movdir
movdir.makedirs_p()
frames_extension = 'PNG'
frames_pattern = '*.' + frames_extension

/Users/nkeim/projects/runtrackpy/examples/bigtracks-demo-movie

Making example data

If you have your own data and want to try it out, skip this part!

bigtracks contains a function for making fake images with random particle motion. We'll save a series of them to the movie directory chosen above.

import runtrackpy.test
import scipy.misc
for framenumber in range(20):
    x, y, img = runtrackpy.test.fake_image(framenumber)
    scipy.misc.imsave(movdir / 'example_%.4i.%s' % (framenumber, frames_extension), img)

Setting parameters

What parameters are accepted? You can do this, or run

runtrackpy.track?

print runtrackpy.track.__doc__

Utilities to drive the Caswell Python tracking code (as modified by NCK).

Functions of note:
    identify_frame() previews feature identification.
    track2disk() implements a complete tracking workflow.

The 'params' dictionaries required below have the following options:
    For identification:
        'identmod': Name of module in which to find identification function.
            Default: This one.
        'identfunc': Name of identification function. 
            Default: basic bandpass-supbixel algorithm.
        'maxgray': Maximum grayscale value of images (default 0 -> best guess)
        'bright': 0 -> dark particles on light background (default); 1 -> inverse
        [Depending on 'identfunc', the following parameters may be different.]
        'featsize': Expected particle feature radius
        'bphigh': Scale, in pixels, for smoothing images and making them more lumpy
        'maxrg': Cutoff for particle radius of gyration --- how extended particle is
        'threshold': Ignore pixels smaller than this value
        'merge_cutoff': Merge features that are too close to each other.
    For tracking:
        'maxdisp': Radius of region in which to look for a particle in the next frame.
            Set too high, and the algorithm will be overwhelmed with possible matches.
        'memory': How many frames a particle can skip, and still be identified if it has
            not moved past 'maxdisp'.

The 'window' dictionaires limit where and when to look for particles. 
Items 'xmin', 'xmax', 'ymin', and 'ymax' set the spatial limits. 'firstframe' 
and 'lastframe' set the range of frames to track, inclusive; the first frame 
is numbered 1. All values are optional. 

If you are using the bigtracks.run module, these can be stored in "bigtracking.ini" 
and "window.ini" in each movie directory.

Simple example of a do-it-yourself tracking pipeline:
    params = dict(featsize=3, bphigh=0.7, maxrg=100, maxdisp=3)
    mytracks = list(link_dataframes(feature_iter(enumerate(allfiles), params), params))
    # 'mytracks' can then be combined into a single DataFrame with the append() method.
See track2disk() for something much more user-friendly.

OK, let's load a sample frame, set some basic parameters, and try them out.

imgfiles = movdir.glob(frames_pattern)
im = imread(imgfiles[0])
def crop():
    pass
    # If you wish, set a restricted view here, so you can zoom in on details.
    #xlim(0, 240)
    #ylim(0, 240)
imshow(im)
gray()
colorbar()
crop()

gcf().set_size_inches(10, 10)
params = dict(bright=1, featsize=5, bphigh=2, threshold=0.5, maxdisp=3 * sqrt(8))

features = runtrackpy.track.identify_frame(im, params)

imshow(im)
plot(features.x, features.y, 'r+', markersize=10)
axis('image')
crop()

We can check things like the quality of subpixel resolution (insufficient statistics in the example dataset)

hist((features.x % 1, features.y % 1));

Processing the movie

In order to speed things up, we do only the first 2 frames.

First, we need to clear any existing tracks file. track2disk() will refuse to overwrite an existing file.

outputfile = movdir / 'bigtracks.h5'
if outputfile.exists():
    outputfile.remove()

runtrackpy.track.track2disk(imgfiles, outputfile, 
    params, 
    #selectframes=range(1, 3), # To speed things up, we could do just 2 frames
    progress=True)

10 particles in frame 20 of 20: /Users/nkeim/projects/runtrackpy/examples/bigtracks-demo-movie/example_0019.PNG

Reading

So that the tracks-reading code can be released under a more flexible software license, it is in the pantracks module.

All read-only access is with BigTracks objects:

import pantracks
bt = pantracks.BigTracks(movdir / 'bigtracks.h5')

Basics

Number of frames in the file

bt.maxframe()

20

Assuming the file is not too big, it may be easiest to just get the whole thing.

bt.get_all()

<class 'pandas.core.frame.DataFrame'>
Int64Index: 200 entries, 0 to 199
Data columns (total 6 columns):
frame        200  non-null values
particle     200  non-null values
x            200  non-null values
y            200  non-null values
intensity    200  non-null values
rg2          200  non-null values
dtypes: float32(6)

However, the database is indexed by frame number and particle ID. This means that requesting data based on one (or both) of these quantities is very efficient, because the computer already knows where to look in the file, and does not have to read the whole thing. Therefore, for larger datasets it makes sense to load single frames as you need them:

ftr = bt.get_frame(1)
print ftr.frame.values[0] # Which frame was this, again?
ftr.head()

1.0

	frame	particle	x	y	intensity	rg2
0	1	0	211.976166	32.355923	20.259562	40.956638
1	1	1	52.189362	49.312527	20.071190	40.645416
2	1	2	152.541245	70.267784	20.394745	42.023090
3	1	3	70.209854	91.285034	20.431799	41.435551
4	1	4	187.088959	109.638260	20.097969	40.736820

Or, if you're in a hurry,

ftr = bt[1]
print ftr.frame.values[0]

1.0

You can iterate over all frames in the file thusly:

for fnum in bt.framerange():
    ftr = bt.get_frame(fnum)
    print fnum, len(ftr)

1.0 10
2.0 10
3.0 10
4.0 10
5.0 10
6.0 10
7.0 10
8.0 10
9.0 10
10.0 10
11.0 10
12.0 10
13.0 10
14.0 10
15.0 10
16.0 10
17.0 10
18.0 10
19.0 10
20.0 10

Advanced

You can make an arbitrary query to the underlying pytables database, which may be more efficient than loading all the data and filtering it yourself.

ftr = bt.query('(x < xmax) & (y < ymax)', {'xmax': 100, 'ymax': 100})
ftr.x.max(), ftr.y.max(), ftr.frame.unique()

(72.738312,
 92.832596,
 array([  1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.,   9.,  10.,  11.,
        12.,  13.,  14.,  15.,  16.,  17.,  18.,  19.,  20.]))

Finally, the pytables Table object is available if you are adventurous, enabling a ton of advanced capabilities. The following is equivalent to calling query_tracks in the previous example, only now we're asking for certain particles.

import pandas
with bt:
    ftr = pandas.DataFrame(bt.table.readWhere('(particle >= 2) & (particle <= 5)'))
ftr.particle.min(), ftr.particle.max(), ftr.frame.unique()

(2.0,
 5.0,
 array([  1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.,   9.,  10.,  11.,
        12.,  13.,  14.,  15.,  16.,  17.,  18.,  19.,  20.]))

Using the BigTracks object as a context, as already seen above, means that the file is opened just once, and so it should speed up "batch" operations:

%%timeit
for fnum in range(10):
    ftr = bt.get_frame(1)

10 loops, best of 3: 150 ms per loop

%%timeit
with bt:
    for i in range(10):
        ftr = bt.get_frame(1)

10 loops, best of 3: 20.9 ms per loop

In all cases, the file is never left open. This avoids the occurrence of Bad Things if its contents are subsequently changed.

Evaluating tracks quality

How many of the particles in the first frame make it to the nth frame? bigtracks.read makes this more convenient. For the example dataset it's pretty boring.

btq = pantracks.bigtracks.compute_quality(bt, frame_interval=1)
btq.head()

	N	Nconserved
1	10	10
2	10	10
3	10	10
4	10	10
5	10	10

The plot shows fluctuations in the number of features identified in each frame, as well as the fraction of particles lost since the first frame.

pantracks.bigtracks.plot_quality(btq)