Summary. This notebook describes the HDF5 file format used to store simulation results. It is also an example on how to use the pytables API to navigate through the HDF5 file.
%run -i load_bromo.py
C:\Data\Antonio\software\src\pybromo Current software revision: 4265047 Fix timestamps storage WARNING -> Uncommitted changes: M brownian.py MM "notebooks/PyBroMo - Reference - Data format and internals.ipynb" M storage.py
# Simulation time step
t_step = 0.5e-6 # seconds
# Time duration of the simulation
t_max = .3 # seconds
# Particles definition
P = gen_particles(15, box)
# Particle simulation definition
S = ParticlesSimulation(D=D, t_step=t_step, t_max=t_max, particles=P, box=box, psf=psf)
The simulation is saved in a HDF5 file, one for each running engine. The file has the following content:
/parameters
Numeric parameters (storead as scalar arrays)
D
t_step
t_max
EID
ID
chunksize
: used for emission
and position
arraysnp
: number of particlespMol
: particles concentration in pico-MolarNon-numeric parameters (stored as group attributes)
box
: the Box()
object (stores boundaries and size)particles
: the Particles()
object (stores a list of particles' starting positions)/psf
NumericPSF()
object on a simulation reload.default_psf
: hard link to the PSF used in the simualation, used as persistent name/trajectories
emission
: 2D array of emission traces: one row per particle. Shape: (num_particles, time)emission_tot
: 1D array of emission trace: total emission from all the particles: Shape: (time)position
: 3D array of positions. Shape (num_particles, 3, time)/timestamps
rate_max
, bg_rate
and seed
.The HDF5 file handle is in S.store.data_file
after you run S.open_store()
:
S.open_store(chunksize=2**20)
# Simulate 3D diffusion and emission
S.sim_motion_em_chunk(total_emission=False, delete_pos=True)
[4428] Simulating particle 0 [4428] Simulating particle 1 [4428] Simulating particle 2 [4428] Simulating particle 3 [4428] Simulating particle 4 [4428] Simulating particle 5 [4428] Simulating particle 6 [4428] Simulating particle 7 [4428] Simulating particle 8 [4428] Simulating particle 9 [4428] Simulating particle 10 [4428] Simulating particle 11 [4428] Simulating particle 12 [4428] Simulating particle 13 [4428] Simulating particle 14
# Generate timestamps
S.sim_timestamps_em_store(max_rate=3e5, bg_rate=2e3, seed=1)
S.compact_name_core()
'7798be_D1.2e-11_15P_32pM_step0.5us'
S.compact_name_core(t_max=True)
'7798be_D1.2e-11_15P_32pM_step0.5us_t_max0.3s'
S.compact_name()
'7798be_D1.2e-11_15P_32pM_step0.5us_t_max0.3s_ID0-0'
print 'Main groups:\n'
for node in S.store.data_file.root:
print node
for n in node:
print '\t%s' % n.name
print '\t %s' % n.title
Main groups: /parameters (Group) 'Simulation parameters' D Diffusion coefficient (m^2/s) EID IPython Engine ID (int) ID Simulation ID (int) chunksize Chunksize for arrays np Number of simulated particles pico_mol Particles concentration (pM) t_max Simulation total time (s) t_step Simulation time-step (s) /psf (Group) 'PSFs used in the simulation' default_psf PSF x-z slice (PSFLab array) xz_realistic_z50_150_160_580nm_n1335_HR2 PSF x-z slice (PSFLab array) /timestamps (Group) 'Timestamps of emitted photons' ts_max_rate300kcps_bg2000cps_seed1 Simulated photon timestamps ts_max_rate300kcps_bg2000cps_seed1_par Particle number for each timestamp /trajectories (Group) 'Simulated trajectories' emission Emission trace of each particle emission_tot Summed emission trace of all the particles
group = '/parameters'
print 'Numeric attributes (nodes) in %s:\n' % group
print S.store.data_file.get_node(group)
for node in S.store.data_file.get_node(group)._f_list_nodes():
print '\t%s' % node.name
print '\t %s' % node.title
Numeric attributes (nodes) in /parameters: /parameters (Group) 'Simulation parameters' D Diffusion coefficient (m^2/s) EID IPython Engine ID (int) ID Simulation ID (int) chunksize Chunksize for arrays np Number of simulated particles pico_mol Particles concentration (pM) t_max Simulation total time (s) t_step Simulation time-step (s)
group = '/parameters'
print 'Attributes in %s:\n' % group
print S.store.data_file.get_node(group)
for attr in S.store.data_file.get_node(group)._v_attrs._f_list():
print '\t%s' % attr
print '\t %s' % type(S.store.data_file.get_node(group)._v_attrs[attr])
Attributes in /parameters: /parameters (Group) 'Simulation parameters' box <type 'instance'> particles <type 'list'>
group = '/psf'
print 'Nodes in in %s:\n' % group
print S.store.data_file.get_node(group)
for node in S.store.data_file.get_node(group)._f_list_nodes():
print '\t%s %s' % (node.name, node.shape)
print '\t %s' % node.title
Nodes in in /psf: /psf (Group) 'PSFs used in the simulation' default_psf (193, 129) PSF x-z slice (PSFLab array) xz_realistic_z50_150_160_580nm_n1335_HR2 (193L, 129L) PSF x-z slice (PSFLab array)
default_psf
attributes¶node_name = '/psf/default_psf'
node = S.store.data_file.get_node(node_name)
print "\n%s %s: '%s'" % (node.name, node.shape, node.title)
print '\n List of attributes:'
for attr in node.attrs._f_list():
print '\t%s' % attr
print "\t %s" % repr(node._v_attrs[attr])
default_psf (193, 129): 'PSF x-z slice (PSFLab array)' List of attributes: dir_ 'psf_data/' fname 'xz_realistic_z50_150_160_580nm_n1335_HR2' x_step 0.0625 z_step 0.0625
group = '/trajectories'
print 'Nodes in in %s:\n' % group
print S.store.data_file.get_node(group)
for node in S.store.data_file.get_node(group)._f_list_nodes():
print '\t%s %s' % (node.name, node.shape)
print '\t %s' % node.title
Nodes in in /trajectories: /trajectories (Group) 'Simulated trajectories' emission (15, 600000) Emission trace of each particle emission_tot (0,) Summed emission trace of all the particles
emission
attributes¶node_name = '/trajectories/emission'
node = S.store.data_file.get_node(node_name)
print "\n%s %s: '%s'" % (node.name, node.shape, node.title)
print '\n List of attributes:'
for attr in node.attrs._f_list():
print '\t%s' % attr
print "\t %s" % repr(node._v_attrs[attr])
emission (15, 600000): 'Emission trace of each particle' List of attributes: psf_fname 'xz_realistic_z50_150_160_580nm_n1335_HR2' seed 1
group = '/timestamps'
print 'Nodes in in %s:\n' % group
print S.store.data_file.get_node(group)
for node in S.store.data_file.get_node(group)._f_list_nodes():
print '\t%s' % node.name
print '\t %s' % node.title
Nodes in in /timestamps: /timestamps (Group) 'Timestamps of emitted photons' ts_max_rate300kcps_bg2000cps_seed1 Simulated photon timestamps ts_max_rate300kcps_bg2000cps_seed1_par Particle number for each timestamp
from IPython.core.display import HTML
HTML(open("./styles/custom2.css", "r").read())