yt-4.0) and trident¶
Author: Cameron Hummels
Traditionally, yt interacts with particle-based datasets by depositing all of the particle fields to an octree-based grid, and then it samples this grid like it would any other grid-based dataset. Unfortunately, the grid-deposition step is costly both in terms of time and memory, and there can be some smoothing effects introduced to particle-based datasets in this process.
Recently, efforts have been made to change how yt interacts with particle-based datasets by avoiding this grid-deposition step altogether and interacting only with the particles themselves. Projections and rays are calculated by determining the line integral through the smoothing kernel of each particle along a line of sight, obviating the need for a grid structure at all. These efforts, known as "demeshening" have been led by Nathan Goldbaum, Meagan Lang, and Matthew Turk. Work by Bili Dong and Cameron Hummels has ensured that these improvements can be harnessed by Trident too. For more details on how demeshening works, please refer to the relevant YTEP https://ytep.readthedocs.io/en/latest/YTEPs/YTEP-0032.html and SciPY presentation https://www.youtube.com/watch?v=pkZgQIGac6I .
Because the demeshened version of yt has some backwards-incompatible changes, it will not go into the mainline of yt until version yt 4.0. Fortunately, it is now available for use in the
yt-4.0 branch of the main yt repository. See below for more information about installing and use of
yt-4.0 and the corresponding branch in Trident (
As an example of the difference between traditional yt with demeshened yt, here are two projections generated with this script of a publicly available medium-resolution FIRE dataset. Notably, the demeshened version takes substantially less time and memory to generate and has no grid artifacts.
import yt fn = 'FIRE_M12i_ref11/snapshot_600.hdf5' ds = yt.load(fn) _, c = ds.find_max(('gas', 'density')) p = yt.ProjectionPlot(ds, 'x', ('gas', 'density'), center=c, width=(1, 'Mpc')) p.set_zlim(('gas', 'density'), 2e-6, 2e-2) p.show()
At this point, nearly all of yt's functionality is working in the demeshened version, but the code is still under development. Results should be cautiously interpreted. That said, it is being used for production level work by many users (including the author), and we encourage people to begin using it for its many benefits in working with particle-based data. If you encounter a bug or a problem with your use case or simulation frontend, it would help us immensely if you could report it. Please contact the yt-users and/or trident-users email lists, jump into slack, or create a github issue with
demeshening tag with the relevant information regarding how to reproduce your bug:
Conda is a package manager providing a clean, stand-alone installation of python that is self-contained in its installation directory. yt & trident require a modern installation of python to work. conda provides that installation. It is possible to install yt without conda with an existing python installation by jumping to the next step.
$ conda -h
If conda is installed, move to the next step. Otherwise install Mini-conda.
Use the appropriate conda install script for your architecture. We recommend getting the latest version of conda for Python3 for your architecture here: https://repo.continuum.io/miniconda/
For modern macs:
$ wget https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh $ bash Miniconda3-latest-Linux-x86_64.sh
For modern linux machines:
$ wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh $ bash Miniconda3-latest-Linux-x86_64.sh
At the end of the installation step, allow conda to add its installation directory to the $PATH.
yt had a number of dependencies, so we need to isntall them prior to installing yt itself:
$ pip install numpy jupyter sphinx gitpython h5py matplotlib cython nose scipy astropy sympy mpi4py
Right now, the demeshened version of yt operates as a separate branch in the main yt repository. Here are some git commands for accessing and installing from this branch, entitled
$ git clone https://github.com/yt-project/yt.git $ cd yt $ git checkout yt-4.0 $ git pull $ pip install -e . $ cd ..
To change to the main version (master branch) of yt:
$ git checkout master # go back to using the main yt version as your active yt installation $ git pull origin master # pull main yt-dev and merge with local master $ pip install -e . # Re-install the master branch of yt (rebuilds cython)
To change to the demeshened version (yt-4.0) of yt:
$ git checkout yt-4.0 # go back to using the yt-4.0 yt version as your active yt installation $ git pull origin yt-4.0 # pull yt-4.0 and merge with local master $ pip install -e . # Re-install the yt-4.0 branch of yt (rebuilds cython)
Currently, the demeshened version of Trident exists as a branch in the main Trident repository. Here are some git commands for accessing this demeshened branch, entitled
$ pip uninstall trident
$ git clone https://github.com/trident-project/trident.git $ cd trident $ git checkout sph-viz $ pip install -e . $ cd ..
$ python >>> import trident
Follow the instructions to download the ion_balance table and verify the code.
You should be good to go. Try generating a
yt.ProjectionPlot, which works better and faster than the one using the octree grid. You should also be able to create Trident
trident.make_simple_ray() or any of the functionality of Trident should work in the demeshened context.
Grab the test FIRE dataset FIRE_M12i_ref11 from the yt sample datasets, run our test script on it, and look at the results.
$ wget http://yt-project.org/data/FIRE_M12i_ref11.tar.gz $ tar -zxvf FIRE_M12i_ref11.tar.gz $ cp /path/to/trident/example/gizmo_script.py . $ python gizmo_script.py $ open projection.png $ open spec_raw.png $ open spec_final.png