Installing yt-4.0 development branch and trident for better particle support

Author: Cameron Hummels

Updated: 8/27/2020

Traditionally, yt interacts with particle-based datasets by depositing all of the particle fields to an octree-based grid, and then it samples this grid like it would any other grid-based dataset. Unfortunately, the grid-deposition step is costly both in terms of time and memory, and there can be some smoothing effects introduced to particle-based datasets in this process.

Recently, efforts have been made to change how yt interacts with particle-based datasets by avoiding this grid-deposition step altogether and interacting only with the particles themselves. Projections and rays are calculated by determining the line integral through the smoothing kernel of each particle along a line of sight, obviating the need for a grid structure at all. These efforts, previously known as "demeshening" and now tied into the new yt-4.0 release, have been led by Nathan Goldbaum, Meagan Lang, and Matthew Turk. Work by Bili Dong and Cameron Hummels has ensured that these improvements can be harnessed by Trident too. For more details on how demeshening works, please refer to the relevant YTEP https://ytep.readthedocs.io/en/latest/YTEPs/YTEP-0032.html and SciPY presentation https://www.youtube.com/watch?v=pkZgQIGac6I .

Because the demeshened version of yt has some backwards-incompatible changes, it will not go into the mainline of yt until version yt 4.0. Fortunately, a lot of progress has occurred in the last few years, and all of these changes are now in the master (i.e., development) branch of the yt repository. See below for more information about installing and use of this version which is required to work with the development branch of Trident.

As an example of the difference between traditional yt with demeshened yt, here are two projections generated with this script of a publicly available medium-resolution FIRE dataset (http://yt-project.org/data/FIRE_M12i_ref11.tar.gz). Notably, the demeshened version takes substantially less time and memory to generate and has no grid artifacts.

import yt
fn = 'FIRE_M12i_ref11/snapshot_600.hdf5'
ds = yt.load(fn)
_, c = ds.find_max(('gas', 'density'))
p = yt.ProjectionPlot(ds, 'x', ('gas', 'density'), center=c, width=(1, 'Mpc'))
p.set_zlim(('gas', 'density'), 2e-6, 2e-2)
p.show()

Standard yt:

title

Demesh yt:

title

1) Install Mini-conda on your system if not installed

Conda is a package manager providing a clean, stand-alone installation of python that is self-contained in its installation directory. yt & trident require a modern installation of python to work. conda provides that installation. It is possible to install yt without conda with an existing python installation by jumping to the next step.

Check if you have conda already installed

$ conda -h

If conda is installed, move to the next step. Otherwise install Mini-conda.

Install Mini-conda

Use the appropriate conda install script for your architecture. We recommend getting the latest version of conda for Python3 for your architecture here: https://repo.continuum.io/miniconda/

For modern macs:

$ wget https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
$ bash Miniconda3-latest-MacOSX-x86_64.sh

For modern linux machines:

$ wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
$ bash Miniconda3-latest-Linux-x86_64.sh

At the end of the installation step, allow conda to add its installation directory to the $PATH.

2) Install yt's basic dependencies using conda

$ conda install numpy cython mpi4py

3) Get and install the development version of yt

Here are some git commands for accessing and installing from the master branch:

$ git clone https://github.com/yt-project/yt.git
$ cd yt
$ pip install -e .
$ cd ..

4) Get development version of yt_astro_analysis package

The yt_astro_analysis package has some modules necessary for Trident to run. Like yt and Trident, its versions are tied to yt, such that its development version is required to work with the development version of yt. You can acquire it and install it similarly to how you got yt from its repository as:

$ git clone https://github.com/yt-project/yt_astro_analysis.git
$ cd yt_astro_analysis
$ pip install -e .
$ cd ..

5) Get development version of Trident

Currently, the development branch of Trident will only work with the master development branch of yt. The development branch of Trident must be installed from its github repository.

If you have stable version of trident installed by pip, uninstall and continue

$ pip uninstall trident

Get development version of trident

$ git clone https://github.com/trident-project/trident.git
$ cd trident
$ pip install -e .
$ cd ..

Run Trident for the first time

$ python
>>> import trident

Follow the instructions to download the ion_balance table and verify the code.

You should be good to go. Try generating a yt.ProjectionPlot, which works better and faster than the one using the octree grid. You should also be able to create Trident LightRays using trident.make_simple_ray() or any of the functionality of Trident should work in the demeshened context.

6) (optional) Test out Trident demeshened functionality on test FIRE dataset

Grab the test FIRE dataset FIRE_M12i_ref11 from the yt sample datasets, run our test script on it, and look at the results.

$ wget http://yt-project.org/data/FIRE_M12i_ref11.tar.gz
$ tar -zxvf FIRE_M12i_ref11.tar.gz
$ cp /your/path/to/trident/example/gizmo_script.py .
$ python gizmo_script.py
$ open projection.png
$ open spec_raw.png
$ open spec_final.png