OHBM 2021 NiMARE tutorial¶

What is NiMARE?¶

NiMARE is a Python library for performing neuroimaging meta-analyses and related analyses, like automated annotation and functional decoding. The goal of NiMARE is to centralize and standardize implementations of common meta-analytic tools, so that researchers can use whatever tool is most appropriate for a given research question.

There are already a number of tools for neuroimaging meta-analysis:

Tool	Scope
	BrainMap includes a suite of applications for (1) searching its manually-annotated coordinate-based database, (2) adding studies to the database, and (3) running ALE meta-analyses. While search results can be extracted using its Sleuth app, access to the full database requires a collaborative use agreement.
	Neurosynth provides (1) a large, automatically-extracted coordinate-based database, (2) a website for performing large-scale automated meta-analyses, and (3) a Python library for performing meta-analyses and functional decoding, mostly relying on a version of the MKDA algorithm. The Python library has been deprecated in favor of NiMARE.
	Neurovault is a repository for sharing unthresholded statistical images, which can be used to search for images to use in image-based meta-analyses. Neurovault provides a tool for basic meta-analyses and an integration with Neurosynth's database for online functional decoding.
	The Seed-based d Mapping (SDM) app provides a graphical user interface and SPM toolbox for performing meta-analyses with the SDM algorithm, which supports a mix of coordinates and images.
	The MATLAB-based MKDA toolbox includes functions for performing coordinate-based meta-analyses with the MKDA algorithm.

The majority of the above tools are (1) closed source, (2) based on graphical user interfaces, and/or (3) written in a programming language that is rarely used by neuroimagers, such as Java.

In addition to these established tools, there are always interesting new methods that are described in journal articles, but which are never translated to a well-documented and supported implementation.

NiMARE attempts to consolidate the different algorithms that are currently spread out across a range of tools (or which never make the jump from paper to tool), while still ensuring that the original tools and papers can be cited appropriately.

NiMARE's design philosophy¶

NiMARE's API is designed to be similar to that of scikit-learn, in that most tools are custom classes. These classes follow the following basic structure:

1. Initialize the class with general parameters

cls = Class(param1, param2)

2. For Estimator classes, apply a fit method to a Dataset object to generate a MetaResult object

result = cls.fit(dataset)

3. For Transformer classes, apply a transform method to an object to return a transformed version of that object

   • An example transformer that accepts a Dataset:

dataset = cls.transform(dataset)

- A transformer that accepts a `MetaResult`:

result = cls.transform(result)

Stability and consistency¶

NiMARE is currently in alpha development, so we appreciate any feedback or bug reports users can provide. Given its status, NiMARE's API may change in the future.

Usage questions can be submitted to Neurostars with the 'nimare' tag, while bug reports and feature requests can be submitted to NiMARE's issue tracker.

Goals for this tutorial¶

1. Working with NiMARE meta-analytic datasets
2. Searching large datasets
3. Performing coordinate-based meta-analyses
4. Performing image-based meta-analyses
5. Performing functional decoding using Neurosynth

Before we start, let's download the necessary data only if running locally¶

The code in the following cell checks whether you have the data, and if you don't, it starts downloading it.

If you're running this notebook locally or using mybinder, then you will need to download the data. You can copy the code below into a new cell, with the Jupyter magic command %%bash at the top of the cell.

If you're running it using binder hosted on neurolibre, then you already have access to the data on neurolibre, and you don't need to run this code snippet.

DIR=$"../data/nimare_tutorial/"
if [ -d "$DIR" ]; then
    echo "$DIR exists."
else 
    mkdir -p $DIR;
    pip install osfclient
    osf -p u9sqa clone  $DIR;
    echo "Created $DIR and downloaded the data";
fi

Basics of NiMARE datasets¶

NiMARE relies on a specification for meta-analytic datasets named NIMADS. Under NIMADS, meta-analytic datasets are stored as JSON files, with information about peak coordinates, relative links to any unthresholded statistical images, metadata, annotations, and raw text.

NOTE: NiMARE users generally do not need to create JSONs manually, so we won't go into that structure in this tutorial. Instead, users will typically have access to datasets stored in more established formats, like Neurosynth and Sleuth files.

We will start by loading a dataset in NIMADS format, because this particular dataset contains both coordinates and images. This dataset is created from Collection 1425 on NeuroVault, which contains NIDM-Results packs for 21 pain studies.

In NiMARE, datasets are stored in a special Dataset class. The Dataset class stores most relevant information as properties.

The full list of identifiers in the Dataset is located in Dataset.ids. Identifiers are composed of two parts- a study ID and a contrast ID. Within the Dataset, those two parts are separated with a -.

Most other information is stored in pandas DataFrames. The five DataFrame-based attributes are Dataset.metadata, Dataset.coordinates, Dataset.images, Dataset.annotations, and Dataset.texts.

Each DataFrame contains at least three columns: study_id, contrast_id, and id, which is the combined study_id and contrast_id.

Other relevant attributes are Dataset.masker and Dataset.space.

Dataset.masker is a nilearn Masker object, which specifies the manner in which voxel-wise information like peak coordinates and statistical images are mapped into usable arrays. Most meta-analytic tools within NiMARE accept a masker argument, so the Dataset's masker can be overridden in most cases.

Dataset.space is just a string describing the standard space and resolution in which data within the Dataset are stored.

Datasets can also be saved to, and loaded from, binarized (pickled) files.

We cannot save files on Binder, so here is the code we would use to save the pain Dataset:

pain_dset.save("pain_dataset.pkl.gz")

Now for a more common situation, where users want to use NiMARE on data from Neurosynth or a Sleuth file.

Downloading and converting the Neurosynth dataset takes a long time, so we will use a pregenerated version of the dataset. However, here is the code we would use to download and convert the dataset from scratch:

nimare.extract.fetch_neurosynth("data/", unpack=True)
ns_dset = nimare.io.convert_neurosynth_to_dataset(
    "data/database.txt",
    "data/features.txt",
)

Searching large datasets¶

The Dataset class contains multiple methods for selecting subsets of studies within the dataset.

One common approach is to search by "labels" or "terms" that apply to studies. In Neurosynth, labels are derived from term frequency within abstracts.

The slice method creates a reduced Dataset from a list of IDs.

A MACM (meta-analytic coactivation modeling) analysis is generally performed by running a meta-analysis on studies with a peak in a region of interest, so Dataset includes two methods for searching based on the locations of coordinates: Dataset.get_studies_by_coordinate and Dataset.get_studies_by_mask.

Running meta-analyses¶

Coordinate-based meta-analysis¶

Most coordinate-based meta-analysis algorithms are kernel-based, in that they convolve peaks reported in papers with a "kernel". Kernels are generally either binary spheres, as in multi-level kernel density analysis (MKDA), or 3D Gaussian distributions, as in activation likelihood estimation (ALE).

NiMARE includes classes for different kernel transformers, which accept Datasets and generate the images resulting from convolving each study's peaks with the associated kernel.

Meta-analytic Estimators are initialized with parameters which determine how the Estimator will be run. For example, ALE accepts a kernel transformer (which defaults to the standard ALEKernel), a null method, the number of iterations used to define the null distribution, and the number of cores to be used during fitting.

The Estimators also have a fit method, which accepts a Dataset object and returns a MetaResult object. MetaResults link statistical image names to numpy arrays, and can be used to produce nibabel images from those arrays, as well as save the images to files.

Multiple comparisons correction¶

Most of the time, you will want to follow up your meta-analysis with some form of multiple comparisons correction. For this, NiMARE provides Corrector classes in the correct module. Specifically, there are two Correctors: FWECorrector and FDRCorrector. In both cases, the Corrector supports a range of naive correction options relying on statsmodels' methods.

In addition to generic multiple comparisons correction, the Correctors also reference algorithm-specific correction methods, such as the montecarlo method supported by most coordinate-based meta-analysis algorithms.

Correctors are initialized with parameters, and they have a transform method that accepts a MetaResult object and returns an updated one with the corrected maps.

Image-based meta-analysis¶

Note that "z" images are missing for some, but not all, of the studies.

NiMARE's transforms module contains a class, ImageTransformer, which can generate images from other images- as long as the right images and metadata are available. In this case, it can generate z-statistic images from t-statistic maps, combined with sample size information in the metadata. It can also generate "varcope" (contrast variance) images from the contrast standard error images.

Now that we have all of the image types we will need for our meta-analyses, we can run a couple of image-based meta-analysis types.

The DerSimonianLaird method uses "beta" and "varcope" images, and estimates between-study variance (a.k.a. $\tau^2$).

The PermutedOLS method uses z-statistic images, and relies on nilearn's permuted_ols tool.

Compare to results from the SPM IBMA extension¶

Adapted from Maumet & Nichols (2014).

Meta-Analytic Functional Decoding¶

Functional decoding refers to approaches which attempt to infer mental processes, tasks, etc. from imaging data. There are many approaches to functional decoding, but one set of approaches uses meta-analytic databases like Neurosynth or BrainMap, which we call "meta-analytic functional decoding." For more information on functional decoding in general, read Poldrack (2011).

In NiMARE, we group decoding methods into three general types: discrete decoding, continuous decoding, and encoding.

• Discrete decoding methods use a meta-analytic database and annotations of studies in that database to describe something discrete (like a region of interest) in terms of those annotations.

• Continuous decoding methods use the same type of database to describe an unthresholded brain map in terms of the database's annotations. One example of this kind of method is the Neurosynth-based decoding available on Neurovault. In that method, the map you want to decode is correlated with Neurosynth term-specific meta-analysis maps. You end up with one correlation coefficient for each term in Neurosynth. Users generally report the top ten or so terms.

• Encoding methods do the opposite- they take in annotations or raw text and produce a synthesized brain map. One example of a meta-analytic encoding tool is NeuroQuery.

Most of the continuous decoding methods available in NiMARE are too computationally intensive and time-consuming for Binder, so we will focus on discrete decoding methods. The two most useful discrete decoders in NiMARE are the BrainMapDecoder and the NeurosynthDecoder. Detailed descriptions of the two approaches are available in NiMARE's documentation, but here's the basic idea:

0. A NiMARE Dataset must contain both annotations/labels and coordinates.
1. A subset of studies in the Dataset must be selected according to some criterion, such as having at least one peak in a region of interest or having a specific label.
2. The algorithm then compares the frequency of each label within the selected subset of studies against the frequency of other labels in that subset to calculate "forward-inference" posterior probability, p-value, and z-statistic.
3. The algorithm also compares the frequency of each label within the subset of studies against the the frequency of that label in the unselected studies from the Dataset to calculate "reverse-inference" posterior probability, p-value, and z-statistic.

Exercise: Run a MACM and Decode an ROI¶

Remember that a MACM is a meta-analysis performed on studies which report at least one peak within a region of interest. This type of analysis is generally interpreted as a meta-analytic version of functional connectivity analysis.

We will use an amygdala mask as our ROI, which we will use to (1) run a MACM using the (reduced) Neurosynth dataset and (2) decode the ROI using labels from Neurosynth.

First, we have to prepare some things for the exercise. You just need to run these cells without editing anything.

Below, try to write code in each cell based on its comment.

After the exercise¶

Your MACM results should look something like this:

And your decoding results should look something like this, after sorting by probReverse: