Retrieve RefMet data from Metabolomics Workbench using REST API

Import Python modules...

In [ ]:
from __future__ import print_function

import os
import sys
import time
import re

import requests

from IPython import __version__ as ipyVersion

print("Python: %s.%s.%s" % sys.version_info[:3])
print("IPython: %s" % ipyVersion)

print()
print(time.asctime())

The URL PATH

The MW REST URL consists of three main parts, separated by forward slashes, after the common prefix specifying the invariant base URL (https://www.metabolomicsworkbench.org/rest/):

https://www.metabolomicsworkbench.org/rest/context/input_specification/output_specification

Part 1: The context determines the type of data to be accessed from the Metabolomics Workbench, such as metadata or results related to the submitted studies, data from metabolites, genes/proteins and analytical chemistry databases as well as other services related to mass spectrometry and metabolite identification:

context = study | compound | refmet | gene | protein | moverz | exactmass

Part 2: The input specification consists of two required parameters describing the REST request:

input_specification = input_item/input_value

Part 3: The output specification consists of two parameters describing the output generated by the REST request:

output_specification = output_item/(output_format)

The first parameter is required in most cases. The second parameter is optional. The input and output specifications are context sensitive. The context determines the values allowed for the remaining parameters in the input and output specifications as detailed in the sections below.

Setup MW REST base URL...

In [ ]:
MWBaseURL = "https://www.metabolomicsworkbench.org/rest"

The “refmet” context

The “refmet” context refers to a standardized reference nomenclature for both discrete metabolite structures and metabolite species identified by spectroscopic techniques in metabolomics experiments. This is an essential prerequisite for the ability to compare and contrast metabolite data across different experiments and studies. The use of identifiers such as PubChem compound IDs and InChiKeys offers only a partial solution because these identifiers will vary depending on parameters such as the salt form and degree of stereochemical detail. In addition, many metabolite species, especially lipids, are not reported by MS methods as discrete structures but rather as isobaric mixtures (such as PC(34:1) and TG(54:2)). To this end, a list of over 160,000 names from a set of over 800 MS and NMR studies on the Metabolomics Workbench has been used as a starting point to generate a highly curated analytical chemistry-centric list of common names for metabolite structures and isobaric species. Additionally, the vast majority of these names have been linked to a metabolite classification system using a combination of LIPID MAPS and ClassyFire classification methods. A name-conversion user interface is provided where users can submit a list of metabolite names and map them to the corresponding Refmet names. This is a work-in-progress with the caveat that many metabolite names generated by metabolomics experiments will not currently map to RefMet identifiers. Nevertheless, RefMet has the ability to greatly increase the data-sharing potential of metabolomics experiments and facilitate "meta-analysis" and systems biology objectives for the majority of commonly encountered metabolite species.

This context provides access to many structural features including InChIKey, exact mass, formula common and systematic names, chemical classification and cross-references to other database

context = refmet

input_item = all | match | name | inchi_key | regno | pubchem_cid | formula | main_class | sub_class

input_value = input_item_value

output_item = all | name | inchi_key | regno | pubchem_cid | exactmass | formula | synonyms | sys_name | main_class | sub_class | name,inchi_key,regno,...

output_format = txt | json

The “all” output item is automatically expanded to include the following items: name, regno, pubchem_cid, inchi_key, exactmass, formula, sys_name, main_class, sub_class, synonyms.

Retrieve and process RefMet data for compounds in JSON format

Setup REST URL to retrieve all available RefMet data for Cholesterol...

In [ ]:
MWDataURL = MWBaseURL + "/refmet/name/Cholesterol/all"

Execute REST request using "request" module...

In [ ]:
print("Initiating request: %s" % MWDataURL)
    
Response = requests.get(MWDataURL)

Check "request" status...

In [ ]:
print("\nStatus Code: %d" % (Response.status_code))

if Response.status_code != 200:
    print("Request failed: status_code: %d" % Response.status_code)

Process JSON results...

In [ ]:
print("\nAvailable RefMet data for a compound using name:\n")

Results = Response.json()

for ResultType in Results:
    ResultValue = Results[ResultType]
    print("%s: %s" % (ResultType, ResultValue))

Retreive and process all available RefMet data for all compounds...

In [ ]:
MWDataURL = MWBaseURL + "/refmet/all"

print("Initiating request: %s" % MWDataURL)
    
Response = requests.get(MWDataURL)

print("\nStatus Code: %d" % (Response.status_code))

if Response.status_code != 200:
    print("Request failed: status_code: %d" % Response.status_code)

print("\nAll available RefMet data for all compounds:\n")

Results = Response.json()

CmpdsCount = 0
ListCmpdsCount = 50

for ResultNum in Results:
    CmpdsCount += 1
    if CmpdsCount > ListCmpdsCount:
        continue
        
    print("\nResultNum: %s\n" % ResultNum)
    for ResultType in Results[ResultNum]:
        ResultValue = Results[ResultNum][ResultType]    
        print("%s: %s" % (ResultType, ResultValue))


print("\nTotal number of compounds: %d" % CmpdsCount)
print("Number of compounds listed: %d" % ListCmpdsCount)

Retrieve and process RefMet data for compounds in text format

Retreive and process all available RefMet data for all compounds...

In [ ]:
MWDataURL = MWBaseURL + "/refmet/all/xx/all/txt"

print("Initiating request: %s" % MWDataURL)
    
Response = requests.get(MWDataURL)

print("\nStatus Code: %d" % (Response.status_code))

if Response.status_code != 200:
    print("Request failed: status_code: %d" % Response.status_code)

print("\nAll available RefMet data for all compounds:\n")

Results = Response.text

CmpdsCount = 0
ListCmpdsCount = 40
for Result in Results.split("\n"):
    Words = Result.split("\t")
    if len(Words) != 2:
        continue
    
    if re.match("^name$", ResultType, re.I):
        CmpdsCount += 1
    
    if CmpdsCount > ListCmpdsCount:
        continue
    
    ResultType, ResultValue = Words
    if re.match("^name$", ResultType, re.I):
        if CmpdsCount < ListCmpdsCount:
            print("\n")
    
    print("%s: %s" % (ResultType, ResultValue))

print("\nTotal number of compounds: %d" % CmpdsCount)
print("Number of compounds listed: %d" % ListCmpdsCount)