funcX is a Function-as-a-Service (FaaS) platform for science that enables you to convert almost any computing resource into a high-performance function serving device. To do this, you deploy a funcX endpoint agent on the resource, which integrates it into the function serving fabric, allowing you to dynamically send, monitor, and receive results from function invocations. funcX is built on top of Parsl, enabling a funcX endpoint to use large compute resources via traditional batch queues, where funcX will dynamically provision, use, and release resources on-demand to fulfill function requests. The function service fabric, which is run centrally as a service, is hosted in AWS.
Here we provide an example of using funcX to register a function and run it on a publicly available tutorial endpoint.
We start by instantiating a funcX client as a programmatic means of communicating with the function service fabric. The client allows you to:
Instantiating a client will force an authentication flow where you will be asked to authenticate with Globus Auth. Every interaction with funcX is authenticated to allow us to enforce access control on both functions and endpoints. As part of the authentication process we request access to your identity information (to retrieve your email address), Globus Groups management access, and Globus Search. We require Groups access in order to facilitate sharing. Globus Search allows funcX to add your functions to a searchable registry and make them discoverable to permitted users (as well as yourself!).
from funcx.sdk.client import FuncXClient
fxc = FuncXClient()
Next we define a Python function, which we will later register with funcX. This function simply sums its input.
When defining a function you can specify *args and **kwargs as inputs.
def funcx_sum(items):
return sum(items)
To use a function with funcX, you must first register it with the service, using register_function
. You can optionally include a description of the function.
The registration process will serialize the function body and transmit it to the funcX function service fabric.
Registering a function returns a UUID for the function, which can then be used to invoke it.
func_uuid = fxc.register_function(funcx_sum,
description="tutorial summation", public=True)
print(func_uuid)
You can search previously registered functions to which you have access using search_function
. The first parameter q
is searched against all the fields, such as author, description, function name, and function source. You can navigate through pages of results with the offset
and limit
keyword args.
The object returned is simple wrapper on a list, so you can index into it, but also can have a pretty-printed table.
To make use of the results, you can either just use the function_uuid
field returned for each result, or for functions that were registered with recent versions of the service, you can load the source code using the search results object's load_result
method.
search_results = fxc.search_function("tutorial", offset=0, limit=5)
print(search_results[0])
print(search_results)
search_results.load_result(0)
result_0_uuid = search_results[0]['function_uuid']
To invoke (perform) a function, you must provide the function's UUID, returned from the registration process, and an endpoint_id
. Note: here we use the funcX public tutorial endpoint, which is running on AWS.
The client's run
function will serialize any *args and **kwargs, and pass them to the function when invoking it. Therefore, as our example function simply takes an arg input (items), we can specify an input arg and it will be used by the function. Here we define a small list of integers for our function to sum.
The Web service will return the UUID for the invokation of the function, which we call a task. This UUID can be used to check the status of the task and retrieve the result.
endpoint_uuid = '4b116d3c-1703-4f8f-9f6f-39921e5864df' # Public tutorial endpoint
items = [1, 2, 3, 4, 5]
res = fxc.run(items, endpoint_id=endpoint_uuid, function_id=func_uuid)
print(res)
You can now retrieve the result of the invocation using get_result()
on the UUID of the task.
fxc.get_result(res)
You might want to invoke many function calls at once. This can be easily done via the batch interface:
def squared(x):
return x**2
squared_uuid = fxc.register_function(squared, searchable=False)
inputs = list(range(10))
batch = fxc.create_batch()
for x in inputs:
batch.add(x, endpoint_id=endpoint_uuid, function_id=squared_uuid)
batch_res = fxc.batch_run(batch)
fxc.get_batch_status(batch_res)
When functions fail, the exception is captured, and reraised when you try to get the result. In the following example, the 'deterministic failure' exception is raised when fxc.get_result
is called on the failing function.
def failing():
raise Exception("deterministic failure")
failing_uuid = fxc.register_function(failing, searchable=False)
res = fxc.run(endpoint_id=endpoint_uuid, function_id=failing_uuid)
fxc.get_result(res)