How To Connect to REST APIs

Carlo Giacometti, WRI

#WolframTechConf

Introducing the Service Framework

ServiceConnect and the framework around it were introduced in version 10 (2014) as a way to interact with API services in a Wolfram-like way. Some of the services added were Facebook and Twitter.

Often companies that provide an API also provide client libraries or SDKs to interact with the API endpoint to make the user’s life easier:

curl https://api.openai.com/v1/responses \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $OPENAI_API_KEY" \
    -d '{
        "model": "gpt-4o",
        "input": "Write a one-sentence bedtime story about a unicorn."
    }'

from openai import OpenAI
client = OpenAI()

response = client.responses.create(
    model="gpt-4o",
    input="Write a one-sentence bedtime story about a unicorn."
)

print(response.output_text)

The service framework was intended to to be a Wolfram like version of that.

Another important goal of the service framework is to neatly handle (and potentially save) the user’s credentials for the API. These could be as simple as no credentials whatsoever, to keys, to various flavours of OAuth.

All of this has become increasingly important in recent years since most of LLM/AI computation is being done through APIs.

Why

The service framework functions had a lot of quirks, especially regarding handling multiple credentials, making sure a specific used the requested set of credentials, and lifetime management of connections and credentials.

The maintenance of old connections and the development of new connections was extremely obscure and undocumented

ServiceSubmit (meant to handle async requests) was effectively implemented as SessionSubmit[ServiceExecute[...]], which meant we had to go to great lengths to hack around this to allow the real async requests that power the Notebook Assistant and Chatbook

Loading times were pretty slow

The code was scattered between Kernel and several packages like OAuth

Where

In addition to the previously documented system symbols, there is now a collection of ServiceFramework symbols to help developing/managing connections

?ServiceFramework`*

Some internal docs: https://www.wolframcloud.com/obj/carlogico/Published/ServiceFramework/Overview.nb

Glossary

ServiceConnection: this is the definition of a service connection, identified by a unique name, eg “ArXiv”, “OpenAI”

Registered ServiceConnection: this is a ServiceConnection that the ServiceFramework is aware of and has been loaded

ServiceObject: this is how a user uses a service connection. It is uniquely identified by the tuple {serviceName, serviceID}. This is associated with a set of user specified credentials

Credentials: could be None, a string, an Association, a SecuredAuthenticationKey (and other expression depending on the exact ServiceConnection definition). It is defined by the user either through a dialog or the Authentication option.

Raw request: these are requests that map exactly the API endpoints. We only verify that the names of the specified parameters are correct.

Processed request: these can be arbitrary functions with defined parameter verification and normalization, and result postprocessing

How to use a connection

Creating a ServiceObject

First, let’s find what connections the framework is aware of:

$Services

Then we can “connect”:

arxiv=ServiceConnect["ArXiv"]

openai=ServiceConnect["OpenAI"]

dropbox=ServiceConnect["Dropbox"]

Note: “connecting” is a bit of a lie for no auth or key based services. It just means that the credentials are stored somewhere, and no internet operation is performed.

Use the Authentication option to pass credentials directly instead of triggering a dialog.

We can control where the credentials are saved using the SaveConnection option. The possible values are True, “Local” or False.

If there already are existing ServiceObjects for a specific connection, if only ServiceConnect[“serviceName”] is requested, the default ServiceObject for the connection will be retrieved.

Information about the service object can be accessed through SubValues:

openai["Properties"]

openai["Requests"]

openai["RequestParameterPatterns"]//Dataset

Bonus: ServiceConnect details

ServiceConnect[serviceName]
(*fetch the default (newest) object, create a new one if not found*)

ServiceConnect[serviceName,"New"]
(*create new object, gives a new uuid*)

ServiceConnect[serviceName,id]
(*use ServiceObject[service,objectid], create a new one with that name if it doesn't exist*)

ServiceConnect[serviceName,Authentication->auth]
(*Always create a new one with new uuid*)

ServiceConnect[serviceName,id,Authentication->auth]
(*new serviceobject{service,objectid} with auth, old auth is removed*)

“UpdateServiceObjects” -> True (forces a sync with cloud)

Bonus: how are credentials stored

Authentication:>val

DeleteObject@ServiceObjects[]

oai=ServiceConnect["OpenAI","delayed",Authentication:>Echo@"delayed"];

ServiceExecute["OpenAI","ChatModelList"]

SaveConnection->False

DeleteObject@ServiceObjects[]

ServiceConnect["OpenAI","saveFalse",SaveConnection->False,Authentication->"false"];
ServiceObjects[]
{
"session"->ServiceFramework`$sessionCredentials,
"local"->SystemCredentialKeys["sf/*"],
"cloud"->ServiceFramework`Management`PackagePrivate`retrieveCloudCredentials[]
}//Column

SaveConnection->”Local”

DeleteObject@ServiceObjects[]

ServiceConnect["OpenAI","saveLocal",SaveConnection->"Local",Authentication->"false"];
ServiceObjects[]
{
"session"->ServiceFramework`$sessionCredentials,
"local"->SystemCredentialKeys["sf/*"],
"cloud"->ServiceFramework`Management`PackagePrivate`retrieveCloudCredentials[]
}//Column

SaveConnection->True

DeleteObject@ServiceObjects[]

ServiceConnect["OpenAI","saveTrue",SaveConnection->True,Authentication->"false"];
ServiceObjects[]
{
"session"->ServiceFramework`$sessionCredentials,
"local"->SystemCredentialKeys["sf/*"],
"cloud"->ServiceFramework`Management`PackagePrivate`retrieveCloudCredentials[]
}//Column

How to use a connection

Performing synchronous requests

ServiceExecute takes the service, the request name and the request parameters.

It is possible to pass either a name or a ServiceObject as the first argument. Passing a specific ServiceObject will use the credentials associated to that.

Note: previously it was possible to pass parameters as a sequence of rules, now they need to be contained in the third argument with either a list or an association.

ServiceExecute["ArXiv","AuthorSearch",<|"Query"->"Borish","SortOrder"->"Descending",MaxItems->2|>]

If an explicit Authentication option is specified, those credentials are used for that specific call, but they are deleted once that call is over.

Options[ServiceExecute]

Performing asynchronous requests

In this case we get everything that URLSubmit supports + “EvaluationResult”.

r=ServiceRequest["ArXiv","AuthorSearch",{"Query"->"Borish","SortOrder"->"Descending"}]

res={};
task=ServiceSubmit[r,HandlerFunctions-><|"BodyChunkReceived"->(AppendTo[res,#]&)|>,HandlerFunctionsKeys->{"BodyChunk","Cookies","Headers","EvaluationResult"}];
task["Properties"]

res

How to use a connection

Managing the lifetime of ServiceObjects

ServiceObjects[]

DeleteObject@ServiceObjects[]

Create a set of connections with different authentication and random names:

Table[ServiceConnect["OpenAI",Authentication->RandomWord[]],4];

Now create a few named connections:

Table[ServiceConnect["OpenAI",Echo@RandomWord[],Authentication->RandomWord[]],2];
Table[ServiceConnect["ArXiv",RandomWord[],Authentication->None],2];

Retrieve a service object with a specific name:

ServiceObject["OpenAI","stridently"]

List all the existing service objects:

ServiceObjects[]

List all “ArXiv” service objects:

ServiceObjects["ArXiv"]

Delete a specific object:

Length@ServiceObjects[]
DeleteObject@ServiceObject["OpenAI","stridently"]
Length@ServiceObjects[]

Delete a list of objects:

Length@ServiceObjects[]
DeleteObject@ServiceObjects["ArXiv"]
Length@ServiceObjects[]

DeleteObject is functionally equivalent to ServiceDisconnect.

How to define a connection

A service connection is fully defined by an association with a few required fields and a lot of optional ones. See a full list here.

The definition can be as minimal as this:

ServiceFramework`DefineServiceConnection[<|
	"ServiceFrameworkVersion" -> "0.1.0",
	"ServiceName" -> "TestConnection",
	"AuthenticationMethod" -> None,
	"BaseURL" -> "https://api.restful-api.dev/",
	"AllowAnyRequests" -> True
|>]

$Services

ServiceConnect["TestConnection"]

ServiceExecute["TestConnection","objects"]

Or to go to the opposite end.

If a connection is defined simply as an association, its name will appear in $Services only after DefineServiceConnection is called.

One way to make $Services aware of a connection before connection registration is to use a “ServiceConnection” extension in a paclet. A required attribute of that is the “Context” property, which will tell the framework which context to load to register the connection, something like:

"Extensions" ->
	{
		...,
		{"ServiceConnection", "Name" -> "ConnectionName", Context -> {"ConnectionContext`"}},
		...
	}

A single paclet can define multiple “ServiceConnection” extensions, e.g.:

PacletObject["WebSearch"]["Extensions"]

Defining Authentication details

There are a few fields in thew connection definition to determine how a connection handles authentication:

Select[ServiceFramework`PackageScope`$connectionSchema[[All,"Name"]],StringMatchQ["*Authentication*"]]

“AuthenticationMethod”: this can only be None (no credentials needed), “APIKey” (which is a catchall category that includes strings or associations of strings like username and passwords), and “OAuth”

“AuthenticationParser”: this determines any processing done to the value of the Authentication option before it is stored. An example for “APIKey” could be:

"AuthenticationParser" -> Function[
	userSpecifiedAuth,
	If[StringQ[userSpecifiedAuth],
		(*The service only accepts keys that start with "myAuth"*)
		If[StringStartsQ[userSpecifiedAuth, "myAuth"],
			userSpecifiedAuth, (*the framework saves it*)
			$Failed
		],
		(*fail if the user's specified authentication is not a string*)
		$Failed
	]
]

The default value for “AuthenticationParser” should work in most cases

“AuthenticationFunction”: this determines how the saved authentication is added to the HTTPRequest. As an example, if we had a string authentication value, and we wanted to add that to the headers of the request as “Authorization: Bearer API_KEY”, we could use:

"AuthenticationFunction" -> Function[<|"Headers" -> {"Authorization" -> "Bearer "<> #}|>]

An additional property related to Authentication is “ClientDialog”, which is called when the user doesn’t specify the Authentication option or uses Authentication -> “Dialog”

The specified function is passed Dynamic[saveQ] as an argument, where saveQ corresponds to the value of the SaveConnection option. If the dialog has a “save connection” checkbox, this value should be modified.

The service framework provides two types of dialog to aid with consistent look, one for one or multiple string keys:

saveQ = True;
ServiceFramework`MultipleKeyDialog[
	"ServiceName",
	{
		"API Key" -> {"apikey", FieldMasked -> True},
		"Other Key" -> {"otherkey", FieldMasked -> True}
	},
	Dynamic[saveQ],
	"howToGetKeysURL",
	"termsURL",
	"instructionsURL"
]

and one for no credentials

ServiceFramework`OtherDialog[
	"ServiceName",
	{"Terms text:", "TermsURL"}
]

There is also support for “AuthenticationSchema”, which aims to unify all previous parameters in a simple association like <|Type”->”ApiKey”,”In”->”Query”|>.

Defining raw requests

The main parameters to define a raw request are the url endpoint, the http method and the names and destinations of the parameters. In addition to that there are various pre and post processing functions for the parameter values and the http request and response themselves:

req = <|
	"URL" -> "http://wrongurl",
	"HTTPSMethod" -> "POST" (*"GET" | "DELETE" | ...*),
	"Headers" -> {"Content-Type" -> "application/json"},
	"QueryParameters" -> {"qName1", "qName2"},
	"BodyParameters" -> {"bName1", "bName2"},
	"PathParameters" -> {},
	"HeadersParameters" -> {"hName1", "hName2"},
	"RequiredParameters" -> {},
	"HTTPResponseProcessing" -> (ServiceFramework`ImportResponse[Echo[#1,"HTTPResponse"], ##2]&),
	"HTTPRequestFunction" -> (Echo[#, "HTTPRequest"] &)
|>;

ServiceFramework`DefineServiceConnection[<|
	"ServiceFrameworkVersion" -> "0.1.0",
	"ServiceName" -> "TestConnection2",
	"AuthenticationMethod" -> None,
	"RawRequests" -> <|"Request" -> req|>
|>];

Look at the raw requests with all the default values filled in:

ServiceFramework`ServiceConnection["TestConnection2"]["RawRequests"]//Dataset

Perform the request:

ServiceExecute["TestConnection2","Request"]

Perform the request specifying some parameters:

ServiceExecute["TestConnection2","Request",{"qName1"->"p1","qName2"->2,"bName1"-><|"s"->2,"test"->5|>,"bName2"->6,"hName1"->7,"hName2"->"string"}]

HTTPRequest[]//GeneralUtilities`PrettyForm

Path parameters can also be used as long as the “URL” parameter is an appropriately defined StringTemplate:

ServiceFramework`DefineServiceConnection[<|
	"ServiceFrameworkVersion" -> "0.1.0",
	"ServiceName" -> "TestConnection3",
	"AuthenticationMethod" -> None,
	"RawRequests" -> <|"Request" -> <|
		req,
		"URL" -> StringTemplate["http://url/`par1`/rest"],
		"PathParameters" -> {"par1"}
	|>|>
|>]

ServiceExecute["TestConnection3","Request",{"qName1"->"p1","bName2"->6,"par1"->"myPar","hName2"->"string"}]

HTTPRequest[]//PF

In general, it is good practice to define a set of raw requests to mimic the API endpoints. As an example, I would replicate this OpenAI endpoint

as:

ServiceFramework`DefineServiceConnection[<|
	"ServiceFrameworkVersion" -> "0.1.0",
	"ServiceName" -> "OAI",
	"AuthenticationMethod" -> "APIKey",
	"AuthenticationFunction" -> Function[key, <|"Headers" ->{"Authorization" -> "Bearer "<> key}|>],
	"RawRequests" -> <|
		"Embedding" ->
		<|
			"URL" -> "https://api.openai.com/v1/embeddings",
			"HTTPSMethod" -> "POST",
			"Headers" -> {"Content-Type" -> "application/json"},
			"BodyParameters" -> {"input", "model", "dimensions", "encoding_format", "user"},
			"RequiredParameters" -> {"input", "model"}
		|>
	|>
|>]

ServiceExecute["OAI","Embedding",{"input"->"text","model"->"text-embedding-ada-002"},Authentication->$OpenAIKey[[1]]]

Defining processed requests

The idea for a basic processed request is that it should be a wolframized version of an endpoint, however the framework supports arbitrary functions.

Execution flow

This is the flow for a processed request:

Check the parameter names

the list of parameters and their default value is defined using the “Parameters” property in the process request definition. We could have something like

...
"Parameters" -> {"ParName" -> Automatic, "OtherParName" -> otherDefault}
...

It is possible to specify which parameters are required and if there are any hidden parameters.

Verify the parameter values

this is defined in the “ParameterVerification” connection property, which is a list of {requestPattern, parameterPattern, validPattern, Optional[isDefaultValidQ]}. An example could be

"ParameterVerification" -> {
	{"MyRequest", "ParName", _?StringQ | Small | Medium},
	{"MyOtherRequest", "ParName", _?IntegerQ},
	{_, "ParName2", _?IntegerQ},
	{"MyThirdRequest", _, _?AudioQ}
}

Normalize the parameter values

this is defined in the “ParameterNormalization” connection property, which is a list of {requestPattern, parameterPattern, replacements, Optional[removedPatterns]}. An example could be

"ParameterNormalization" -> {
	{"MyRequest", "ParName", {Small -> "150", Medium -> "250"}},
	{"MyOtherRequest", "ParName", {i_?IntegerQ /; i < 2 :> i + 1, i_ :> i + 2}},
	{_, "ParName2", {}, Automatic}
}

If removedPatterns is defined, and at runtime a parameter matches it, that parameter will be removed from the parameter list that will be passed to the raw request (allowing the developer/user to rely on the defaults provided by the service)

Rename the parameters

the name map is defined in the “ParameterMap” connection property, which is an association with the wolframized names as keys and the endpoint parameter names as values.

"ParameterMap" -> <|
	"FrequencyPenalty" -> "frequency_penalty",
	...
|>

Process all the parameters

Once the parameters have passed verification, normalization and renaming, the full association is passed to ”PreprocessingFunction” property of the processed request

Execute the request

Post process the response

This is done using the “ExecuteResultProcessing” property of the processed request in the case of ServiceExecute, or the “HandlerDataProcessing” property of the processed request in the case of ServiceSubmit

Defining processed requests

The Execute/Submit Function

Let’s take a simple service connection with a single raw request definition:

base = <|
	"ServiceFrameworkVersion" -> "0.1.0",
	"ServiceName" -> "RestfulAPI",
	"AuthenticationMethod" -> None,
	"BaseURL" -> "https://api.restful-api.dev/",
	"RawRequests" -> <|
		"RawObjects" -> <|"URL" -> "objects", "QueryParameters" -> {"id"}, "HTTPSMethod" -> "GET"|>
	|>
|>;

ServiceFramework`DefineServiceConnection[base]

ServiceExecute["RestfulAPI","RawObjects","id"->5.1]

In the simplest case, a processed request could simply be

processed = <|
	base,
	"ParameterVerification" -> {{_, "ID", _?Positive}},
	"ParameterNormalization" -> {{_, "ID", {r_Real :> Round[r]}}},
	"ParameterMap" -> <|"ID" -> "id"|>,
	"ProcessedRequests" -> <|
		"Objects" -> <|
			"ExecuteFunction" -> ServiceFramework`Request["RawObjects"],
			"Parameters" -> {"ID" -> Automatic}
			|>
		|>
|>;
ServiceFramework`DefineServiceConnection[processed]

Now we get all the parameter verification and processing:

ServiceExecute["RestfulAPI","Objects","invalid"->5]

ServiceExecute["RestfulAPI","Objects","ID"->"wrong"]

ServiceExecute["RestfulAPI","Objects","ID"->5.1]

ServiceExecute["RestfulAPI","Objects","ID"->5]

We can define a value for ServiceSubmit:

ServiceFramework`DefineServiceConnection[
<|
	processed,
	"ProcessedRequests" -> <|
		"Objects" -> <|
			"ExecuteFunction" -> ServiceFramework`Request["RawObjects"],
			"SubmitFunction" -> ServiceFramework`Request["RawObjects"],
			"Parameters" -> {"ID" -> Automatic}
			|>
		|>
|>];

res={};
t=ServiceSubmit["RestfulAPI","Objects","ID"->5,HandlerFunctionsKeys->{"BodyChunk"},HandlerFunctions-><|"BodyChunkReceived"->(AppendTo[res,#]&)|>];
TaskWait@t;
res

We cold use a more complicated “ExecuteFunction” definition:

ServiceFramework`DefineServiceConnection[
<|
	processed,
	"ProcessedRequests" -> <|
		"Objects" -> <|
			"ExecuteFunction" -> Function[
				Module[
					{res, possibleIDs},
					Echo[Row[{"The parameters for \"ExecuteFunction\" are: ", Sequence@@Riffle[{##}, ", "]}]];
					res = ServiceFramework`Request["RawObjects"][##];
					If[Length[res] === 0,
						possibleIDs = ServiceFramework`Request["RawObjects"][{}, ##2];
						Failure["NoItemWithSpecifiedID", <|"PossibleIDs" -> possibleIDs[[All,"id"]]|>]
						,
						res
					]
				]
			],
			"Parameters" -> {"ID" -> Automatic}
			|>
		|>
|>];

ServiceExecute["RestfulAPI","Objects","ID"->100]

ServiceExecute["RestfulAPI","Objects","ID"->12]

Thanks!!!

Init

ServiceConnect;
ServiceFramework`Management`PackagePrivate`wipeAllCredentials[]