G
G
Golem SDK
Search…
⌃K

Service Example 6: External API request

Example showing how to make a REST call to an external, public API from a VM running on a Provider node.
The example depicts the following features:
The full code of the example is available in the yapapi repository: https://github.com/golemfactory/yapapi/tree/master/examples/external-api-request

Prerequisites

As with the other examples, we're assuming here you already have your yagna daemon set up to request the test tasks and that you were able to configure your Python environment to run the examples using the latest version of yapapi. If this is your first time using Golem and yapapi, please first refer to the resources linked above.
This example involves Computation Payload Manifest.
Computation Payload Manifest making use of Outbound Network requires either:
  1. 1.
    Requestor certificate that's trusted by the Providers
  2. 2.
    an instance of a Provider with the particular domain this example uses added to its domain whitelist
  3. 3.
    an instance of a Provider with the requestor's self-signed Certificate imported into its keystore
The following example will show cases 2. and 3. so it will be necessary to start a local instance of a Provider.

Example app

An example app will request an external API using Provider's network and then it will print the API response to the console.

1. Manifest file

For an app to make an Outbound Network request it needs to declare which tools it will use and which URLs it will access in a Computation Payload Manifest.
Our example will make an HTTPS request using curl to a public REST API with the URL https://api.coingecko.com.
Computation Payload Manifest will need to have following objects:
  • net computation constraints with URLs the app will access (https://api.coingecko.com)
  • script computation constraint with commands app will execute (curl)
  • payload defining Golem image containing tools used by the app (curl)
Example Computation Payload Manifest must follow a specific schema, and for our example it will take form of following manifest.json file:
{
"version": "0.1.0",
"createdAt": "2022-07-26T12:51:00.000000Z",
"expiresAt": "2100-01-01T00:01:00.000000Z",
"metadata": {
"name": "External API call example",
"description": "Example manifest of a service making an outbound call to the external API",
"version": "0.1.0"
},
"payload": [
{
"platform": {
"arch": "x86_64",
"os": "linux"
},
"urls": [
"http://yacn2.dev.golem.network:8000/docker-golem-script-curl-latest-d75268e752.gvmi"
],
"hash": "sha3:e5f5ddfd649525dbe25d93d9ed51d1bdd0849933d9a5720adb4b5810"
}
],
"compManifest": {
"version": "0.1.0",
"script": {
"commands": [
"run .*curl.*",
],
"match": "regex"
},
"net": {
"inet": {
"out": {
"protocols": ["https"],
"urls": ["https://api.coingecko.com"]
}
}
}
}
}
Created file should be verified using JSON schema.
Then it needs to be encoded in base64:
base64 --wrap=0 manifest.json > manifest.json.base64

2. Yapapi example app

Base64-encoded manifest can be configured using yapapi.payload.vm.manifest function, resulting in following external_api_request.py file:
import asyncio
from yapapi import Golem
from yapapi.services import Service
from yapapi.payload import vm
class OutboundNetworkService(Service):
@staticmethod
async def get_payload():
return await vm.manifest(
manifest = open("manifest.json.base64", "rb").read()
# later we may add here manifest signature, digest algorithm, and app author's certificate
min_mem_gib = 0.5,
min_cpu_threads = 0.5,
# capabilities used to reach Provider with a correct VM Runtime
capabilities=["inet", "manifest-support"],
)
async def run(self):
script = self._ctx.new_script()
future_result = script.run(
"/bin/sh",
"-c",
f"GOLEM_PRICE=`curl -X 'GET' \
'https://api.coingecko.com/api/v3/simple/price?ids=golem&vs_currencies=usd' \
-H 'accept: application/json' | jq .golem.usd`; \
echo \"Golem price: $GOLEM_PRICE USD\";",
)
yield script
result = (await future_result).stdout
print(result.strip() if result else "")
async def main():
async with Golem(budget=1.0, subnet_tag="testnet") as golem:
await golem.run_service(OutboundNetworkService, num_instances=1)
await asyncio.sleep(60)

3. Verification of a request with Computation Payload Manifest

Providers verify the incoming request with a Computation Payload Manifest by checking if it arrives with a signature and App author's certificate signed by a certificate they trust. If there is no signature, they verify if URLs used by Computation Payload Manifest are whitelisted.
There are two ways to make our local Provider verify the request:
  • Whitelisting of the domain used by the app
    Add api.coingecko.com to Provider's domain whitelist:
    ya-provider whitelist add --patterns api.coingecko.com --type strict
  • Signing manifest and adding signature with a certificate to the request
    With a generated and base64-encoded certificate and a signature, the get_payload() function takes the following form:
    # ...
    async def get_payload():
    return await vm.manifest(
    manifest = open("manifest.json.base64", "rb").read(),
    manifest_sig = open("manifest.json.base64.sign.sha256.base64", "rb").read(),
    manifest_sig_algorithm = "sha256",
    manifest_cert = open("golem_requestor.cert.pem.base64", "rb").read(),
    min_mem_gib = 0.5,
    min_cpu_threads = 0.5,
    capabilities=["inet", "manifest-support"],
    )
    # ...

4. Launching the app

With both Requestor and Provider yagna nodes and ya-provider running in the background run:
python external_api_request.py
(keep in mind to set YAGNA_APPKEY and YAGNA_API_URL env variables pointing to the local Requestor node)