JS Task API Examples: selecting providers

Introduction

You can select providers using different criteria, i.e. defining requirements in a demand or applying filters on providers' proposals. You can:

Select a provider based on minimal requirements for remote computer CPU, disk storage, RAM.
Select a provider based on the whitelist/blacklist.
Select a provider based on the proposed costs using a custom filter.

Prerequisites

Yagna service is installed and running with the try_golem app-key configured.

How to run examples

Create a project folder, initialize a Node.js project, and install the @golem-sdk/task-executor library.

mkdir golem-example
cd golem-example
npm init
npm install @golem-sdk/task-executor

Copy the code into the index.mjs file in the project folder and run:

node index.mjs

Filtering providers based on minimal requirements:

You can define minimal requirements for an environment provided by a node by stating a minimal number of:

CPU cores minCpuCores,
RAM minMemGib,
disk space minStorageGib or
CPU threads minCpuThreads.

You can do this in the TaskExecutor options:

import { TaskExecutor, pinoPrettyLogger } from "@golem-sdk/task-executor";

(async function main() {
  const executor = await TaskExecutor.create({
    package: "golem/alpine:latest",
    //minCpuCores : 2,
    //minMemGib : 8,
    //minStorageGib: 10,
    minCpuThreads: 1,
    logger: pinoPrettyLogger(),
    yagnaOptions: { apiKey: "try_golem" },
  });

  try {
    await executor.run(async (ctx) => console.log((await ctx.run("echo 'Hello World'")).stdout));
  } catch (err) {
    console.error("An error occurred:", err);
  } finally {
    await executor.shutdown();
  }
})();

warning

Be careful, filtering is done internally by Yagna and if your requirements turn out to be too demanding you will not receive any proposals from providers and your requestor script will terminate after the timeout.

Job timeout log

Selecting providers based on the whitelist

In some situations, you might need your tasks to be executed on a certain provider or exclude specific providers. If you know providers' IDs or names you can use the proposalFilter option and use one of the predefined filters:

ProposalFilters.whiteListProposalIdsFilter(),
ProposalFilters.blackListProposalIdsFilter(),
ProposalFilters.whiteListProposalNamesFilter().
ProposalFilters.blackListProposalNamesFilter()

All these filters will accept an array with IDs, or names of the providers, which should be accepted or excluded.

info

For this example, you might need to update the provider's list in the whiteListsIds. Go to the Golem Network Stats and scroll the list to find a provider working on Testnet. Then click on its name and copy its ID.

import { TaskExecutor, pinoPrettyLogger, ProposalFilterFactory } from "@golem-sdk/task-executor";

/**
 * Example demonstrating how to use the predefined filter `allowProvidersByName`,
 * which only allows offers from a provider whose name is in the array
 */

const whiteListNames = ["provider-2", "fractal_01_3.h", "sharkoon_379_0.h", "fractal_01_1.h", "sharkoon_379_1.h"];
console.log("Will accept only proposals from:");
for (let i = 0; i < whiteListNames.length; i++) {
  console.log(whiteListNames[i]);
}

(async function main() {
  const executor = await TaskExecutor.create({
    package: "golem/alpine:latest",
    proposalFilter: ProposalFilterFactory.allowProvidersByName(whiteListNames),
    logger: pinoPrettyLogger(),
    yagnaOptions: { apiKey: "try_golem" },
  });

  try {
    await executor.run(async (ctx) =>
      console.log((await ctx.run(`echo "This task is run on ${ctx.provider.name}"`)).stdout),
    );
  } catch (err) {
    console.error("An error occurred:", err);
  } finally {
    await executor.shutdown();
  }
})();

info

You can read provider names from ctx workContext or from the proposal. We will look into proposals in the next section.

Selecting providers based on the proposed costs using a custom filter

In this example, we will show a custom filter that can be used to select the best provider. We will use it to filter based on the price, but it can be used to filter by any other attribute that is included in the provider proposals or even scan the market to see what is proposed.

The whole process begins with requestor demand. The network will respond with proposals from active providers. Proposals are then negotiated until an agreement is reached. Once the requestor and provider sign the agreement, activity can be started and TaskExecutor can execute them.

Let's see how to use it:

import { TaskExecutor, pinoPrettyLogger } from "@golem-sdk/task-executor";

/**
 * Example demonstrating how to write a custom proposal filter.
 */

var costData = [];

const myFilter = (proposal) => {
  let decision = false;
  let usageVector = proposal.properties["golem.com.usage.vector"];
  let counterIdx = usageVector.findIndex((ele) => ele === "golem.usage.duration_sec");
  let proposedCost = proposal.properties["golem.com.pricing.model.linear.coeffs"][counterIdx];
  costData.push(proposedCost);
  if (costData.length < 6) return false;
  else {
    costData.shift();
    let averageProposedCost = costData.reduce((part, x) => part + x, 0) / 5;
    if (proposedCost <= 1.2 * averageProposedCost) decision = true;
    if (decision) {
      console.log(proposedCost, averageProposedCost);
    }
  }
  console.log(costData);
  console.log(proposal.properties["golem.node.id.name"], proposal.properties["golem.com.pricing.model.linear.coeffs"]);
  return decision;
};

(async function main() {
  const executor = await TaskExecutor.create({
    package: "golem/alpine:latest",
    proposalFilter: myFilter,
    logger: pinoPrettyLogger(),
    yagnaOptions: { apiKey: "try_golem" },
    startupTimeout: 60_000,
  });

  try {
    await executor.run(async (ctx) => {
      const result = await ctx.run('echo "This task is run on ${ctx.provider.id}"');
      console.log(result.stdout, ctx.provider.id);
    });
  } catch (err) {
    console.error("An error occurred:", err);
  } finally {
    await executor.shutdown();
  }
})();

Note that customFilter is a function that accepts a proposal object as its parameter and should return true or false depending on the decision based on the proposal properties.

Our custom function collects pricing data until we have a set of 5 proposals. Then it accepts proposals only if the price is lower than average from the last five proposals.

Provider price is calculated as the product of prices defined per specific usage counter.

The counters are defined in golem.com.usage.vector property and the prices are defined in golem.com.pricing.model.linear.coeffs. The last element in the price coeffs array is a fixed element of the total price (one can consider it the one-time price for deployment).

Note that the sequence of the counters is not fixed, therefore we need to find the index of the specific counter. In our examples, we take into account only the price related to the usage of the total environment (as our example task has a very short execution time).