Compute jobs
A compute job is a short, ordered list of steps that run in digest-pinned
images on a build Grove. It is a pipeline run of kind compute, so it goes
through the same planner (policy, compute budget, accelerator entitlement and
budget), the same placement and the same admission as a CI step. There is no
separate scheduler.
Compute job routes require a bearer token and team actions: compute:read
for reads and compute:submit for submission and cancellation. These map to
viewer and developer roles respectively, and API keys can carry the matching
capability grants. Compute is not gated by a host.compute platform feature;
GPU requests can still be refused by accelerator entitlement, budget, or
placement policy.
Submitting
Section titled “Submitting”POST /compute/jobs{ "team_id": "<uuid>", "steps": [ { "name": "prep", "image": "registry.example.com/tools@sha256:...", "run": "python prep.py" }, { "name": "train", "image": "registry.example.com/train@sha256:...", "run": "python train.py", "resources": { "memory_mb": 16384, "cpus": 4, "accelerator": { "type": "gpu", "count": 1, "vendor": "nvidia" } }, "timeout_seconds": 3600 } ], "placement": { "grove": "my-grove" }}The answer is 202 with the job. Its id is the pipeline run id. Poll
GET /compute/jobs/{id}, sdk.computeJobs.get(), canopy compute jobs get,
or the host_get_compute_job MCP tool.
Use the Host SDK
Section titled “Use the Host SDK”The computeJobs resource exposes submit, list, get, logs, artifacts, and
cancel methods. Initialize @canopy/sdk with an API key that has the
required team capability; see SDK authentication.
import { createCanopySDK } from '@canopy/sdk';
const sdk = createCanopySDK({ apiKey: process.env.CANOPY_API_KEY });const job = await sdk.computeJobs.submit({ teamId: process.env.CANOPY_TEAM_ID!, steps: [ { name: 'hello', image: 'oven/bun:1-alpine@sha256:5acc90a93e91ff07bf72aa90a7c9f0fa189765aec90b47bdbf2152d2196383c0', run: 'bun --version', }, ],});
const state = await sdk.computeJobs.get(job.id);const logs = await sdk.computeJobs.logs(job.id, { limit: 100 });console.log(state.status, logs.logs);submit() returns after the server accepts the job; it does not wait for the
steps to finish. Keep the API key in an environment variable and never put
secret values in a job’s env map.
Submit safely more than once
Section titled “Submit safely more than once”Give a job a clientReference (1 to 128 characters from A-Za-z0-9._:-) and a
repeated submit of the same request returns the job that already exists instead
of starting another. The same reference with a different request answers 409
(compute_client_reference_conflict); another team may use the same reference
for its own job. After a submit that timed out, find out whether the job was
created before submitting again:
const existing = await sdk.computeJobs.getByClientReference(teamId, "nightly-2026-11-02");labels (up to 16 string pairs) are stored and returned with the job. They are
for your own bookkeeping and never decide who may see or change a job.
Use the CLI
Section titled “Use the CLI”Submit a one-step job with flags:
canopy compute jobs submit \ --team "$CANOPY_TEAM_ID" \ --image 'oven/bun:1-alpine@sha256:5acc90a93e91ff07bf72aa90a7c9f0fa189765aec90b47bdbf2152d2196383c0' \ --run 'bun --version'For multiple steps, write the API-shaped request to a JSON file and submit
it with --file. A --team value overrides team_id in the file.
canopy compute jobs submit --team "$CANOPY_TEAM_ID" --file compute-job.jsoncanopy compute jobs submit --team "$CANOPY_TEAM_ID" --file compute-job.json --client-reference nightly-2026-11-02 --label owner=mlcanopy compute jobs get "$JOB_ID"canopy compute jobs get --team "$CANOPY_TEAM_ID" --client-reference nightly-2026-11-02canopy compute jobs logs "$JOB_ID" --limit 100canopy compute jobs artifacts "$JOB_ID"canopy compute jobs cancel "$JOB_ID"List jobs with canopy compute jobs list --team "$CANOPY_TEAM_ID"; use
--status, --client-reference, --limit, and --cursor to filter and continue pagination.
The CLI accepts the same team permissions as the API. --json is available
on the commands for machine-readable output.
Use the Host MCP server
Section titled “Use the Host MCP server”The Host MCP server registers compute tools in the opt-in compute profile.
Configure that profile as described in MCP server setup.
The tools are host_submit_compute_job, host_get_compute_job,
host_list_compute_jobs, and host_cancel_compute_job. Selecting a profile
only controls which tools the server exposes; the API still checks the
authenticated user’s compute:read or compute:submit permission. The cancel
tool is destructive, and the server marks it accordingly.
| Step field | Meaning |
|---|---|
name |
Required. Lowercase letters, digits, - and _, up to 63 characters, unique in the job. |
image |
Required for a container step. Must be pinned by digest (@sha256:...); a tag is a 400. A native step on a Mac takes none. |
run |
Required for a container or native step. The shell command. |
secret_env |
Names of secrets delivered to the step as environment variables of the same name (see Secrets). |
native |
Runs the step on a native machine of the team: platform (linux/amd64, darwin/arm64), and optionally grove, pool or unavailable (wait or fail). Trusted jobs only; egress is absent or ["*"]. |
source |
A checkout step instead of a command: provider (github, gitlab, bitbucket), repository and a ref or a commit (see Check out a repository). |
env |
Plain variables, at most 64. Never put secrets here. The CANOPY_ prefix is reserved. |
egress |
Hostnames the step may reach (a single leading *. label is allowed). Empty or absent means no network. |
resources |
memory_mb (64 to 262144), cpus (0.1 to 128) and accelerator (type: gpu, count, vendor nvidia or apple, memory_mb). |
timeout_seconds |
1 to 86400. |
retry |
max_attempts (1 to 10) and backoff_seconds. |
inputs |
Names of earlier steps this step waits for. Omitted means the step before it; an empty array means none. |
A job has 1 to 20 steps, and a step can only name earlier steps in inputs.
Unknown fields are rejected with a 400. Reuse is always off: a job never skips
a step because an identical one ran before.
Contain untrusted code
Section titled “Contain untrusted code”trust: "untrusted" runs the job contained: it cannot ask for an accelerator
(compute_untrusted_accelerator) or a native machine (compute_untrusted_native),
and its steps run in a VM when the team has VM isolation. isolation: "vm" makes
that a requirement: when the team has no VM isolation the submit is refused with
compute_isolation_unavailable instead of running in a container. A trusted job
cannot ask for vm.
Check out a repository
Section titled “Check out a repository”A step with source checks a repository out with the team’s clone identity and
needs the compute:checkout action (developer level). The ref is resolved to a
commit when the job is submitted and the commit is shown on the step
(steps[].source). The checkout holds a read-only credential minted at dispatch
and deletes .git before the workspace is captured, so no later step sees a
credential or a remote. A repository without a usable clone identity answers 422
(compute_source_identity_missing); an unknown ref answers 422
(compute_source_ref_not_found). GitHub credentials expire within an hour;
GitLab and Bitbucket use the team’s stored token, which only the checkout step
holds.
Secrets
Section titled “Secrets”secrets carries up to 16 values the job brings, each named in some step’s
secret_env. They are held in Vault only, delivered to the named steps as
environment variables at dispatch, deleted when the job ends, and never appear in
the stored job, a log or any response. The request hash for client_reference
counts the secret names, not their values. A secret_env name that is not in
secrets is read from the team’s pipeline secrets.
placement.grove pins every step to the named Grove. The Grove must have the
build role and must belong to your team (staff may name any). Without it the
normal automatic placement picks the Grove.
In the app
Section titled “In the app”Team → Compute jobs lists the team’s jobs, newest first, and filters them by status. Open a job to see each step with its status, how long it ran, the machine it ran on and why that machine was chosen (the machines that were ready, the ones that had to be waited for and the ones refused with their reason), the output the machine reported, and a Cancel job button for a job that has not finished. Cancelling asks first.
The GPU budget card above the list shows the month’s accelerator seconds used against the allowance, how many accelerators one run may use, and whether a step over the budget is refused (hard) or runs with a warning (soft). The same numbers are at GET /teams/{team_id}/compute-budget (compute:read): max_accelerators_per_run, seconds_budget (null for no cap), budget_mode, source (team for an allowance Canopy set for the team, plan, default), plan_name, period_start, consumed_seconds, remaining_seconds and previous_consumed_seconds. Usage is counted per calendar month in UTC.
Reading and cancelling
Section titled “Reading and cancelling”| Route | Needs | Answer |
|---|---|---|
GET /compute/jobs?team_id= |
compute:read |
A page of job summaries, newest first. status, client_reference, limit (1 to 100) and cursor filter and page it. |
GET /compute/jobs/{id} |
compute:read |
The job with each step’s status, attempts, exit code, error and the Grove it was placed on. |
GET /compute/jobs/{id}/logs |
compute:read |
The output the Grove reported, oldest first. cursor, limit (1 to 1000, default 500) and step page and filter it. |
GET /compute/jobs/{id}/artifacts |
compute:read |
The output digests of the succeeded steps. Digests only: there is no download yet. |
GET /compute/jobs/{id}/artifacts/{step} |
compute:read |
The same for one step. |
POST /compute/jobs/{id}/cancel |
compute:submit |
Cancels every unfinished step and asks the Grove to stop what it started. The job reports cancelling until the Grove has stopped everything and cancelled only then; cancelling again returns the current job. A job that finished some other way answers 409. |
compute:read is a viewer-level action and compute:submit a developer-level
one; API keys can be granted either through capability grants. A job of another
team looks like a job that does not exist.
Each entry of GET /compute/jobs/{id}/logs has an id, the step it came
from, the stream (stdout or stderr), the text and created_at. A Grove
reports output in batches while a step runs, so a running job’s log trails the
step; a step that failed or timed out also carries the last lines of its output.
Pass the next_cursor of one page as cursor to get only what came after it.
next_cursor is the id of the last entry returned, the cursor you sent when
nothing new arrived, and null only before anything was ever returned. Log
rows are removed by the same retention job as build logs.
Job status is queued until a step starts, then running, then succeeded,
failed or cancelled; a job whose cancel was requested is cancelling until
every Grove-side job has stopped. When a step fails, the steps after it are
skipped.
Alternatives and placement
Section titled “Alternatives and placement”A step may list alternatives instead of image, run, native and resources: complete ways to run it on one platform
each, in order of preference. Host runs the step on the first alternative whose machine is free, so one job can ask for
“an NVIDIA container, or else an Apple host process”:
{ "team_id": "...", "placement": { "providers": ["own", "canopy"], "wait_seconds": 86400 }, "steps": [{ "name": "train", "alternatives": [ { "platform": "linux/amd64", "image": "registry.canopy.pm/platform-registry/trainer@sha256:...", "run": "python3 -m trainer", "resources": { "cpus": 4, "memory_mb": 16384, "accelerator": { "type": "gpu", "vendor": "nvidia", "memory_mb": 5200 } } }, { "platform": "darwin/arm64", "run": "bash run.sh", "resources": { "cpus": 8, "accelerator": { "type": "gpu", "vendor": "apple", "memory_mb": 9000 } } } ] }]}A linux alternative runs the digest-pinned image in a container (on a Kubernetes Grove or a native Linux machine); a darwin alternative runs the command on the machine. At most four alternatives; only trusted jobs may use them.
placement takes grove (pin every step to one Grove: one your team owns, one offered to every team, or any for staff),
pool (only machines with this placement label; exclusive with grove), providers (own for your team’s machines,
canopy for Canopy’s capacity; both by default, your own first), wait_seconds (60 to 604800, default 86400) and role (accelerator, inference or runner: required when an app service submits with its own key, refused for anyone else; the machines bound to the service for that role are tried first). A
step’s own native.grove or native.pool cannot be combined with placement.grove or placement.pool
(compute_placement_conflict). canopy compute jobs submit takes --grove, --pool, --providers own,canopy and
--wait <seconds>.
Waiting for a machine
Section titled “Waiting for a machine”A step that needs a machine nobody can give it yet stays pending, and the job
stays queued: a waiting step is not running. The step carries
waiting {reason, since, candidates} until it starts. The wait survives a
restart of Canopy; Host looks again every fifteen seconds or so and starts the
step the moment a machine is free.
waiting.reason |
Meaning |
|---|---|
gpu_busy |
every suitable GPU is held by another job |
machine_asleep |
the machine is asleep or offline; the step starts when it wakes |
machine_unavailable |
the machine is draining or in maintenance |
owner_paused |
the owner of the machine paused it, it is outside the hours the owner set, or it is on battery; the step starts when it accepts work again |
no_fitting_gpu |
no machine has a GPU that fits the request |
not_entitled |
Canopy’s capacity is the only fit and this team’s plan has no GPU allowance |
over_budget |
the team’s GPU seconds for the month are used up |
concurrency_limited |
the team already runs as many GPUs as its allowance allows |
capacity_busy |
the machine’s CPU, memory or job budget is held by other steps; the step goes back to the queue and is placed again |
pinned_unavailable |
the machine named in placement.grove is unavailable |
A step with native.unavailable: fail fails at once instead of waiting. A
waiting step gives up after placement.wait_seconds (a pipeline step: the
scheduling window) and fails naming the reasons it saw. Cancelling a job that is waiting ends it at once; no machine was ever
involved.
Once a step is placed it carries machine {grove, provider, accelerator, started_at, finished_at}: the machine it ran on, whether it was one of the
team’s own (own) or Canopy capacity (canopy), the GPU’s vendor, model and
memory, and the times the machine was actually held. started_at is when the
machine started the step, not when the job was submitted.
Link a Cortex organization
Section titled “Link a Cortex organization”In Canopy, open Organization settings → Cortex organization. You can enable Cortex for the current Canopy Organization or link a Cortex organization you belong to. The CLI can also show, link, or unlink the current relationship:
canopy teams cortex-org "$CANOPY_TEAM_ID"canopy teams cortex-org "$CANOPY_TEAM_ID" --link "$CORTEX_ORG_ID"canopy teams cortex-org "$CANOPY_TEAM_ID" --unlinkThe caller needs cortex_org:manage (admin role) and must be a member of the
Cortex organization being linked. Each Canopy Organization can link one Cortex
organization. When several Canopy Organizations link the same Cortex organization,
its jobs run for the first one; the others are linked without a Host identity
(host_link: failed) until the first unlinks.
Replacing a link removes the previous identity key before provisioning the new
one.
The link record is saved even if identity provisioning fails. The REST
PUT /teams/{team_id}/cortex-org response reports host_link: "provisioned"
or "failed". The current SDK wrapper and canopy teams cortex-org command
return or display the link record but omit this provisioning status. Call the
REST route directly when you need to confirm provisioning; repeating the link
request retries it.
When provisioning succeeds, Canopy creates a dedicated Cortex agent identity
with compute:submit, compute:read and compute:checkout, then gives Cortex
a 90-day key for that identity. Cortex stores the key encrypted and rotates it
when it is within 30 days of expiry; the previous key stays valid for ten
minutes so requests in flight can finish. Unlinking disables the identity,
revokes its keys, asks Cortex to forget the key, and requests cancellation for
running compute jobs carrying that Cortex organization’s cortex_org label.
Cortex agent runs execute through this link as untrusted, VM-isolated Compute Jobs. Until the runner image is configured, an agent run waits as unserved and starts as soon as it is. Treat Cortex agent execution as experimental. See the Cortex developer guide.
Compute objects: bytes in and out of a job
Section titled “Compute objects: bytes in and out of a job”A job step cannot take input artifacts yet, so a caller that has to hand a step a large input, or collect a large result, uses a compute object: a slot in Host’s object storage that belongs to your team.
POST /compute/objects with team_id and role (input or output, optional max_bytes; at most 2 GiB for an input and 4 GiB for an output) needs compute:submit and returns the object with two transfer URLs signed by Host. The caller fills an input object by sending the bytes with PUT to put_url (a Content-Length is required; one upload is accepted, a failed one can be retried); a step reads it with a plain GET of get_url, so the step needs no credential. For an output object the step does the PUT and the caller reads the result with GET /compute/objects/{id}/content (needs compute:read; X-Content-SHA256 is the hash Host recorded while storing). GET /compute/objects/{id} shows state (pending or ready), sha256 and size_bytes. The URLs work for 24 hours; the object and its bytes are deleted after 48 hours. Machines reach the object through Host (the storage behind it is private), so this works for a Mac or a Linux PC that joined with canopy forest join.
Not supported yet
Section titled “Not supported yet”These answer a 422 with a machine-readable error_code rather than being
ignored:
input_artifacts(compute_input_artifacts_unsupported).
Also missing: downloading artifacts. SDK artifact methods return digests only; they do not download artifact content.
A step runs on the Grove the same way a CI step does, but a job has no build
behind it, so nothing about it appears in build pages, build activity or
lifecycle events: its state is the job (GET /compute/jobs/{id}) and its output
is the job’s logs. While a step runs the engine worker that started it waits for
it, so a long job holds one pipeline runner slot until it finishes.