Running your builds on your own machines
A native Groves machine of your team can take ordinary pipeline work: container steps and image builds. Three things have to agree before a step goes there, and each can say no:
- The team’s execution policy says which work may leave Canopy’s machines.
- The owner of the machine says which work it takes, when it joined:
canopy forest join --accept steps,image-builds,secrets. Nothing is accepted by default. - The step itself: a step that names
native:still runs where it names; a pull request from a fork never leaves Canopy’s machines.
The execution policy
Section titled “The execution policy”canopy execution-policy show --team <team_id>canopy execution-policy set --team <team_id> --own-machines allowed --image-builds allowedcanopy execution-policy show --env <environment_id> --effectiveThe team’s policy is the ceiling: a project or an environment can only restrict it. own_machines and image_builds are
off, allowed or preferred (the default is off); fallback says whether work may go back to Canopy’s machines when
none of yours can take it, and step_secrets and open_egress default to the safe side. A step can also ask in the pipeline:
step('test', { image: CI_IMAGE, run: 'bun test', placement: { providers: ['own', 'canopy'] } });A step can also name one machine or a pool of your machines, and say what to do when none can take it:
step('test', { image: CI_IMAGE, run: 'bun test', placement: { providers: ['own'], grove: 'my-mac', unavailable: 'fail' } });The step’s own words come first, and the team’s policy is still the ceiling:
providerslimits where the step may run. A step that leavescanopyout never goes to Canopy’s machines, whatever the team’sfallbacksays.groveandpoollimit the machines of yours that may take it. A step that names a machine is never moved to another one or to Canopy’s machines.unavailable: 'fail'fails the step at once when no allowed machine can take it, with the reason each machine gave. The default,wait, queues it until the wait ends.- The team’s
fallbacklets a step with no such restriction go back to Canopy’s machines. It never overrides a step’s own restriction.
canopy pipelines plan <graph> --explain --verbose --node <node> prints where a step could run and why not on each machine, for ordinary steps and image builds the policy offers to your machines as well as for steps that name one:
Placement now: build-image (offered to your own machines first) ready: singularity-9dcc refused: gios-macbook-pro-9d5a - incompatible:platform_mismatch (this machine builds linux/arm64, the image is for linux/amd64)A machine is ready when it could take the step now, waiting when it will once something changes (asleep, paused, full), and refused when it cannot, with the reason. When more than one machine is ready, the one with the most free budget is chosen. A step that is not offered to your machines (Canopy compute comes first in your policy) shows only the policy.
What runs, and where
Section titled “What runs, and where”- Container steps run in a read-only container on a Linux machine, with the CPU share low enough that your own work on the machine wins, and the network off when the step declares none. A step that lists hosts, and a step that uses secrets on a machine whose owner has not allowed secrets, stay on Canopy’s machines.
- Container steps on a Mac run in a Linux guest created for that one step, with your step’s directories shared into it and no other part of the Mac visible. The guest has no network interface, gets 2 CPUs and 2 GiB unless the step names limits, and is removed when the step ends. A Mac takes a
linux/arm64step, and alinux/amd64step under Rosetta, only after its agent ran a test step in a guest of that platform. A step that lists hosts inegressruns there too: the guest reaches only those hosts, on ports 80 and 443 and never a private address, through a proxy on the Mac that the guest talks to over a relayed socket, and the Mac takes such steps only after a guest proved that path (the allowed host answers, another is refused, going round the proxy fails). A step that needs the open network stays on another machine. A step that namesnative:is not affected. - Image builds (Dockerfile and buildpacks) run in a rootless BuildKit container on a Linux machine, and in a Linux guest created for that one build on a Mac. The image records which machine built it.
- Several platforms:
canopy environments update <environment_id> --build-platforms linux/amd64,linux/arm64builds one image per platform and publishes one image. Canopy has no arm64 build machine, so the arm64 image needs a machine of yours; the build plan fails before anything runs when there is none. See Native Groves.
What a machine does not protect
Section titled “What a machine does not protect”A step on your own machine is not sandboxed from that machine the way a step on a Canopy build machine is.
- A capability is offered only while it works. A machine offers image builds only after a real test build passed on it, and it runs that test again every six hours: if Docker has stopped, the builder image is gone or the login no longer works, the machine stops offering image builds at once, says why in its log, and offers them again when the test passes.
- A job cannot fill your disk. Every 15 seconds the agent reads the free space of the disk the jobs live on, and a job is stopped when it falls below 2 GiB or 5% of the disk, whichever is larger. The job fails with a reason that names both numbers, and the machine keeps what it needs. The Docker data directory is not watched.
- A container step or image build runs with the limits above, but the network is open unless the step has none or lists the hosts it needs, and a Dockerfile can reach everything the machine can reach, including other machines on your network. A container step that lists hosts in
egressruns on a Linux machine in a container with no network at all: it reaches only a proxy the agent runs, through a unix socket, and the proxy lets through only those hosts, on ports 80 and 443, and never a private address. It cannot reach the machine’s own services or any other address, and the proxy of one job is not reachable from another. An image build still has an open network. - A Mac builds and runs container steps in a guest, but a step that names
native:runs as an ordinary process of the user the agent runs as. - Whatever a step reads from your repository is on the machine while it runs.
- Your own work and a step share the machine’s memory and disk; the owner’s controls (pause, hours, a battery stop, a CPU and memory budget) say how much a step may use.
Use machines you would trust with the code of the repositories whose steps you send there.
The owner of the machine
Section titled “The owner of the machine”canopy forest configcanopy forest config set --paused truecanopy forest config set --window 22:00-07:00 --refuse-on-battery true --cpu-budget-milli 4000 --memory-budget-mib 8192canopy forest config set --refuse-on-low-power true --refuse-when-hot truecanopy forest config runs on the machine itself and changes only the flags you pass. On a Mac it asks for your password,
because the agent’s settings belong to its service user. canopy forest status shows each machine’s budget, what running jobs
use and, when it is not accepting jobs, why.
A paused machine, one outside its hours, one on battery, one in Low Power Mode or one slowed down by heat (when you ask for those) stops accepting work; a step meant for it waits (owner_paused)
or goes to another machine. canopy forest groves update <name> --portable-steps on --image-builds on --secrets on changes
what a joined machine accepts.
When a machine goes away
Section titled “When a machine goes away”Each job holds a lease the machine renews every time it reports in. A machine that stops reporting loses its jobs after the lease runs out (five minutes): a container step that can safely run again is run again on another machine and is not counted against its retries, and anything else fails with a message that the machine was lost. A step that ran on a machine of yours is reused later only while your policy still lets that step run there.
Seeing where a step ran
Section titled “Seeing where a step ran”canopy builds show <build_id> --env <environment_id> and the pipeline view name the machine a step ran on and, for a
multi-platform build, the machine that built each platform. In the app, a step that was offloaded to one of your machines has a small
laptop icon next to its name on the build page, and hovering it names the machine; a step that ran on Canopy’s compute has none. The
Performance page of a machine is under Native Groves.
Building linux/amd64 images on an Apple silicon Mac
Section titled “Building linux/amd64 images on an Apple silicon Mac”A Mac builds linux/arm64 natively. It can also build linux/amd64 under Rosetta, and offers that only after it built a test image for
linux/amd64 in a guest itself (the agent’s own preflight, no flag to set; the machine’s owner can switch it off with
FOREST_IMAGE_BUILD_EMULATED_AMD64=false). The team’s execution policy decides whether a build may use emulation:
canopy execution-policy set --team <team_id> --emulated-builds allowed (the command replaces the whole level, so repeat the other
settings you want to keep). An emulated build is slower than a native one, so a machine that builds linux/amd64 itself is preferred.
Why a step on your machine failed
Section titled “Why a step on your machine failed”A step that fails on one of your machines carries a failure code in its reason, and the failure report says what it means:
| Code | Meaning | Retried |
|---|---|---|
registry_auth_failed |
The registry refused the machine’s credential during an image build. A fresh credential is issued on the next attempt. | Yes |
artifact_verification_failed |
An input artifact did not match its signed digest, so the machine refused to use it. | No |
unsupported_feature |
The machine lacks a feature the step needs (an older agent, no Rosetta lane, no egress filter). Another machine is tried when there is one. | Elsewhere |
platform_mismatch |
The step needs a platform the machine does not run. | No |
invalid_spec |
The machine refused the step’s specification. | No |
A step that only prints one of these words never changes how its failure is classified: a command that exits non-zero is your code.