Skip to content

Native Linux nodes

canopy forest join makes a Linux machine a native compute node of your team: it enrolls, heartbeats, offers its NVIDIA GPU and runs the pipeline steps placed on it. Every step runs in a Docker container, never directly on the machine. This is available to every team.

  • A Linux machine (x86-64 or arm64) with systemd, about 5 GiB of free disk in your home directory, and a user session (log in on the machine, or over ssh with a full login).
  • Docker Engine, which your user may use (docker version works without sudo).
  • For GPU steps: an NVIDIA driver and nvidia-container-toolkit configured for Docker (sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker).
  • The Canopy CLI, logged in (canopy login), and to be an administrator of the team the machine will join.

No root access is needed to join. Nothing outside your home directory and your user’s systemd manager is changed.

canopy forest join # add --no-auto-placement to keep it to steps that name this machine

The command checks the machine, then:

  1. mints a single-use enrollment token for your team,
  2. downloads the signed release and verifies its Ed25519 signature and sha256 digest before anything runs,
  3. shows one summary of what will be installed and asks once (--yes skips only that question),
  4. installs the agent under ~/.local/share/canopy-forest and starts it as the systemd user service canopy-forest-agent, and turns on linger so it keeps running when you are logged out,
  5. waits for the agent’s first accepted heartbeat, removes the enrollment token from disk, and runs functional checks: the service runs, Canopy lists the machine, the heartbeat is fresh, Docker answers, a container starts the way the agent starts a step (as your user, every capability dropped), and, when the machine has a GPU, a container can see it.

It reports success only when every check passed, and prints the result to Canopy so the machine’s page shows it. Running canopy forest join again verifies and repairs; it never enrolls twice. --ingress is for a Mac and is refused here.

If a GPU is visible to nvidia-smi but Docker has no nvidia runtime, the machine joins without offering the GPU and the command says how to fix it; run canopy forest join again afterwards.

A step reaches a Linux machine when it says native: { platform: 'linux/amd64' } (or linux/arm64) and names a digest-pinned image:

step('train', {
native: { platform: 'linux/amd64', pool: 'gpu' },
image: 'pytorch/pytorch@sha256:...',
run: 'python train.py',
resources: { memoryMb: 4096, cpus: 4, accelerator: { type: 'gpu', vendor: 'nvidia', memoryMb: 4096 } },
})

The agent runs the step with docker run as your user, with every capability dropped and no-new-privileges, with the step’s memory, cpu and pid limits enforced, its workspace mounted at /workspace, a private HOME and /tmp, and the team’s cache at /cache (CANOPY_CACHE_DIR). The environment of your shell and the rest of the machine are not passed in, and a secret reaches the container through the Docker client’s environment, never its command line. The container is removed when the step ends, is cancelled or times out, and a container left by an agent that died is removed when it starts again. A step cannot enforce an egress allowlist here (egress may only be ['*']).

An image on one of Canopy’s registries needs no login on the machine: for each job Canopy mints a credential that may pull that one repository, for four hours, and the agent logs the Docker client in with it for the pull only. The login lives in a private directory that is removed when the step ends, never reaches the container or its environment, and is masked in the step’s output. An image on any other registry is pulled as the machine itself would pull it.

One GPU step runs per GPU. A step asking for count: 2 gets two GPUs, and waits while fewer are free. memoryMb on the accelerator is checked against the memory of the smallest card. See Native Groves for placement, pools and automatic placement, which work as they do for a Mac.

canopy forest status # the team's native Groves, and this machine's service and heartbeat
canopy forest verify # re-run the functional checks and record them with Canopy
canopy forest doctor # redacted diagnostic bundle: preflight, service, agent status, log, Canopy's view
canopy forest update --check # is a newer signed release available?
canopy forest update # install it, restart, wait for a heartbeat, go back if there is none
systemctl --user status canopy-forest-agent
journalctl --user -u canopy-forest-agent -f
Symptom Cause and fix
NVIDIA ... (not usable from Docker) or the gpu step is skipped Docker has no nvidia runtime. Install nvidia-container-toolkit, run sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker, then canopy forest join again.
The runtime is configured but docker info --format '{{json .Runtimes}}' does not list nvidia docker is talking to another daemon, usually Docker Desktop (docker context ls shows desktop-linux *). Run docker context use default, or remove Docker Desktop and keep Docker Engine. The agent and every Docker call of canopy forest always use the default context, so a selected Desktop context no longer takes over the agent.
error getting credentials ... docker-credential-desktop: executable file not found ~/.docker/config.json still names a credential helper of an uninstalled Docker Desktop. Delete the "credsStore" line. Every image pull fails until you do.
The GPU shows in verify but Canopy lists no accelerator The agent was started before the GPU was usable. canopy forest join rewrites the service and restarts it when its unit differs (CLI 1.6.1 and later); on older CLIs run systemctl --user restart canopy-forest-agent.
nvidia-smi: error while loading shared libraries in the GPU check The check image has no glibc. Update the CLI (canopy update); the check runs in a glibc image since 1.5.1.
canopy forest leave # revoke the Grove in Canopy, stop and remove the agent
canopy forest leave --purge # also remove its identity and the team's cache

leave revokes the Grove in Canopy first; if Canopy refuses, nothing is removed and you can run it again.

--json prints one JSON object per step and a final result. Exit code 0 means success, 1 a failed step, and 2 a machine or invocation that cannot proceed (not Linux x86-64 or arm64, no Docker or no right to use it, no systemd user session, not enough disk, no terminal and no --yes, several teams and no --team).