Skip to content

Isolation & the trust boundary

touchstone runs an agent's shell commands and file edits against a Sandbox, through a Cell's Executor. It is important to be precise about what that isolates — because the default is not a security boundary.

Read this before running an untrusted Case or an untrusted model

With the default LocalExecutor, Case-authored shell and every model tool-call run on the host, with your full environment — API keys included. Only a Case container: block gives you a real isolation boundary. If you do not fully trust both the Case and the model, use a container.

What LocalExecutor actually isolates

The default Executor (LocalExecutor, used whenever a Case has no container: block) runs every command as a host subprocess. Its "isolation" is exactly two things:

  • cwd — commands run in the Cell's sandbox directory, so relative paths stay local, and the harness's file/search tools are path-confined to the sandbox (sandbox_fs, path-escape attempts are rejected).
  • env — the framework never mutates os.environ; each command gets an explicit environment (ADR 0002, ADR 0004).

That is all. There is no process, filesystem, or network boundary. A setup.run step, a command/pytest grader, and — critically — a shell command the model chooses to run all execute on the host as your user, seeing whatever your host environment contains (including API keys and other secrets). cwd/env isolation keeps parallel cells from stepping on each other; it is not a security sandbox. (A16)

The openai harness bash tool

The openai harness exposes a bash tool that runs a model-supplied string. On a LocalExecutor that string runs on the host, with the full host environment, exactly like any other host command.

The harness's other tools (read_file/write_file/edit_file/grep/list_dir) are confined to the sandbox by path resolution — but that confinement is cosmetic for security purposes as long as bash exists: a model can simply run bash to read ~/.aws/credentials, exfiltrate an env var, or reach the network. Path-confining the file tools does not contain the shell tool. So without a container: block, sandbox_fs confinement gives you clean traces, not containment. (B3)

To make this impossible to miss, the harness emits a loud, one-time warning the first time bash is about to run on a host executor, naming the risk and pointing here.

Untrusted Cases or untrusted models REQUIRE a container

If either the Case (its prompt, setup, graders) or the model is not fully trusted, a container: block is required for real isolation — it is the only backend that actually confines the agent's effects:

container:
  image: python:3.12-slim
  harness: true                       # also run the agent's effects inside the sandbox (ADR 0014)
  env_passthrough: ["OPENAI_API_KEY"] # ONLY the vars you name cross the boundary
  # backend: harbor                   # remote sandboxes (ADR 0015) for stronger isolation

With a container:

  • The agent's bash, file writes, provisioning, and graders all run inside the container, not on your host (ADR 0014).
  • Only the host env vars you list in env_passthrough cross into the container; the rest of your host environment (and its secrets) stay out.
  • For stronger, remote isolation, set backend: harbor to run the Cell in a Harbor sandbox (ADR 0015).

The in-process openai harness needs nothing baked into the image — its loop stays in the controller and only its effects are routed into the sandbox — so opting into a container is cheap. See Authoring cases → container.

Opt-in hard gate

Beyond the warning, you can make an unconfined bash a hard error. Set the environment variable:

export TOUCHSTONE_REQUIRE_CONTAINER=1

With this set, the openai harness refuses to run bash on a host executor (returning an error to the model instead of executing) unless the Cell has a container: block. It is off by default so existing Cases are unaffected; turn it on to enforce "never run model shell on the host" across a run.

Summary

Setup cwd/env isolation Real trust boundary Safe for untrusted Case/model
LocalExecutor (default, no container:) yes no no
container: (docker) yes yes yes
container: { backend: harbor } yes yes (remote) yes