Skip to content

Live checks for the Trial sandbox #1009

Closed

Description

@JacobStephens2

Parent

#977

What to build

These are Day shift checks on the thirdshift host, made before the sandbox is built on them. They're for a human because they need the real subscriptions and logins.

  • Does each Harness log in, refresh and save its login inside bwrap, with only its login file bound? Claude uses a claude setup-token token in its environment instead.
  • Does each Harness work through HTTPS_PROXY over a relay to a Unix socket? Which hosts does each provider need?
  • How do you turn each Harness's web search off?
  • Does a fresh home turn each Harness's memory off?
  • Does interruption still reach the Harness through bwrap's process namespace?
  • How much memory does a Trial really use beside two building Runs?

The results go in the sandbox research note.

Progress (2026-10-21)

The checks that need no real login are done, with test processes, fake credentials and fresh homes. Their results are in section 10 of the sandbox research note (#1045).

  • Done:
    • Interruption reaches the Harness through bwrap's process namespace, by thirdshift's own stop path, with and without --new-session, and nothing outlives the sandbox.
    • A fresh home turns memory off for all six Harnesses: each keeps its memory under its home.
    • HTTPS_PROXY over the relay works, and all six Harnesses honour it. OpenCode needs NO_PROXY=127.0.0.1,localhost. The hosts each contacted with a fake credential are recorded.
    • Web search off: the switches for Claude, Codex, Grok, Muse and OpenCode are recorded.
  • Still open, since these need the real logins:
    • each Harness logging in, refreshing and saving its login inside bwrap, with only its login file bound, and Claude with a claude setup-token token (Turn on lingering and save Claude's setup token #1011);
    • the hosts used with a real subscription login, such as Codex's ChatGPT login and refresh, and agy's API host;
    • agy's web search, which has no switch in its help or settings;
    • a Trial's memory beside two building Runs. A snapshot of the sessions running now is recorded.

Acceptance criteria

  • Each question is answered for all six Harnesses where it applies, and recorded in the sandbox research note.

Activity

  1. JacobStephens2 commented on Oct 21, 2026

    @JacobStephens2
    OwnerAuthor

    This was generated by AI.

    I ran the checks that need no real login. They used test processes, fake credentials and fresh homes, so no real login was bound or read. The results are in section 10 of the sandbox research note, in #1045.

    Answered:

    • Interruption: thirdshift's stop path walks the process tree and signals every group it finds. Done that way, SIGTERM and SIGINT both reach a process inside bwrap's PID namespace, with and without --new-session. Killing the outer bwrap leaves nothing running inside.

    • Memory: all six Harnesses keep their state under their home, so a fresh home per Trial starts each with no memory. That includes Codex's memories_1.sqlite, and agy's brain and implicit.

    • HTTPS_PROXY: the relay works, and all six honour it.

      • Codex also syncs plugins from GitHub at startup, and carries on when that's refused.
      • OpenCode needs NO_PROXY=127.0.0.1,localhost to reach its own local server, and must be given a provider, or it falls back to a free model.
      • The proxy refuses GitHub, and names that resolve to loopback.
    • Web search off:

      • Claude: --disallowedTools WebSearch WebFetch;
      • Codex: -c web_search="disabled";
      • Grok: --disable-web-search;
      • Muse: --disable-web-tools;
      • OpenCode: OPENCODE_CONFIG_CONTENT with webfetch and websearch denied.

      Production sessions pass none of these today. Only release summaries do.

    Still open, because these need your real logins:

    • each Harness logging in, refreshing and saving its login inside bwrap, with only its login file bound, and Claude with a setup token (Turn on lingering and save Claude's setup token #1011);
    • the hosts used with a real subscription login;
    • agy's web search, for which I found no switch. Its google_search grounding runs at Google, so a proxy can't stop it;
    • a Trial's memory beside two building Runs.

    I didn't bind real logins, because a Codex refresh that fails inside a sandbox can leave production holding a used refresh token. #1012 waits on these checks, and #1013 and #1015 wait on #1012.

  2. JacobStephens2 commented on Oct 23, 2026

    @JacobStephens2
    OwnerAuthor

    This was generated by AI.

    Progress (2026-10-23)

    I ran the checks that need real logins on the thirdshift host. Each Harness ran in bwrap with:

    • a fresh home;
    • only its login file bound;
    • the network only through a proxy that logs each host.

    Nothing but inodes, sizes and modification times was recorded from the logins. The results are in section 11 of the sandbox research note, in #1179.

    • Logins inside bwrap.
      • Codex, Grok 1.0.67, Muse and agy answered with only their login bound.
      • agy refreshed inside the sandbox and wrote its token in place, and agy outside the sandbox still signed in with it.
      • OpenCode answered, given a copy of its database.
      • Claude: 401 with the saved claude.setup_token, and the same outside the sandbox. The saved token is 76 characters, most likely half of a wrapped paste. Waiting on the token being saved again.
      • Codex's refresh wasn't due. Its access token expires on 2026-10-24 at 20:27.
    • Hosts with real logins, for The Trial sandbox's network: an allowlisting proxy #1013's allowlist. They're recorded per Harness. Claude's hosts after sign-in are missing until its token works.
    • agy's web search can't be turned off. search_web still ran with read_url(*) and execute_url(*) denied, and with the strict preset. agy rejects search_web as a deny entry. The search goes through agy's API host, so a proxy can't stop it either.
    • A Trial's memory.

    New for #1012:

    • Grok saves its login by rename. Its auth.json got a new inode when its login was renewed, so a renewal inside the sandbox can't be saved through a single-file bind.
    • OpenCode 2.0.24 keeps its key in opencode.db, beside its sessions, and no longer reads OPENCODE_AUTH_CONTENT. A Trial needs the key through the config's {env:…} substitution, or a database of its own through OPENCODE_DB.
    • Codex needs the folder around its binary, for codex-code-mode-host.

    This stays open until Claude passes with a working setup token.

  3. JacobStephens2 commented on Oct 23, 2026

    @JacobStephens2
    OwnerAuthor

    This was generated by AI.

    Progress (2026-10-23, later)

    • Claude with a setup token. After the token was saved again, now 108 characters, Claude Code 2.1.309 answered in bwrap with no login file bound, only CLAUDE_CODE_OAUTH_TOKEN. Its hosts were:
      • api.anthropic.com;
      • http-intake.logs.us5.datadoghq.com, its telemetry.

    All four questions are now answered for all six Harnesses, in #1179, which closes this issue when it merges. One thing is still to be seen: a Codex refresh inside the sandbox. None was due, since its access token expires on 2026-10-24 at 20:27. Codex's source writes in place, and an in-place write through a single-file bind was confirmed with a test file.

  4. added a commit that references this issue on Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions