Skip to content

Stopping at a plan-limit warning #1008

Closed

Description

@JacobStephens2

Parent

#977

What to build

thirdshift reads each Harness's limit warnings:

  • Claude's rate-limit events, both warnings and rejections;
  • Codex's rate-limit reports, at a threshold;
  • the other Harnesses' warnings, once Live checks for the Trial sandbox #1009's checks show how each one reports them.

At a warning, an Experiment starts no new Trial, and a Trial cut short is void. The Experiment pauses until it's run again. #978's Scoring run uses the same detection.

Acceptance criteria

  • Stream tests detect Claude's warning and rejection events and Codex's limit reports.
  • An Experiment that sees a warning starts no new Trial and pauses, and running it again resumes it.
  • Runs are unaffected. Only Experiments and the Scoring run stop.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified, ready for an AFK agent

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions