Skip to content

Your rechecks of borderline and decisive Scores #1006

Closed

Description

@JacobStephens2

Parent

#977

What to build

  • A CLI command records your recheck of a Trial's Score, pass or fail with a reason. It's kept beside the Grader's Score, with its history.
  • Each Experiment lists the Scores waiting on you: the borderline ones, and the decisive ones, whose flip would change the Verdict.
  • Its Verdict is final only once you've rechecked them all. Until then it reads as waiting on rechecks.

Acceptance criteria

  • An Experiment with a decisive Score shows it as waiting on rechecks, and recording your recheck finalizes the Verdict.
  • Both the Grader's Score and yours are kept.
  • thirdshift experiments <id> --json lists the Scores waiting on a recheck.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified, ready for an AFK agent

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions