Void Trials #1007

Parent

#977

What to build

The Void Trial rules are fixed with the Bar.

• A Trial is void when:
  • the sandbox didn't start;
  • the Harness couldn't reach its provider;
  • the login failed or expired;
  • the provider returned a server error before any work;
  • a plan-limit stop ended it;
  • the host, or a memory cap that Runs don't have, killed it.
• A fail, not a void: a timeout, a context-window overflow, a refusal, or anything the agent did inside its sandbox.
• Re-runs: a Void Trial is re-run fresh, up to twice, and shown with its cause, counted per Arm.
• Drop-outs: an Eval task whose Trial is still void after its re-runs drops out for both Arms. If more than a tenth drop out, the Verdict is inconclusive.
• Caps: an Arm that keeps hitting its memory cap is reported as a finding.

Acceptance criteria

• A fake login failure makes a Trial void. It's re-run, and its Eval task drops out after two more failures.
• A timeout counts as a fail, never a void.
• Drop-outs leave both Arms, and the Verdict becomes inconclusive past a tenth.
• Void causes are shown per Arm in thirdshift experiments <id>.