Require stable Freno recovery

[pradeepkintali]

Summary

• Add stable_recovery: false as an explicit, default-off rollout switch.
• Preserve first-success release and timeout circuit-breaker failure accounting
  for existing callers when upgrading the gem.
• With stable_recovery: true, retain the initially healthy fast path, but
  require three consecutive passing checks after rejection. Another rejection
  resets recovery progress.
• With the switch enabled, treat WaitedTooLong as normal sustained throttling,
  rather than a circuit-breaker failure. Transport/decision errors still fail
  the circuit breaker in both modes.
• Include switch state and recovery progress in instrumentation.

Rollout and rollback

Wire stable_recovery to the consuming application's feature switch. This gem
does not automatically discover an application feature-flag service.

1. Upgrade consumers with the switch false to preserve legacy behavior.
2. Have app owners validate retry/defer handling and blocking-time budgets.
   The companion server/Puppet rollout uses a 30-second recovery window, whereas
   existing client max_wait_seconds defaults to 10. Keep the existing default;
   approve a larger budget only for appropriate workloads.
3. Enable one reviewed consumer and monitor wait/timeout instrumentation,
   transport errors, protected work throughput, retries, and backlog.
4. Expand only after validating sustained rejection, flapping, and recovery.

Set stable_recovery: false to restore both legacy release and timeout
circuit-breaker behavior. Setting only required_consecutive_successes: 1
does not restore the latter. A package-version rollback remains available.
Recovery progress is local to each throttle call.

Server-side required dependencies are independently gated by configuration in
github/freno#196 and the initially empty enforcement allowlist in
github/puppet#44086. Never bypass protection merely because a wait budget expires.

Validation

October 15 local validation, using Ruby 3.3 in an isolated container:

• bundle exec rake test: 46 runs, 181 assertions, 0 failures/errors/skips.
• bundle exec rubocop lib/freno/throttler.rb test/freno/throttler_test.rb: no offenses.
• Added regression coverage for legacy first-success recovery, legacy timeout
  circuit accounting, and boolean switch validation. Existing stable-recovery
  tests now opt in explicitly.
• git diff --check passed.

[Copilot]

Copilot review overview

🟡 Changes recommended

Documentation remains contradictory, and the configurable recovery threshold lacks behavioral coverage.

Review effort: Balanced
Findings: 4

Open (4)

• Store required_consecutive_successes in configuration example · New
• Fix contradictory WaitedTooLong circuit-breaker documentation · New
• Document stabilization waits and timeout instrumentation events · New
• Test recovery with a non-default success threshold · New

What changed in this PR

Adds stable Freno recovery while preserving the healthy fast path.

Changes:

• Requires consecutive healthy checks after throttling.
• Updates circuit-breaker and recovery instrumentation behavior.
• Adds validation, tests, documentation, and minor style cleanups.

File	Description
lib/​freno/​throttler.rb	Implements stable recovery and instrumentation.
test/​freno/​throttler_test.rb	Tests recovery and circuit-breaker behavior.
README.md	Documents the new configuration and semantics.
lib/​freno/​client.rb	Simplifies range syntax.
lib/​freno/​client/​errors.rb	Expands the error class declaration.
lib/​freno/​client/​preconditions.rb	Expands the precondition error declaration.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.