feat(client): retry only failures another try can fix, with jittered backoff

[gonzalo-zenrows]

Why

When a request fails in a way that another try can fix, ZenRowsClient should wait and try again, not fail at once and not hit the API again on a fixed schedule. For example, a cold Extract request can briefly answer 503 with "retry in a minute, not charged". Today retries are off by default. When they are on, every client waits the same fixed time, so many clients retry together.

What changed

ZenRowsClient now has one retry policy, the same as the Go SDK, Node SDK and CLI PRs that ship with it:

• Retried: 429, 502, 503, 504, and a failure to connect (connection refused, DNS failure, connect timeout). The API does not charge any of these statuses.
• Not retried: any other status, including 500 and every 422 (both were retried before), and a read timeout. After a read timeout the server may still finish the request and charge it, so a retry could charge it twice.
• Wait between tries: the server's Retry-After, capped at retry_max_wait. Without it, a random time between 0.25 s and min(retry_max_wait, 2 ** attempt) s ("full jitter": attempt 0 waits up to 1 s, attempt 1 up to 2 s, and so on). An unreadable Retry-After falls back to the random wait.
• Total time limit: the client starts no retry whose wait would end more than retry_budget seconds after the call's first try. The first try's own duration counts.
• The extra 422 retry loop in _worker is removed, so there is one retry layer: a urllib3 Retry subclass (_JitteredRetry) mounted on the session. It overrides only hooks that urllib3 1.26 and 2.x share. pyproject.toml now declares urllib3>=1.26, because allowed_methods needs it.
• extract(mode="auto") and extract_async are unchanged: a 403 REQS007 or 402 AUTH010 still falls back once to Autoparse. A 503 is retried before the fallback check sees the final response.
• A read timeout still raises requests.exceptions.ReadTimeout, as before.

New defaults and how to opt out

Parameter	Before	Now
retries	0 (off)	2
retry_max_wait (new, keyword-only)	n/a	30.0 seconds: cap for one wait
retry_budget (new, keyword-only)	n/a	90.0 seconds: limit for the whole call

To keep the old behaviour of no retries, pass retries=0:

client = ZenRowsClient("YOUR-API-KEY", retries=0)

Callers who already set retries keep their retry count and get the new statuses and waits. This is a feat because the default changes. release-please will make it a minor release with a changelog entry.

Tests

tests/test_retries.py now runs the client against a local HTTP server, so each test exercises urllib3's real retry path. It covers:

• the wait bounds on a 503
• Retry-After honoured, and capped at 30 s and at retry_max_wait
• the time limit, on a fake clock, including a slow first try
• no retry on 400, 401, 404, 422 (both bodies), 500, and 413 with Retry-After
• no retry on a read timeout
• retry after connection refused
• POST retried
• the default of 2, and retries=0 turning retries off
• the Autoparse fallback, sync and async

Results:

• make check: passed.
• make test: 275 passed, on urllib3 2.8.0.
• The retry, client, fetch/extract and concurrency tests also pass on urllib3 1.26.20 with Python 3.10.

[gonzalo-zenrows]

Same retry policy across all clients (429/502/503/504 and pre-response connection errors; 2 retries by default; full jitter; Retry-After capped at 30 s; 90 s total budget; timeouts never retried):

• Python SDK: feat(client): retry only failures another try can fix, with jittered backoff #62
• Go SDK: feat(api): retry transient failures by default with full-jitter backoff zenrows-go-sdk#43
• Node SDK: feat(client): retry uncharged failures by default, with full-jitter backoff and a time budget zenrows-node-sdk#58
• CLI: feat(fetch): retry uncharged busy answers so a brief capacity wait does not fail the command cli#64
• MCP: feat(fetch): retry not-served scrape and extract calls so a busy API does not fail the tool call zenrows-mcp#80