Skip to content

refactor!: remove proxy tiers and Request.forefront - #2336

Merged
vdusek merged 2 commits into
v2from
remove-proxy-tiers
Oct 16, 2026
Merged

vdusek merged 2 commits into
v2from
remove-proxy-tiers

Conversation

@vdusek

@vdusek vdusek commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator

Removes tiered_proxy_urls from ProxyConfiguration, together with _ProxyTierTracker, ProxyInfo.proxy_tier, Request.last_proxy_tier and the proxy_tier argument of new_url() / new_proxy_info(). JS v4 dropped tiers as well (apify/crawlee#3599).

Also removes Request.forefront, since the tier tracker was the only code that set it. Following apify/crawlee#4157, a retried request and a request replaced by the error_handler go to the back of the queue. A request deferred by a RequestThrottledError is reclaimed at the front, where v1 put it at the back.

The proxy management guide replaces the "Tiered proxies" section with "Switching proxies on retries". That section shows a new_url_function that picks the proxy by request.retry_count. The upgrade guide gets a section for each removal. The example will need an update in #2298, which drops the request argument from new_url_function.

Closes: #2299
Closes: #2300

✍️ Drafted by Claude Code

@vdusek vdusek added the t-tooling Issues with this label are in the ownership of the tooling team. label Oct 16, 2026
@vdusek vdusek self-assigned this Oct 16, 2026
@codecov

codecov Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.66667% with 2 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (v2@08747f7). Learn more about missing BASE report.

Files with missing lines Patch % Lines
src/crawlee/proxy_configuration.py 81.81% 2 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff          @@
##             v2    #2336   +/-   ##
=====================================
  Coverage      ?   94.14%           
=====================================
  Files         ?      184           
  Lines         ?    13577           
  Branches      ?        0           
=====================================
  Hits          ?    12782           
  Misses        ?      795           
  Partials      ?        0           
Flag Coverage Δ
unit 94.14% <86.66%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek
vdusek force-pushed the remove-proxy-tiers branch from ca488ac to 0e51ff8 Compare October 16, 2026 07:48
@vdusek
vdusek requested review from Mantisus and Pijukatel October 16, 2026 07:48
@vdusek
vdusek marked this pull request as ready for review October 16, 2026 07:49
@vdusek
vdusek merged commit 758e546 into v2 Oct 16, 2026
35 checks passed
@vdusek
vdusek deleted the remove-proxy-tiers branch October 16, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove the internal forefront field from Request Remove tiered_proxy_urls from ProxyConfiguration

3 participants