Skip to content

perf(admin): push file list filters, sort and pagination into SQL - #552

Open
jcbl1 wants to merge 1 commit into
vastsa:masterfrom
jcbl1:upstream/05-admin-sql
Open

jcbl1 wants to merge 1 commit into
vastsa:masterfrom
jcbl1:upstream/05-admin-sql

Conversation

@jcbl1

@jcbl1 jcbl1 commented Oct 24, 2026

Copy link
Copy Markdown

FileService.list_files no longer loads FileCodes.all() (including every text payload) and filters/sorts/paginates in Python. Status, type, health and keyword predicates are translated to SQL, ordering and offset/limit run in SQLite, and only the returned page (<=100 rows) is materialized.

build_file_health_summary now uses SQL counts instead of walking every row; healthy/attention are derived from the danger/warning/success partition.

Equivalence is covered by tests/test_admin_query_pushdown.py, which compares every filter/sort result against the previous per-item Python rules.

10k-row benchmark: list_files 1629ms/29.6MB -> 24.6ms/0.1MB; dashboard health summary 1483ms/29MB -> 13.8ms/<0.1MB.

FileService.list_files no longer loads FileCodes.all() (including every text
payload) and filters/sorts/paginates in Python. Status, type, health and
keyword predicates are translated to SQL, ordering and offset/limit run in
SQLite, and only the returned page (<=100 rows) is materialized.

build_file_health_summary now uses SQL counts instead of walking every row;
healthy/attention are derived from the danger/warning/success partition.

Equivalence is covered by tests/test_admin_query_pushdown.py, which compares
every filter/sort result against the previous per-item Python rules.

10k-row benchmark: list_files 1629ms/29.6MB -> 24.6ms/0.1MB;
dashboard health summary 1483ms/29MB -> 13.8ms/<0.1MB.
@vercel

vercel Bot commented Oct 24, 2026

Copy link
Copy Markdown

@jcbl1 is attempting to deploy a commit to the vastsa's projects Team on Vercel.

A member of the Team first needs to authorize it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant