ci: use pre-commit for documentation Lychee checks

[rwgk]

Highlights

Bring documentation Lychee checks into the same pre-commit workflow used during development, and catch authored-link failures during ordinary PR updates.

• One hook configuration and native version pin: development, PR CI, and nightly use the configured Python hooks and their shared lychee-bin==0.24.2 dependency. Remove the separate Lychee action, its binary pin, and the obsolete upstream-hook Dependabot exception.
• Earlier feedback: Linux and Windows pre-commit jobs check authored Markdown and reST on every PR push, without waiting for full CI. Fresh cached successes from the shared default-branch baseline (main) avoid repeating requests for established external links.
• Clearer failure diagnosis: the existing offline shell smoke checks run first. Structured native reports and bounded retries help distinguish hook setup problems from link and network failures.
• Retain generated-documentation coverage: the documentation build checks links and anchors in the HTML it just produced. Nightly still builds documentation and checks both authored and rendered content through those same hooks.

Tradeoffs

Authored links are checked more frequently because pre-commit runs on pushes that may not trigger full CI. Cache reuse should keep the usual overhead modest, but new URLs, expired or missing caches, and slow or failing sites still cost requests and time. Each pre-commit job can make up to three Lychee warm-up passes, followed by one strict full pre-commit run; successful external checks are reused between passes. PRs consume the shared baseline without publishing their own snapshots, so newly introduced URLs can be checked again on later pushes.

Successful external checks expire after three days rather than one, providing breathing room when nightly refreshes fail or are missed. Nightly still checks links afresh every day. When fresh results are unavailable, the tradeoff is that a cached success can conceal a newly broken external link for up to 72 hours.

Authored and rendered URL caches remain separate even when their URLs overlap. Authored checks leave fragments unchecked; rendered checks validate them. Lychee's cache does not record which validation occurred, so sharing an authored success can falsely accept a missing anchor. This was reproduced with pinned Lychee 0.24.2 against a local server. Keeping the caches separate prevents that false pass, at the cost of some duplicate requests.

How it works

Where	What runs	Final result
Linux/Windows pre-commit	Offline hook smoke, restore authored URL cache, warm only Lychee, then run the full pre-commit suite once	Ordinary strict pre-commit result, including unresolved HTTP 429s
Full-CI documentation build	Restore rendered URL cache; check generated HTML and fragments after building docs	Fail unresolved links, retaining the existing allowance for at most ten distinct URLs when all remaining failures are HTTP 429
Nightly cache refresh	Check authored links afresh; build docs and check rendered links afresh; publish successful results separately	Unresolved links produce warnings; build, setup, execution, and report failures remain errors

Repository URLs retain their existing targets in this PR. Checkout-aware repository links are a separate follow-up in #3143. Standalone CI-tool unit tests remain in nightly CI.

Implementation details and suggested review order

Retries and diagnostics

The CI driver invokes the actual configured hook and collects every native invocation's JSON report. Pre-commit may split long input lists, so each invocation writes its own report and exit status before the driver validates and combines them.

Additional hook attempts run only when every remaining failure is recognized as transient HTTP trouble: timeouts, 408, 429, or 5xx. CI allows three total attempts, with 60/120-second cooldowns. Permanent failures stop additional attempts. The full pre-commit suite runs only once, so formatter changes remain failures.

Missing, malformed, or inconsistent reports fail explicitly, as does a combined scan that checks no links. Per-attempt reports and a final summary are retained as CI artifacts for seven days.

Shared caches

Nightly supplies fresh authored and rendered baselines; Linux runs on the default branch also publish authored results. Only successful external checks are persisted, with their original timestamps and a three-day expiration. Reusing a result does not extend its lifetime. Local filesystem targets are checked afresh. A missing baseline starts a normal cold check.

URL results can be reused across operating systems. Installed pre-commit environments retain their separate OS/Python-specific caches. A shared helper derives compatible URL-cache names from the hook configuration, native wheel pin, checker configuration, and implementation files. Hashing whole files is conservative: even a comment or display-name change in a listed file can invalidate the cache.

Suggested review order

1. Hook definitions: authored and manually invoked rendered hooks share one native-wheel dependency.
2. Native wrapper, CI driver, and retry policy: rendered inputs, reports, and the configured-hook retry path.
3. Cache descriptor, then the pre-commit, nightly link, and documentation-build workflows.

The CI link-checking guide collects the maintenance and failure policies in one place. Focused tests cover report aggregation, retry decisions, cache naming, and rendered invocation.

Validation

All-files pre-commit passes locally on both stack layers after the final cache-policy and maintenance-comment update at d7d7bd8f3af2. The stack is based on main at 576ef68874f2.

Full CI was requested for the preceding revision, 9a247823b2f, before that final update.

Earlier validation of this implementation

Before this rebase and the final review refinements:

• Full CI passed all 95 jobs, along with Linux and Windows pre-commit, including Bash, PowerShell, and cmd smoke coverage.
• Manually triggered nightly cache production, warm Linux/Windows consumption, and authored/rendered consumption passed. Consumers restored the exact producer snapshots and did not publish replacements.
• Local implementation validation passed: 257 CI-tool tests and 36 subtests, full all-files pre-commit, permanent offline hook smoke, source documentation build, and actual authored/rendered hook checks.

Checklist

• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[rwgk]

/ok to test bdb3299

[rwgk]

/ok to test f87a871

[github-actions]

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-3142/
https://nvidia.github.io/cuda-python/pr-preview/pr-3142/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-3142/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-3142/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

[rwgk]

/ok to test 9a24782

[rwgk]

/ok to test

[coderabbitai]

📝 Summary

Summary by CodeRabbit

• CI Improvements
  • Documentation link checks now cover authored Markdown and reStructuredText in standard checks, and rendered HTML in documentation builds. Rendered checks also validate page fragments.
  • Link checks can retry transient failures, reuse cached results, and retain reports for review. Nightly checks start with fresh results and report unresolved links as warnings.
  • The documentation-build workflow no longer checks authored-document links; those checks run in the standard pre-commit workflow.

The change replaces the documentation-link composite action with shared Lychee pre-commit hooks, cache and retry tools, and updated CI workflow steps for authored and rendered documentation. It also adds tests and CI documentation for the new checks.

Changes: Documentation Link Checks

Layer / File(s)	Summary
Lychee hooks and cache keys .gitignore, .pre-commit-config.yaml, ci/tools/lychee_cache.py, ci/tools/tests/test_lychee_cache.py, lychee.toml	The authored hook adds a cache-age limit, and a manual rendered-HTML hook is added. The cache helper builds separate authored and rendered cache paths and keys from policy-file contents. Tests cover key stability, cache paths, and output validation.
Retry behavior ci/tools/retry_lychee.py, ci/tools/tests/test_retry_lychee.py	The retry helper uses an injected callback to run attempts and allows empty reports only when requested. Tests use callbacks to check retry behavior and report handling.
Hook execution and report collection ci/tools/run_lychee.py, ci/tools/run_lychee_precommit.py, ci/tools/tests/test_run_lychee.py, ci/tools/tests/test_run_lychee_precommit.py	The Lychee runner supports rendered HTML inputs and per-invocation reports. The pre-commit runner selects a hook, aggregates native reports, and delegates retries. Tests cover rendered inputs, report validation, and retry policies.
CI workflows and check policy .github/actions/check-doc-links/action.yml, .github/dependabot.yml, .github/workflows/*.yml, ci/README.md	The composite action is removed. Workflows invoke the new runner and add cache restore or publish steps and report uploads. The CI documentation describes the authored and rendered checks and cache behavior.

Suggested reviewers: mdboom, juenglin

Priority: ⬇️ Low

Change: Feature

Merge Risk: 🔵 Low · up to d7d7b

Documentation checks have the required nightly inputs, but the new test may fail on some developer machines. Resolve the expected path in the test; this is a bounded merge risk.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA/cuda-python/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 1993dd35-a388-4bbb-8736-2bd9cca57c52

📥 Commits

Reviewing files that changed from the base of the PR and between 576ef68 and d7d7bd8.

📒 Files selected for processing: 17

• .github/actions/check-doc-links/action.yml
• .github/dependabot.yml
• .github/workflows/build-docs.yml
• .github/workflows/lychee.yml
• .github/workflows/pre-commit.yml
• .gitignore
• .pre-commit-config.yaml
• ci/README.md
• ci/tools/lychee_cache.py
• ci/tools/retry_lychee.py
• ci/tools/run_lychee.py
• ci/tools/run_lychee_precommit.py
• ci/tools/tests/test_lychee_cache.py
• ci/tools/tests/test_retry_lychee.py
• ci/tools/tests/test_run_lychee.py
• ci/tools/tests/test_run_lychee_precommit.py
• lychee.toml

💤 Files with no reviewable changes: 2

• .github/dependabot.yml
• .github/actions/check-doc-links/action.yml

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

[coderabbitai]

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

suggestion: The expected cache path in this test is not resolved, so the test can fail on macOS and some Windows hosts.

run_lychee.main resolves LYCHEE_CACHE_DIR with Path(...).resolve() (run_lychee.py Line 41). This resolved path is used as cwd and as the parent of inputs.txt. The test compares both values against the unresolved tmp_path / "rendered cache". On macOS, tmp_path sits under the /var → /private/var symlink, so resolve() returns a different path. In that case the run.assert_called_once_with(...) check fails, and --files-from gets a different string than the test expects. Linux CI does not show the problem, but local runs on macOS fail. The fix is to resolve the expected path in the test.

--- "a/ci/tools/tests/test_run_lychee.py"
+++ "b/ci/tools/tests/test_run_lychee.py"
@@ -51,8 +51,8 @@
     (docs / "guide.html").write_text('<h1 id="section">Guide</h1>')
     (docs / "_static").mkdir()
     (docs / "_static/theme.html").write_text("ignored theme")
-    cache = tmp_path / "rendered cache"
+    cache = (tmp_path / "rendered cache").resolve()
     monkeypatch.setenv("LYCHEE_CACHE_DIR", str(cache))
     monkeypatch.setattr(run_lychee, "ROOT", tmp_path)
     monkeypatch.delenv("LYCHEE_REPORT_DIR", raising=False)
     with (

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	cache = tmp_path / "rendered cache"
	monkeypatch.setenv("LYCHEE_CACHE_DIR", str(cache))
	cache = (tmp_path / "rendered cache").resolve()
	monkeypatch.setenv("LYCHEE_CACHE_DIR", str(cache))