Repository navigation
fix(llm): pick CUDA only when an NVIDIA GPU actually answers, add engine Repair - #388
Merged
Merged
Conversation
…ine Repair has_nvidia_gpu() only checked that nvidia-smi was on PATH. A leftover nvidia-smi.exe with no NVIDIA driver/GPU behind it (e.g. an AMD machine) got the CUDA llama.cpp build, and llama-server fell back to CPU. It now requires `nvidia-smi -L` to succeed and list a GPU; the result is cached since /llm/status calls it on every poll. Machines that already received the wrong build had no way back: the UI only offers "Install engine" while the engine is missing. Settings > Agent now has an Engine > Repair action (GET /llm/binary/install?repair=true) that re-detects the GPU, downloads and extracts the new build into a staging dir, then swaps it in place of bin/. The working engine is only touched once the new one is fully extracted, so a failed download keeps it intact. Fixes #386
- test_auto_orient_does_not_flip_a_mesh_over: trimesh's update_faces keeps the cut-away vertices, so the exported bowl still carried the whole sphere's vertices and the "center" was measured on the orphan top cap. Drop unreferenced vertices; auto_orient itself was correct. - test_mesh_ops_run_without_undeclared_networkx runs repair_mesh, which needs pymeshlab: skip it when pymeshlab is missing, like the other repair/inspection tests.
2 tasks done
lightningpixel
approved these changes
Oct 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #386.
Problem
has_nvidia_gpu()only checked thatnvidia-smiwas onPATH. On a machine wherenvidia-smi.exeis present but no NVIDIA driver/GPU answers (the reporter has an RX 9070 XT),_pick_assets()installed the Windows CUDA build of llama.cpp, so llama-server could not use the GPU and ran on CPU.Users who already got the wrong build were stuck: the Agent settings only show "Install engine" while the engine is missing.
Changes
api/services/llm_server.py):has_nvidia_gpu()now runsnvidia-smi -Land requires a zero exit code plus at least oneGPU N:line. Cached, since/llm/statuscalls it on every poll. No usable NVIDIA GPU → Vulkan build, as before.GET /llm/binary/install?repair=true, Settings › Agent › Engine › Repair): re-detects the GPU, downloads the build, extracts it intobin.new, stops the loaded llama-servers, then swapsbin→bin.old,bin.new→bin. The current engine is only touched once the new one is fully extracted, so a failed or cancelled download keeps a working install, and no stale backend DLL (ggml-cuda.dll) survives the swap.Testing
api/tests/test_llm_gpu_detection.py(11 tests): nvidia-smi missing / failing without driver / listing no GPU / crashing / working; Vulkan vs CUDA+cudart selection; repair replaces the old files, keeps them on failed download and on a locked folder, plain install untouched.MODLY_LLM_DIRseeded with a stale DLL: release b10075, CUDA 12.4 + cudart picked, stale DLL gone,llama-server --list-devicessees the GPU. The same release resolves tollama-b10075-bin-win-vulkan-x64.zipwhen no NVIDIA GPU is detected.binworks even while an exe inside it is running, so the swap cannot leave a half-deleted engine; the old copy stays inbin.oldand is removed on the next repair.npm test(pre-push hook) passes.tscadds no error, eslint clean.Also in this PR: two mesh-ops tests that failed on a clean
devSeparate commit, they blocked the pre-push hook:
test_auto_orient_does_not_flip_a_mesh_over: trimesh'supdate_faceskeeps the cut-away vertices, so the exported bowl still carried the whole sphere and the test measured the bowl's "center" on the orphan top cap. The fixture now drops unreferenced vertices;auto_orientitself was correct.test_mesh_ops_run_without_undeclared_networkxrunsrepair_mesh, which needs pymeshlab: it is now skipped when pymeshlab is missing, like the other repair/inspection tests (all 42 mesh-ops tests pass with pymeshlab installed).