Skip to content

Add targeted address, padding and local-width repairs - #136

Merged
RetroGameDeveloper merged 3 commits into
mainfrom
feat/targeted-address-lifetimes
Oct 28, 2026
Merged

RetroGameDeveloper merged 3 commits into
mainfrom
feat/targeted-address-lifetimes

Conversation

@RetroGameDeveloper

@RetroGameDeveloper RetroGameDeveloper commented Oct 28, 2026 •

Copy link
Copy Markdown

What changed

Add four bounded deterministic repair families to the shared draft-repair and try_local_repairs pipelines:

  • Stage a scalar store's byte address through a pointer temporary.
  • Route repeated pointer-to-integer casts through an inline helper on supported 32-bit processors.
  • Remove or halve unreferenced ordinary local padding arrays. When a measured MIPS diff consists solely of a consistent prologue/epilogue/saved-slot frame shift, try two bounded missing-padding sizes through the optional iterative phase or current-source local trials. The processor reader owns the instruction-specific checks.
  • Narrow a scalar local when the candidate already casts it to a byte/halfword type.

Existing budgets, revision checks, durable miss caching and ordinary exact judges remain in force. Imported projects and ROM mode share these candidates. Volatile, escaping, ambiguous and unsupported cases are skipped; no compiler settings change automatically.

Evidence

Worked only on a private copy of an N64 checkpoint, with no paid helper sessions:

  • The existing bounded repair pass tried 431 variants across 55 functions above 99%, with no new exact matches.
  • Replayed 107 previously unreviewed repair examples; existing families reproduced 14 historical successes. The new padding and local-width families reproduced two more historical successes.
  • The new address, pointer-cast and local-width families found three new exact matches, verified through try_local_repairs and the normal compiler/link judge.
  • A manual stack-frame reconstruction found a fourth exact match; the measured-frame extension now reproduces this match automatically through try_local_repairs too. All four new matches have automatic repair reproductions. Missing-frame growth is MIPS-only until other processors implement its reader; unhinted padding removal/shrinking remains shared.
  • Four newly verified functions total 1,568 bytes / 392 instructions. Game sources, ROM, database and transcripts remain outside this repository.

Validation

  • 85 focused tests passed, including both ROM/imported judge entry points and bounded safety cases.
  • Ruff and git diff --check passed.
  • Full local suite before the final measured-frame extension: 1,967 passed, 157 skipped.
  • After that extension and processor-reader integration: 100 focused regression tests passed, including iterative search and both mode-specific judge paths.
  • After merging the latest main: 107 focused repair/iterative tests plus 38 adjacent permuter tests passed. All four automatic matches were reproduced again through the normal app judge on the private copy.
  • The merge conflicts with the latest main are resolved; both repair pipelines are retained. Cross-platform CI is running for 7af304b.

@github-actions github-actions Bot added the enhancement New feature or request label Oct 28, 2026
@RetroGameDeveloper
RetroGameDeveloper merged commit 33fe158 into main Oct 28, 2026
8 checks passed
RetroGameDeveloper pushed a commit that referenced this pull request Oct 31, 2026
* Visual C++ 2017 (PC) compiler pack, run through wine on Linux and macOS

The msvc2017 pack: Visual C++ 2017's 14.16 toolset (cl 19.16.27027.1) and the
Windows 10 SDK 10.0.17763, pinned by sha256. Two compilers, selectable in
Settings and by try_compiler: MSVC/14.16 (32-bit) and MSVC/14.16-x64.
wibo can't run them, so on Linux and macOS they run through wine (wine.py),
which the user manually installs; the older compiler packs stay on wibo.
Games built with 2017 are still refused at import (subject of the next PR).

* The Windows 10 SDK is now a toolchain of its own; changed installs are
offered again

The msvc2017 pack kept the Windows 10 SDK's headers inside each of its
version folders, so the SDK was on disk twice (14.16 and 14.16-x64, about
130 MB each), and every future pack created that uses that SDK (2015, 2019)
would have downloaded and stored it again. The SDK is now its own
toolchain, winsdk-10.0.17763, which a pack names in `headers`: installed
first, once, and each version folder gets an includes.txt saying where it
is. The build searches it after the compiler's own Include, so Visual C++'s
sal.h and concurrencysal.h still win as before; compiled code and data are
identical to the merged layout's. The pack goes from 350 MB to 220 MB on
disk.

Nothing compared an install with toolchains.toml, so a changed entry left
existing installs as they were, silently (here: a 2017 pack without its SDK
folder). An install now records a fingerprint of its entry (parts, hashes,
where each goes, requires); when the entry changes, the install counts as
not installed, Compilers & tools says it changed and offers to install it
again, and the archives already downloaded are reused. Installs made
before fingerprints are judged by the archives their manifest records,
and are given the fingerprint when they still match, so updating
decompdecomp reinstalls nothing that didn't change (tested for every
toolchain in the registry).

Multiple tools needing the same toolchain would download simultaneously
causing the destination to be corrupt. If multiple threads need the same
toolchain, it is now installed one at a time.

* The Mac wine lookup test skips on Windows, which doesn't need wine

* search Homebrew, MacPorts, Gcenx paths for wine

* The wine lookup's comment says where each install puts wine

* Review fixes: the wine prefix is required, manifests are written atomically, each pack names itself

- compile.py refuses --wine without --wine-prefix (it would have used ~/.wine)
- installed() and install() write the manifest to a file beside it and move it over (os.replace)
- wine.problem() names the pack that needs wine, not always Visual C++ 2017
- wineboot runs under limits.capped(), as every heavy tool
- a 2012-2015 game's message and the agents' x86 guide say what the packs have now

* Wine is found when a 2017 file builds, and its server leaves by itself

Review of #136. configure.py wrote wine's path and prefix into build.ninja, so
a project configured before the pack or wine was installed couldn't build a
file switched to 14.16 (ninja configures again with the project's own
arguments, which had none), and tool_args gave --wine to projects whose
older configure.py refuses it. Now configure.py only marks a 14.x file for
wine; tools/compile.py finds wine and the app's prefix when the file builds,
in DECOMPNET_WINE and DECOMPNET_WINE_PREFIX, which the app's build steps get
from the console (Console.build_env, set in ForkProject.run). tool_args has
nothing about wine, and starts nothing.

- wineserver -p600: the server leaves ten minutes after its last compile,
  however the app ended, and at once when the app quits (wine.stop)
- a prefix that fails fails the install with wineboot's reason, not a
  later "install wine"
- DECOMPNET_WINE_PREFIX also names a scratch prefix: the PC tests use one
  and stop its server

* Rules and texts say where wine and Visual C++ 2017 come from

Review of #136. AGENTS.md and docs/CONVENTIONS.md give wine, a system tool
for the wine = true packs, as the one exception to pinned tools, with why.
CONTRIBUTING says where to take a compiler the decomp community keeps no copy
of (2015 and newer: decomp.me's newest x86 Visual C++ is 2005 SP1), and the
install confirmation names each toolchain's own source instead of saying
every one is kept by the community.

* No wine server is started that nothing will use

wine's idle countdown (wineserver -p600) only begins once a wine program has
run and ended (measured with wine 9.0: a -p20 server nobody used was still
up after 70 s; one a program used left 25 s after it ended). prepare() ended
by starting a server, so every install left one running for good. Only the
build starts it now (tools/compile.py, right before it runs cl.exe through
it), and its test checks that prepare leaves none.

* The idle time is compile.py's alone, and the PR's lines stay near 120 characters

wine.py kept a copy of compile.py's SERVER_IDLE_SECONDS; since the app no
longer starts the server itself, only the build uses it. compile.py keeps it,
saying why it can't share one with the app (it's copied into each project
and runs without decompdecomp). The lines this PR added over 120 characters
are wrapped, the Compilers & tools card's template too (the same HTML for
every card). Left as they are: a Markdown table row, and toolchains.toml's
URLs, inline keep tables and one-line descriptions, as its other entries.

* More formatting cleanup

* The 2017 pack's id is msvc2015, the version that is to join it

The packs are named after the first version they hold (msvc: 6.0-7.1,
msvc2005: 2005-2010), and Visual C++ 2015 Update 3 is to join this one beside
2017, both run through wine. An install is filed under its pack's id, and a
renamed id would leave every install behind, so it's chosen now, before the
pack is released. What users read still says 2017 (its name, its card), until
2015 is in it.

* VISUAL-CPP.md lists Visual C++ 2017, which this pack adds

The builds table gets 2017 15.9's 32-bit and 64-bit compilers (msvc2015), the
import's steps say they can be chosen per file though 2017 games aren't
started yet, a paragraph says the pack runs through wine where the older ones
run through wibo, and 2017 15.9 leaves the builds that would help next.

* Final changes, another formatting touch up.

---------

Co-authored-by: risner <8061434+risner@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant