Repository navigation
perf: keep the value stack's height in a register, stacked on cheaper calls - #81
Open
matthargett wants to merge 10 commits into
Open
matthargett wants to merge 10 commits into
matthargett wants to merge 10 commits into
Conversation
Entering a function reserved and zero-filled all three value-stack lanes even when the function never touches the 64- or 128-bit lane, which is the common case. Skip a lane whose locals and operand-stack depth are both zero. A single-result return now moves the result down and truncates once, instead of popping, truncating and pushing it back.
Every call and return between two functions cloned the callee's `Shared<WasmFunction>` and dropped the previous one, two refcount updates each way, and looked the function up in the store. A module instance now keeps its own functions, which the store allocates contiguously. The interpreter holds the instance for the whole run, so the executor borrows the executing function and its module from it: a call or return within the instance switches a reference, and a direct call to one of the module's own functions skips the address table, the host check and the owner check. Execution that continues in another instance's function (a call through an import, table or reference, a return, or an exception unwinding into it) ends the run, and `InterpreterRuntime` resumes that frame with an executor for its instance. Fuel and time budgets carry over, so a run suspends at the same points as before.
The match loop spilled the executor's function and reloaded it, then the instruction slice's pointer and length, on every step. The executing function is borrowed from the module instance, not from the executor, so the loop can hold its instructions in a local and reload them only when a step switches functions.
The instance kept its own copy of the module's function list for the executor to borrow from, which cost an allocation and a reference count per function on every instantiation, and as many again on drop. Hold the module instead, which costs one reference count per instance.
This was referenced Oct 15, 2026
On arm64_32 (watchOS) the Rust ABI passes the 8-byte Instruction by reference, so every dispatch stores it and the next handler loads it back. The C convention passes it in a register, and `C-unwind` still lets a host function's panic unwind. Other targets keep the Rust ABI.
The height of a value-stack lane is its own field, over a Vec that holds every slot the lane has reached. Slots above the height hold stale values that are written before they are read again. The tail-call handlers can then pass the height between them in a register and write it back without unsafe code. Entering a function writes its locals, and its operand-stack reservation if that is at most 64 slots, so later entries at that height compare once. A larger reservation stays capacity, and a push writes each slot the first time it reaches it, so a function whose deep branch rarely runs keeps no pages for it; once the locals have slots, later entries compare and go on. With 64 stores of a module whose untaken branch declares a 30000-deep i64 stack, each store keeps next's footprint (35 KB on macOS, 16-19 KB on glibc) instead of 273 KB and 250 KB. v128.any_true and i8x16.all_true test the vector with integer arithmetic: LLVM scalarized their iterator forms once pushes gained that path. Only builds with debug assertions check indices against the height. CI also runs the tests optimized with them, keeping the release profile's wrapping arithmetic. Entering a function zeroes one or two locals with direct stores, and only the ones in between with memset.
The executor takes the store's value stack when it is created and gives it back when it is dropped, and around host calls, which read their arguments from the store and may call back into wasm. Every handler receives the executor as a `&mut`, so a stack access reached through it needs one load fewer (the store pointer), and the compiler knows that no store to a slot or to memory changes a stack's height, so it can keep the heights in registers within a handler.
…lers Each tail-call handler receives the height of the 32-bit value stack, which most instructions push to or pop from, as its last argument. It writes the height to the stack before its body and reads it back before dispatching, and with the stack in the executor the compiler can keep it in a register in between, as it does in most handlers. A handler no longer loads the height its predecessor stored, a store-to-load round trip through memory on every dispatch. The Unbudgeted handlers now take six integer arguments and the Bounded ones five, which stay in registers on arm64 (eight) and x86-64 System V (six). Windows x64 passes four, so there the height goes on the stack along with the Instruction, which already does without this change.
…ows x64 Windows' calling convention passes four integer arguments in registers, so every dispatch stored the Unbudgeted handlers' last two and the next handler loaded them back. On Windows x64 the handlers now use the System V convention, which passes six, so the 32-bit height stays in a register. The instruction still goes through the stack: under the System V rules this enum is passed in memory. A CI job runs the tests with tail calls on Windows.
Windows x64 passes four integer arguments in registers. The bounded handlers (fuel and time budgets) take four besides the 32-bit height, so passing the height put a fifth argument on the stack under either convention: the height under the Rust ABI, the instruction under System V, which passes this enum in memory. On Windows x64 the bounded handlers keep the Rust ABI and leave the height out: they read it from the stack before dispatching, as before the height was passed. The unbudgeted handlers keep the System V convention and the height.
matthargett
force-pushed
the
perf/stack-height-on-cheaper-calls
branch
from
October 17, 2026 04:59
ccb5032 to
81c5b50
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack, in merge order: #74 → #81, this PR, which is #79's six commits on top of #74. #79 is the same six commits on
nextalone. All three share one table layout for side-by-side comparison.#79's six commits on top of #74. #74 removes the call path's refcount updates and lookups; #79 removes the height's store-to-load round trip on every dispatch. Neither change depends on the other, and on most rows the two add up.
Results
CoreMark is −15.5% averaged over the five chips. The EH parser gets #74's gain and the suite fixtures #79's; GC binary trees and argon2id on the i7-8650U stay within ±2.5%.
Change in cycles per call against
nextwithnightly-tail-calls, median of five interleaved runs per build. i7-8650U: a Surface Book 2 (Kaby Lake, 2017) on Windows x64, pinned to one core, core cycles counted with VTune. A10X: Apple TV 4K (1st gen). A12, A13: iPhone XS Max and iPhone SE, performance cores. M4: performance cores. argon2id, compress and decompress arecargo bench-suite's fixtures, the EH parser and GC binary trees are WasmBench rows, and CoreMark runs with a clock that advances a fixed step per read, so every build runs the same iterations.With the default (match-loop) dispatch, which #79 doesn't target, #74 carries the gain: CoreMark −11.3% on the A13's and −3.4% on the A14's performance cores with both PRs, against −12.7% and −7.0% with #74 alone.