Recode v1.6: Cutting Stack Traffic in an x86-64 JIT

Problem. Recode v1.5 could compile correct native integer functions, but many emitted instructions merely stored and reloaded virtual registers. Change. I replaced much of that traffic with bounded register homes, tightened instruction lowering, grouped batch sampling, and reused compiler preparation. Evidence. The final seven-round comparison reports code bytes, kernel timing, public-call timing and regressions separately.

After v1.5 batching, the remaining execution cost was easier to isolate. Batching had reduced the number of times callers entered the public runtime, but the compiled body was still using the stack for almost every virtual-register write. A few arithmetic operations could produce more memory instructions than useful arithmetic. On top of that, the C batch loop still entered native code for each record, and compilation repeated some validation and layout work.

For v1.6 I worked on the code it generated and the costs around that code. I kept the interpreter and public semantics intact. I did not add an LLVM-like optimization pipeline, change the IR, expose unchecked function pointers, or assume that shorter machine code would automatically improve whole-application performance. The implementation is covered by profiling PR #32 through release PR #36.

Separating the kernel from the runtime

The private profiling harness measures at least four different things: native kernel calls with preformed inputs, public scalar calls, public batches, and compilation stages. These are different measurement boundaries. A native arithmetic kernel can take single-digit nanoseconds while the public managed scalar API takes tens. The latter performs handle lookup, policy decisions, synchronization, lifetime protection, input handling, output handling and periodic sampling. Improving the first number alone is not enough to claim the second improved by the same proportion.

The v1.5 profiling baseline, at 128 records, measured 4.314 ns per arithmetic kernel call, 60.664 ns per public scalar call and 21.986 ns per public batched record. Its emitted arithmetic function had 81 text bytes, 16 statically disassembled instructions, four explicit stack loads and five stores. These are medians from the profiling harness, not the later final v1.5/v1.6 comparison. The baseline's seven per-process arithmetic kernel samples ranged from 4.270 to 7.882 ns, so differences near the noise floor deserve caution.

I used objdump on generated binary fixtures, counted static instructions and explicit stack accesses, and compared those counts with timed execution. A static disassembly count is not a retired-instruction count. The loop body may execute many times, and a ret implicitly accesses the stack even when a diagnostic lists zero explicit stack loads/stores. This machine did not have perf available for hardware-counter evidence.

These costs must not be added together or swapped across harnesses. The full methodology and raw per-process samples are in performance-1.6.md.

Fixed physical homes instead of unconditional spills

The old integer emitter gave virtual registers stack homes. A limited RAX-forwarding optimization removed some reloads, but each write still went to the stack. v1.6 introduced a small register allocator in src/integer_codegen.c. It counts retained uses and assigns the highest-weighted virtual registers to an available bounded pool of caller-saved physical registers. An assigned virtual register keeps the same physical home across the whole function. Registers that cannot get a physical home retain deterministic stack slots.

The implementation's pool, in x86 register-number notation, starts with RDX, R8, R9, R10, R11, RDI and RSI. That is not a promise that all seven are always available. A bounded-memory entry reserves the registers required for data, length, result and fault offset; retained ARGUMENT operations also prevent assigning over the input argument register before it has been read. RAX and RCX remain scratch registers. The emitted function neither makes calls nor clobbers callee-saved registers, preserving the Linux System V ABI requirements.

const int pool[] = {2, 8, 9, 10, 11, 7, 6};
for (size_t slot = 0; slot < sizeof(pool)/sizeof(*pool); ++slot) {
    int reg = pool[slot];
    if ((p->memory && reg != 11 && reg != 7 && reg != 6) ||
        (reg == 7 && arguments[0]) ||
        (reg == 6 && arguments[1])) continue;
    /* Select the highest weighted remaining virtual register. */
}

The conditional and register pool are taken from the actual v1.6 allocator; the loop body's selection is summarized rather than presented as a verbatim complete function. See the implementation for the exact weight scan and spill layout.

Using fixed homes solves a particular control-flow problem cheaply. If each basic block chose its own physical registers, every branch join and loop backedge would need reconciliation moves, spills or a global liveness analysis. Fixed register homes avoid edge shuffles and let backedges observe the same representation. The trade-off is weaker allocation in high-pressure or phase-changing programs. The allocator does not use graph coloring, SSA renaming, next-use distances or a full global liveness fixed point. It can still spill many registers, and its heuristic is not optimal.

The dead-write pass is also deliberately local. Walking backward within a block, it drops a pure write if that value cannot be read before being overwritten. At branch boundaries it conservatively treats registers as live. A memory load is never dropped merely because its result is unused: the load itself can fail a bounds check, and eliminating that failure would change observable behavior. The program remains immutable; the emitter records skips in a bounded preparation plan.

Fewer generated instructions, not different semantics

Assigning registers removes many load/store pairs. The emitter can then operate directly on physical registers and reuse a commutative destination if it aliases either input. Noncommutative operations protect right-hand aliases, and shifts use the required CL count register. Constants whose complete 64-bit value fits in an unsigned 32-bit value use a zero-extending 32-bit move; larger values use the full-width materialization. Comparison lowering uses explicit REX prefixes where eight-bit register encodings would otherwise select legacy high-byte registers.

None of these transformations removes required read checks, changes an IR branch target or shifts a reported fault to a different source instruction. The backend still emits ENDBR64 on entry, retains the mapping's writable-then-executable transition and uses checked code-offset relocation. A new regression matrix tests operand aliases, all shift counts, multiple live virtual registers, high-pressure backedges and native ABI sentinels.

The arithmetic fixture is a good small example because its final assembly is retained in the repository and independently reassembled using GNU as. This is the actual documented v1.6 sequence, not a hypothetical replacement for a compiler output:

endbr64
mov  %rdi, %rdx       # keep the first argument in a register
mov  %rsi, %r9        # keep the second argument in a register
mov  $2, %r8d         # constant, zero-extended to 64 bits
imul %r9, %r8        # multiply in a register
add  %r8, %rdx        # add without a stack reload
mov  %rdx, %rax       # ABI return register
ret

The instruction order and operands match the committed arithmetic.s; the trailing annotations above only describe their roles. The benchmark's final disassembly count changed from 81 to 27 bytes, 16 to eight static instructions, and four loads plus five stores to zero explicit stack accesses. GNU as produced the same 27 bytes as Recode's fixture, with SHA-256 e829d5318170b3790ac9979f9a87b87ba08b781c9ece4fb89bcd424943e60661. It remains one fixture: the bounded-record example still has explicit memory work, and the record fixture's static instruction count actually rose from 60 to 61 while its generated text fell from 275 to 248 bytes.

Text size and page allocation should not be conflated. A 27-byte function still consumes a page-rounded 4,096-byte executable mapping in the tested setup. Recode did not add a shared code arena in this release.

The fast path still calls native code per record

v1.5 already performed one handle lookup and one tier selection per batch. The inner C loop, however, still checked whether every record was a sampling position and did per-record modulo and saturated-counter bookkeeping even when no clock read was needed. In PR #34 I grouped successive unsampled native records between the existing sampled positions. A run reuses the validated arity, executes its items and advances the successful-prefix count at the run boundary. Sampled records continue through the original timing path.

That optimization is constrained by observable semantics. Records must run in order. A bounded-load fault must leave the failed result and later slots untouched. completed must report only successful items, while function statistics count attempted items. Tier policy must still select at request boundaries, not mid-batch; the native mapping remains pinned for the entire request. Tests exercise sample periods of 1, 2, 3, 7, 16, 17 and UINT64_MAX, including errors immediately before and after sampling boundaries.

One limitation matters enough to state without qualification: v1.6 still enters the generated native function once per record. It does not implement a fused native batch loop, SIMD execution or a public unchecked execution API. A fused body would require different register ownership, a loop ABI, and precise failure/sampling exits. That would be a separate implementation and validation effort, not a hidden part of the reported gain.

Preparing once, publishing once

Code generation also had redundant compilation work. The private interface separately validated and laid out a program during size queries and again when publishing its executable mapping. PR #35 added a prepared compilation plan: validate and calculate register allocation/layout once, then pass that plan to the mapping path. Standalone private emitter functions still validate their inputs; the managed runtime borrows its immutable owned program during the protected translation lifetime.

A straight-line program ending in one return no longer allocates temporary control-flow graph facts just to check initialization. A definition bitset is enough for that case. Programs with branches or early returns retain the full reachability and must-definition analysis. Publication checks capacity before mapping, emits into writable storage and calls mprotect to make the mapping executable. Existing rollback and failed-unmap ownership accounting remain active.

The separate compiler operation probe observed two temporary allocations before and zero after for a simple compilation, with one mmap and one mprotect in both versions. It did not establish a general RSS improvement. A private stage probe can also measure map + emit + publish + free, which is not interchangeable with the runtime's compilation_ns counter. That difference matters when comparing small microsecond-scale compilation intervals.

The final v1.5 versus v1.6 comparison

The final comparison uses immutable v1.5.0 baseline 9f49d55296614cadeaadd7338b60bfac642a53fb and v1.6 candidate implementation eeb68033b0cdaeeae53dce69e21ea59fb202ff29. The merged release is based on that tested implementation with subsequent release documentation. Both libraries were independently archived, rebuilt and consumed by an identical checked harness. The environment was an Intel N150 with GCC 16.2.1, -O3 -DNDEBUG, glibc 2.44 and Linux 7.3.0-4-cachyos. Optional adapters used LLVM 23.1.4 and MIR dc2c763.

Seven fresh alternating processes per library and rotating engine order produced the reported medians. CPU affinity was pinned to CPU 0 for latency; frequency, background activity and thermal state were not fixed. Batch measurements used prepared inputs and wrote all outputs; scalar timing additionally included input formation and checksums. The figures below come from the same final harness, not the earlier v1.5 installed-record experiment.

At 128 records all six FIXED batch fixtures improved, from about 11% for records to 32% for bitwise and loop8. The arithmetic comparison is a 23.3% reduction in the measured median, not a guarantee for other programs or machines. In the same collection, the arithmetic native kernel moved from 4.290 to 3.886 ns, while the public scalar API moved only from 59.943 to 58.927 ns. Loop8 kernel time fell from 18.598 to 10.172 ns. The disparity between kernel and public-call gains is exactly why the measurement boundaries matter.

LLVM and MIR still ran the warm, preformed-input arithmetic workload faster: 3.599 and 4.638 ns per record versus Recode v1.6's 14.599. The optimized C reference measured 4.317 ns. These adapters do not implement Recode's identical error/status interface, and they use valid views for the comparison. Recode has a different cost profile for compiling supported small programs, not a general warm-execution advantage.

There are real regressions. A one-item arithmetic batch moved from 73.249 to 75.754 ns per record. Conditional scalar calls moved from 65.069 to 65.542 ns. Branch and record compilation increased from 21.181 to 22.123 µs and 18.581 to 20.000 µs respectively. Sixty-four short-lived FIXED contexts moved from 762.510 to 786.917 µs. Four of six compilation fixtures improved, but the other two did not. Reducing native text is not synonymous with reducing cold setup time.

The concurrent results are also inconclusive. One seven-process collection showed regressions for two and four threads; a repeat reversed their ordering. The report retains both rather than using the favorable repeat as proof of scalability. The thread study includes thread startup, join and checking, with CPUs 0..3, so these figures are not isolated native-kernel speedups. Arithmetic bulk samples at 128 records spanned 17.606..20.901 ns before and 12.442..15.678 after, another reminder that seven process samples have variability.

Correctness and release boundaries

The v1.6 verification record documents GCC/Clang Release and ASan/UBSan configurations, source-compatible installed consumers, focused GCC TSan tests, native differential checks, ABI sentinels, guard-page failures and rollback cases. A 600-program randomized corpus includes 256-instruction programs with 8, 16 and 64 registers; alias and high-pressure loop cases exercise emitted native results against independent reference evaluations. It also documents a 100,000-run bounded fuzz smoke. Those are repository-recorded tests for the Recode release; I did not run them as part of authoring this portfolio page.

Compiler sanitizers do not instrument generated native instructions. Passing ABI, regression and memory-failure tests is evidence against known faults, not a proof of sandbox isolation, security or total race freedom. Public headers and the 31 exported function signatures remain unchanged from v1.5, but the v1.6 shared-library SONAME is 1.6. Existing consumers should rebuild against matching headers and library rather than assume cross-minor binary substitution.

What v1.6 still leaves expensive

The biggest cost for tiny managed scalar calls is still outside the optimized arithmetic body. Handle lookup, synchronization, sampling, per-call lifetime management and the public dispatch path remain. Public batching still loops in C and invokes the native body individually for each record. Higher register pressure still spills because the allocator has fixed, bounded homes rather than a global optimal assignment. Bounded read-only loads retain their checks and instruction offsets. Tiny functions still occupy page-rounded RX mappings, and a full shared executable arena would require additional lifetime and W^X work.

Recode remains a restricted integer execution library plus its separate narrow byte translator, not an optimizer for arbitrary native binaries. It does not provide hostile-code sandboxing or execution deadlines. The final performance report supports substantial code-size and batch improvements on the tested arithmetic, bitwise and loop fixtures, along with lower arithmetic compilation latency. It also shows slower warm native throughput than the measured LLVM/MIR adapters and cases where v1.6 regressed. I kept the compiler relatively small, and the remaining costs are visible rather than hidden behind the faster rows.