JIT Compiling Code in 5μs
malisper.me
August 23, 2026
12 min read
55/100
Summary
Pgrust’s JIT compiler reportedly compiles code in about 5 microseconds, allowing the PostgreSQL-compatible database project to JIT-compile every SQL query rather than only selected queries. The implementation directly generates ARM64 machine code instead of relying on LLVM or generated C/C++, options the developer says impose higher compilation overhead. A demonstration regular-expression engine supports literal strings, concatenation, and repetition, then uses copy-and-patch compilation to turn its parsed expression tree into executable code at runtime. Reusable ARM64 instruction templates, or stencils, perform character comparisons, branching, backtracking-stack operations, success checks, and failure handling. The runtime allocates executable memory with mmap, copies generated instructions into it, invalidates the instruction cache, and calls the memory as a Rust-compatible function on macOS ARM64. For the regex b(an)*, benchmark results put the JIT implementation near handwritten code and substantially ahead of an interpreter. Across inputs from 9 to 2,049 bytes, the JIT took 3.8 to 470 nanoseconds and delivered 11.7x to 19.7x speedups over the interpreter; handwritten code took 3.8 to 393 nanoseconds. The developer says AI assistance reduced the difficulty of producing and modifying assembly-level code, which historically limited adoption of custom JIT compilers.
Key Takeaways
- Pgrust’s developer says its JIT compiler compiles code in roughly 5 microseconds, enabling JIT compilation for every SQL query.
- The example JIT targets ARM64 directly through copy-and-patch stencils rather than using LLVM or generated C/C++ code.
- In benchmarks of b(an)*, the JIT ran 11.7x to 19.7x faster than the interpreter and performed similarly to handwritten matching code.
- The generated matcher uses a backtracking stack, NUL-terminated input, executable mmap memory, and instruction-cache invalidation on macOS ARM64.
What the discussion said
Despite the article’s focus on ultra-fast JIT compilation, the only sustained AI-relevant exchange asked whether coding models can meaningfully help with compiler work. One commenter treated JIT implementation as exactly the kind of problem where producing code is not a clerical exercise but the central difficulty: it demands command of machine-level execution, optimization tradeoffs, and security constraints. That framing implicitly challenges the popular claim that programming is rarely the bottleneck. The reply was much harsher about current public models. It argued that AI can be useful for familiar, bounded work such as interface components, but falls apart when the obstacle is deep domain complexity rather than routine software construction. From that perspective, generating plausible-looking compiler code is not the same as understanding correctness, architecture-specific behavior, or the performance and security consequences of code generation. No participant offered a concrete success story in which an available model had built or substantially advanced a real JIT compiler. The discussion therefore landed on a narrow but pointed conclusion: advanced systems work remains a poor test case for today’s public coding assistants, even if AI may still accelerate less demanding implementation tasks.
Where opinion split
The dispute is whether AI meaningfully lowers the barrier to difficult compiler engineering. One side sees JITs as proof that code itself can be the hard part and thus a valuable target for AI assistance; the opposing view is that public models handle routine coding but lack the domain understanding needed for genuinely complex compiler work.
Coverage & discussions
Hacker News · malisper.me
Community discussions
Community Sentiment
Positives
- JIT compilers highlight a class of work where generating implementation details could matter enormously, rather than merely polishing already-understood product code.
Concerns
- Current public coding models are seen as unreliable once compiler work hinges on deep domain knowledge, where plausible code can conceal severe correctness, performance, or security failures.
- AI appears most useful for bounded, familiar tasks, not the machine-level reasoning required to design and validate a production-quality JIT.
Related Articles
We tasked Opus 4.6 using agent teams to build a C Compiler
Feb 5, 2026
How AST-grep Rewrote Tree-sitter in Rust and Made It 30% Faster
Jul 26, 2026
I rewrote PostHog's SQL parser, 70x faster, while barely looking at the code
Jun 24, 2026
Eight years of wanting, three months of building with AI
Apr 5, 2026
Things I've Done with AI
Mar 9, 2026