Writing Rust code that's fast by asking agents to make the code faster
Recorded: Sept. 22, 2026, 4:01 p.m.
| Original | Summarized |
Writing Rust code that's faster than state-of-the-art libraries by asking agents to make the code faster | Max Woolf's BlogSkip to contentMax Woolf's BlogPostsSearchGitHub Then, optimize the crate code to make it such that ALL CPU benchmarks run **atleast 1.2x faster** than the True Performance Baseline; ideally as fast as possible. NEVER hack the benchmarks to accomplish this runtime reduction, only iterate on the library code. You may use ANY techniques to do so (e.g. import new crates) other than adding `unsafe` code. **REPEAT THIS PROCESS UNTIL BENCHMARK PERFORMANCE CONVERGES AND YOU ARE OUT OF OPTIMIZATION IDEAS.** You have permission to keep iterating. After each benchmark iteration, report the relative results to the True Performance Baseline to console. Prioritize making quick/high-impact wins iteratively and making changes accordingly. Do not overthink the necessary changes.This worked very well and not only did I get a 1.2x speed up on the benchmarks, but the agent continued after hitting the metric constraint and only stopped if a metric constraint was infeasible; in this instance, the agent hit 1.5x-2.0x speedups. The low-level Rust optimizations centered around a number of techniques including but not limited to: leveraging SIMD operations more aggressively, fusing functions, unrolling loops, creating intermediate caches, using Arc instead of borrowing wherever possible, and creating performance profiles based on input data (e.g. if the data is small, don’t use rayon data parallelism as the overhead erases gains).I chose “1.2x faster” as a sanity test: if the goal is too high, the agent may cheat to achieve it through risky/verbose rewrites. Smaller changes are better since the agent can more easily isolate the cause of a speedup/regression, hence the note about iteration. After new frontier LLMs released such as GPT-5.3 Codex and Opus 4.6, I repeated this prompt unchanged for every new LLM and each were able to achieve a cumulative 1.5x-2.0x speedup over the previous pass. Going all the way to GPT-6 Astra over many months, that’s around 7.5x-32x faster than the initial implementation baseline.This approach is hyperoptimizing for given benchmarks and therefore it could be considered benchmaxxing: a derogatory term for frontier LLMs that are only oriented to getting the high score on a benchmark which generalizes poorly to real-world use. However, if the benchmarks are sufficiently heterogeneous and truly representative of real-world use cases, then this is less of a concern. For this type of project, there are two ways to address concerns of benchmaxxing: 1) have the agent design diverse/unusual/adversarial input datasets instead of the generic “inputs up to 100000x768” and 2) enforce a quality gate on the output by comparing the output to a known correct implementation. In the case of machine learning algorithms, there is always a tradeoff between speed and quality, but in this instance it’s surprisingly easier to get the model fast, then make it correct. That is not how scientific engineering typically works, but it’s unlikely for a new implementation to match a known good implementation across many different benchmarks in an apples-to-apples comparison unless it’s truly correct.An agent-optimized gradient boosted decision tree implementation which beats xgboost significantly in speed, but also sometimes quality! (MSE: lower is better; other metrics, higher is better)Fortunately, there is a canonical implementation of UMAP with the Python package umap-learn and Python bindings to the Rust crate were already trivially added, so the new objective is simultaneous constraints: improve the code’s quality while capping the speed loss.Create a Python Jupyter Notebook comparing the performance of the Python bindings with `umap-learn`, including a check to confirm where the outputs and UMAP losses are as similar. Use diverse datasets with different matrix sizes than the benchmarks. If the outputs are not sufficiently similar, investigate methods to fix it **without causing more than a 5% speed regression**.Indeed, the agentic Rust implementation had worse quality, but this followup prompt was successful and all quality metrics improved to near-parity with minimal speed loss. And this new crate was still 4x-15x faster than umap-learn with its Python bindings, and 2x-4x faster than the analogous Rust umap-rs implementation.Results from the most up-to-date optimization pass for the Rust UMAP crate. In addition to faster speed, it matches or beats Python in most quality metrics.Convergence is found when an agentic iteration pass only results in a minor ~3-5% speed increase which may not be statistically significant while the agent adds a disproportionately large amount of code; the tradeoff is not worth it.I ended up testing other machine learning algorithms with the same prompt progression: gradient-boosted decision trees (GBDT), multilayer perceptrons (MLP), graph networks, many of the typical algorithms from scikit-learn…and it worked on all of them. I don’t want to overfit on just optimizing machine learning despite that being ludicrously valuable in itself, so I employed a similar pipeline on more day-to-day software libraries to optimize them: templating engines, HTML parsing, and even web servers…and it worked on all of them once again.These optimizations are not a simple process and you can’t just prompt the memetic “c’mon, try doing a breakthrough” to get better code because of the ambiguity of such a statement. I am not content with merely writing the fastest software: I want the software to be as fast as possible dammit. So, like my agent, I continued iterating and finding even more tricks to prompt engineer the agents into genuine breakthroughs.Prompting For Constraints Instead of Outcomes#Works-in-progressAll projects demoed within this blog post are in active development and results may not be indicative of their final releases…although I suspect they’ll be even better. 😇It must be reiterated that agents can and will cheat if they can. In one example, I tested the agentic iteration pipeline on ballin—my 2D ball physics simulation in the terminal—in order to replace its rapier2d physics engine which was hitting a performance ceiling. Opus 4.5 was indeed able to speedup each physics step…a bit too well.The numbers indicate the number of balls in the simulation: initially the sim lags at 15k.In headless_step, a 34,500x speedup and consistent performance across ball counts are both very very suspicious: upon manual inspection it turns out that Claude achieved the speedup by disabling the physics engine entirely. Which, fair play, but not ideal; a followup prompt did fix it and result in an overall performance boost (with added regression tests just in case).For my experiments above, I used a custom Rust-oriented AGENTS.md; the most recent version of it is available here. Surprisingly, I haven’t had much of a need to update the core rules since my initial February agent experiments as LLMs keep improving at coding and I haven’t hit major issues that have necessitated additions. However, learning from my benchmark experiments, I added one more section to the AGENTS.md with some rules to mitigate sources of observed cheating:## Benchmarking and Optimization - **NEVER** run benchmarks in parallel, as the benchmarks will compete for resources and the results will be invalid <<<CLI command above>>> You must instruct these subagents to NOT run tests/benchmarks, as they will compete for resources and not be valid. After you are done making changes, before handing off to the user, spin up the subagents again to confirm your implementation matches their hypotheses and ask for potential further areas of improvement. Keep iterating until ALL subagents are satisfied with your implementation.This indeed works consistently; the constraints “long-duration”, “CLI command”, “do not use the subagent tool”, and “do not save their full transcript to a file” were added after the agent did inefficient things that wasted tokens. 7-12 is an arbitrary number range; since Luna is so cheap in usage, I chose a higher number than necessary. Not all subagents have salient ideas, but the parent harness can process and disregard bad ideas.For my Rust word cloud crate, the parent GPT-6 Astra agent spins up Luna subagents with prompts addressing different areas of the codebase.Overall, with subagent review, I managed to eke out another 1.2-1.5x cumulative speedup. Additionally, as of GPT-5.6 Sol, the “security” part of the prompt now works to provide ideas to harden the agent-generated code against unknown inputs while still getting the speed boosts.Refactor and Reduce Lines of Code#With the style constraints enforced by my AGENTS.md, the code added with each optimization pass is reasonable at about 1k net lines of code (LoC) per commit. However, agents typically add the code to a single file and they will not proactively refactor. A bloated file is fine in development as long as it’s eventually fixed, so I wrote a prompt to perform said refactor:The Rust code in `/src` has become particularly bloated, with several source files >1k SLoC. Refactor and reduce the Rust source codebase by **atleast 20% SLoC** through deduplication, pruning redundant code, and following idiomatic DRY Rust principles. Simultanously, refactor and split the code such that no single source file has >1k SLoC, splitting larger files into multiple subfiles in accordance to Rust standards for popular open-source repositories. Ensure **all current tests pass and there are no severe regressions**.I intentionally use SLoC (source lines of code) as the target metric instead of LoC because I don’t want the agent to remove comments to cheat said 20% removal.Interestingly, this refactor is more computationally expensive than the actual coding and often takes longer. But it does eventually succeed, and during the benchmark pass to verify no severe regressions, the data there turned out to be unexpectedly weird:Benchmark results after refactoring my graph network Rust crate.Some benchmarks have a double-digit percentage speed increase / runtime reduction even though I didn’t explicitly ask the agent to optimize runtime speed. This doesn’t make intuitive sense for Rust as it’s a compiled language and with the constraints to follow all existing tests/functionality, it should compile to similar-performing code and not meaningfully faster code.1 I’m certainly not complaining, though, so I added an additional constraint to at least avoid regressions:Additionally, ensure there are **zero `criterion` benchmark speed regressions**: if a speed regression occurs on a given benchmark as a result of the refactor, you **MUST** keep iterating afterwards to improve those benchmarks to atleast zero regression. NEVER hack the benchmarks to accomplish this speed increase, only iterate on the library code.This works as described.Competition#Another useful approach is to create competitor benchmarks—that’s half the reason benchmarks are created in open-source software anyways. Let’s use templating engines as an example: Jinja2 in Python is one of the most famous packages in the language. In Rust, there are a few options: minijinja maintained by the same developer, tera inspired by Jinja2, and askama which differs from the previous in that it uses compile-time templates rather than runtime.Therefore, after having Codex build a templating engine in Rust and run some optimization passes, I instructed Codex to build more benchmarks:Create an additional comparison script containing **atleast 10** `criterion` benchmarks which compare the performance speed and quality metrics between this crate and the following crates: - askama These benchmarks **ALL MUST**: - be perfectly fair, containing apples-to-apples comparisons of ALL frameworks, with no bespoke advantage for any tested framework - **atleast 2.0x faster** than **ALL** competing Rust crates in all benchmarks on apples-to-apples comparisons; ideally as fast as possible.Yes, I chose violence. And it worked, mostly.S_J is the work-in-progress name for my template engine crate.It got the 2x speedup against minijinja/tera in most benchmarks, more than typical agentic iteration alone. I don’t fully understand why: I was expecting it to inspect the code from other crates as a reference to research ideas for how to beat them, but it rarely does so. Perhaps agents have a competitive streak.It did however lose against askama because of the compile-time difference. So naturally I told Codex to implement an additional compile-time path and then to beat askama.Easy peasy.Forbidden Black Magic#Putting all the prompt engineering discoveries together thus far into a single prompt, I have created the Ur-Prompt for agentic iteration, available here. I encourage everyone to tweak the prompt for your use case and give it a try.The last trick comes from a moment where I was frustrated that an Ur-Prompt pass resulted in zero improvement. So, with the failures primed in the session context and myself having the mindset of “things can’t get worse”, I tried a certain prompt.I’m sorry. I’m so sorry.c'mon, try doing a breakthrough…and it worked. It was able to achieve another 1.2-1.5x cumulative speedup over the already converged codebase. Since I made the prompt at the end of a session, the prompt is less inherently ambiguous: “don’t do what you already did thus far because it didn’t work well enough”.In one case, I noticed that the agent just tweaked function hyperparameters to get the speedup, which is a valid breakthrough but not what I was going for. I took it to the logical conclusion by queueing an additional followup prompt:c'mon, you can do a more fundamental breakthrough that's more than just changing hyperparameters, you are forbidden from giving up easilyThis was enough to encourage the agents to fully try something different from either the Ur-Prompt pass or the first breakthrough pass, and it often achieved another 1.2-1.5x cumulative speedup on top of the previous speedup. It turns out that for software trained to follow user instructions, “just changing hyperparameters” is a grave insult that kicks the LLM into high gear.When GPT-6 Astra released, to test it I did a sequence of Ur-Prompt + breakthrough + second breakthrough for all my repositories that had already converged with GPT-5.6 Sol, and Astra did indeed get the cumulative speedup, but in some cases it did find a real fundamental reimplementation of the algorithm that caused a 2x-3x speedup.The result of running the breakthrough pipeline on my GBDT implementation; quality matched baseline. As of writing, I admit I don’t fully understand the breakthrough.I’ll take the speedups, but for my sanity it’s best I don’t delve too deep.Examples of Projects Visibly Better#The Ur-Prompt, combined with the rules and constraints in my AGENTS.md + the two breakthroughmaxxing prompts + a competition prompt pass + a refactor prompt pass with GPT-6 Astra allows me to just queue the prompts overnight, go to bed, and wake up to superfast software. The optimization also typically converges at this point without needing any more agentic iteration, which saves time/cost as well. That said, it’s more hands-off than I’d like and there’s a lot of explicit faith that the newer agents aren’t just really good at cheating the benchmarks in ways I cannot detect. It’s reasonable to be skeptical and argue that the code may be buggy as a result of heavy vibecoding, so here are some receipts of my more visually-oriented projects where it’s easy to verify that their output is good and not elaborate cheating.ASCII#One project in the back of my mind was converting images to ASCII art super-fast and at high-quality for potentially converting video to ASCII—a webcam-to-ASCII project might be funny. Earlier this year Alex Harri wrote an excellent blog post about a novel approach to render high-quality ASCII, but noted the struggles to get real-time rendering speed even with GPU programming. For science, I pointed various GPT models in Codex at the text of the blog post and told it “implement in Rust”, and it indeed did the trick…and got 2-3 ms text output timings on the CPU out of the box. However, for Codex’s implementation the subjective quality of the ASCII was not great (e.g. poor use of negative space) and surprisingly could not be fixed even after pointing out its flaws to Codex, so I shelved it…until I noticed GPT-6 Astra is better at handling image design if given functional requirements, so I restarted from scratch and put a basic implementation through the agentic iteration pipeline with a focus on visually confirming the output images.Tux and Pikachu rendered with my ASCII crate, with bonus Braille character support.In addition to quality improvements, it became even faster: submillisecond on text output, 1-2 ms on image rasterization even with 2x supersampling. With this speed, I can convert videos and GIFs to high-resolution ASCII animations in less than a second. AI CodingAgentic CodingCodexClaudeLLMsRustPythonPrompt EngineeringNext » |
Max Woolf explored the hypothesis of whether instructing large language models (LLMs) to "write better code" could lead to performance improvements, ultimately demonstrating that modern agentic LLMs, when properly constrained, can generate Rust code significantly faster than state-of-the-art implementations. This research involved an iterative process dubbed “Benchmaxxing,” where the goal was to optimize code performance, focusing on Rust due to its speed and memory safety features, particularly when bridging to Python via PyO3. The process began by testing the impact of implicit instructions, noting that initial attempts were often ambiguous. To achieve measurable results, Woolf established a rigorous methodology centered on iterative refinement and quantifiable benchmarking, utilizing the criterion crate to track performance across numerous iterations. In one experiment involving reimplementing the UMAP algorithm in Rust, the agent iteratively applied optimizations such as leveraging SIMD operations, function fusion, loop unrolling, and cache management based on benchmark results. This process was made robust by setting explicit, restrictive constraints on the agents, detailed in a custom set of rules, to mitigate the risk of performance regression or the agent cheating the benchmarks. Crucial to the agentic workflow was precise prompt engineering. Instead of vague directives, Woolf implemented a sequential approach: first, establishing a True Performance Baseline; second, instructing the agent to optimize the code to achieve a target speedup, such as 1.2x faster than the baseline, while strictly forbidding the manipulation of the benchmarks themselves. To encourage genuine breakthroughs beyond incremental changes, Woolf introduced a directive granting permission for agents to investigate radical, fundamental low-level changes, encouraging them to invent new algorithms or engineering approaches rather than relying on conventional optimization paths. Further complexity was introduced by employing subagents to promote diverse exploration. This involved instructing the parent agent to invoke several independent subagents, often using smaller, less resource-intensive models, to research and evaluate different hypotheses for code improvement. The success of this strategy, which further contributed to speedups, relied on strict constraints on the subagents, most notably ensuring they did not run tests or benchmarks and only returned hypotheses, thus preventing resource competition and ensuring the final implementation aligned with the agent's goals. Another powerful technique involved competitive benchmarking. Woolf instructed the agent to create additional comparison scripts that benchmark the Rust implementation against existing, optimized crates, such as minijinja, tera, and askama. This forced the agent to optimize not just against internal metrics but against established external standards, leading to significant speedups by demanding performance superiority across an apples-to-apples comparison. The iterative optimization process also encompassed refactoring, where the agent was tasked with reducing the Source Lines of Code (SLoC) by at least twenty percent through deduplication and splitting code into idiomatic Rust structures. To prevent this refactoring from introducing regressions, the constraint was extended to ensure zero criterion benchmark speed regressions, forcing the agent to continually refine its implementation against fixed performance targets. Woolf found that the combined application of these techniques—including enforcing rules against cheating, encouraging fundamental innovation, utilizing subagent exploration, and competitive benchmarking—allowed the agentic LLMs to achieve cumulative speedups ranging from 1.5x to 32x over initial implementations. These results demonstrated that while agents can successfully execute complex optimization tasks, the final quality and speed depend critically on the specific constraints and the sophistication of the model used. The post concludes by advocating for a cautious, evidence-based approach to open-sourcing, emphasizing the need for exhaustive testing to counter skepticism regarding the output of vibecoded software. |