Section Insights
Bun's Transition from Zigg to Rust
Why did Bun rewrite its codebase from Zigg to Rust?
Bun's rewrite was primarily motivated by the need for stability, as the previous Zigg implementation had issues with manual memory management and garbage collection conflicts. The transition was executed rapidly, utilizing AI to manage the complexity of the codebase.
- The rewrite took only 11 days, involving 652 commits and a million lines of code.
- AI played a crucial role in managing the porting process and handling multiple instances simultaneously.
- The transition aimed to enhance stability and performance, addressing issues with memory management.
Initial Reactions and Workflow Challenges
What were the initial reactions to Bun's code rewrite and how did the workflow evolve?
Initial reactions included skepticism and concerns about the code's functionality. Jared, the lead, had to adjust the AI workflow to prevent conflicts among agents and ensure efficient code commits.
- Jared faced challenges with AI agents conflicting during the rewrite process.
- The workflow involved multiple AI agents for writing, reviewing, and fixing code.
- Adjustments were necessary to streamline the process and avoid disk space issues.
Testing and Quality Assurance
How did Bun ensure the quality of the Rust port during testing?
Bun implemented a rigorous testing workflow, utilizing AI to loop through commands and fix errors. The comprehensive test suite included memory leak tests and stress tests, although it faced challenges with disk space.
- The testing process was extensive, involving random test files and multiple AI agents.
- Bun's test suite was designed to cover various scenarios, including memory management and performance under load.
- Despite challenges, the workflow successfully identified and fixed compiler errors.
Performance Improvements and Controversy
What performance improvements were observed after the rewrite, and what controversy arose?
Post-rewrite, Bun showed performance improvements, including faster startup times and reduced memory leaks. However, controversy emerged from Andrew Kelly's critique, suggesting the rewrite was not purely technical but rather a result of mismanagement.
- The Rust port resulted in significant performance gains and bug fixes.
- Andrew Kelly criticized the decision, claiming it reflected deeper issues within Bun's development process.
- The transition sparked discussions about the role of AI in software development and community dynamics.
Transcript
0:00 Bunre wrote its entire codebase from Zigg to Rust in just 11 days, and that version will soon be released into production. Now, that on its own is a crazy story, but then Andrew Kelly, the creator of Zigg, wrote a blog post, and let's just say he does not agree that this was a technical decision. There are accusations of hacks on top of hacks, Jared's management style, and a line about tasteless AI enthusiasts. Let's grab the popcorn.
0:26 Sabun version 1.3.14 is going to be the last release written in Zigg and version 1.4 will ship with the new Rust codebase which is a port that happened in only 11 days between May 3rd and May 14th and contains 652 commits with around a million lines of code added. That is one hell of a PR review. Now, obviously, they used AI to do this, and at that peak, they had 64 clawed instances running simultaneously across four git work trees with a peak of 58 commits in 1 minute, all on a pre-release version of Fable 5. All in all, they estimate that the total price of this rewrite, if they had used the API, would be $165,000.
1:01 And we recently passed 165,000 subscribers. So, if you haven't yet, you should subscribe, too. So, why did they actually rewrite Bun? I mean, Zigg is a very good language. Well, the main reason that Jared gives is stability. Bun's whole thing is about being fast and up until now they've been using Zigg with manual memory management, no automatic cleanup, constructors or destructors and no borrow checker. In Zigg, cleanup is expected to be written out explicitly at each call site with defer. The problem with that is that bun sits on top of JavaScript core which is garbage collected. So you end up with garbage collected JavaScript values constantly crossing into manually managed zig memory. This is just a snippet of a few of the stability related bugs that they fixed in the last version of bun. And essentially they had had enough. In Zigg, memory safety was enforced by style guides and code review. In Rust, the borrow checker would enforce it at compile time. And as Jared says, a large percentage of these bugs were used after free, double free, and forgot to free in an error path. And in safe rust, these are all compiler errors. And compiler errors are a better feedback loop than a style guide. That is the core reason for this port. So did he just open up claude code in the bun repo and say port this to Rust and make no mistakes? Well, obviously not. They first had two questions they needed to answer. First, do they pour it all at once? And second, how do they keep bun in Rust the same as bun in Zigg? For that first question, the answer was yes, they're going to do all of it at once, as an incremental rewrite adds temporary code that you hope gets deleted eventually, but sometimes doesn't, and it would be painful in the short medium term. For the second question, they did choose to do a faithful rewrite rather than an idiomatic rewrite. So, the Rust code deliberately preserves the Zigg architecture, and you can see this in a lot of the provided code snippets. The functions flow and read nearly identically. This is actually the same way that the TypeScript team handled the Go Rewrite, although that one I don't think they had access to Fable for. With those questions answered, then the next step was to get going with the port.
2:45 First, they wrote up a document called porting.mmd, which mapped out the zig patterns to their Rust equivalents. And this was actually one of the first signs that Bun was considering this move, and it got trending on Twitter and Hacker News when people noticed this on GitHub. Funny enough, Jared actually replied on Hacker News saying, "This whole thread is an overreaction. 302 comments about code that does not work. We have not committed to rewriting. There's a very high chance all of this code gets thrown out completely. It seems that did not happen. After the porting guide, they also had a file called lifetimes.tsv that specifies the intended lifetime of strct fields. And this was actually done using a prompt like this in Claude. With those files set up, they're ready to spin up their AI agents and begin with a trial of three files. And the workflow they settled on has one implement agent writing the new Rust file and two adversarial review agents checking over it. And these are actually separate clawed instances that only received the diff. And after this, a fixer agent would then apply that suggestions. I guess everything went pretty well in the three-file test, as Jared then asked Claude to loop the workflow on all, 1448 zig files. That didn't go completely smoothly though, as about 2 minutes in, one claude ran get stash before it committed and another ran git stash pop and then git reset. So each agent was fighting with each other. And if Jared had used separate work trees for all of these, he would actually run out of disk space because the bungate repository is so big and eventually the changes do need to be compiled and seen together.
4:03 To fix this, Jared simply asked Claude to not run git stash or any git command that didn't relate to committing a file along with no cargo commands or any command that was basically slow. After this slight modification, Claude resumed the workflow and everything was working well just pretty slowly. So Jared split it out into four workflow shards, each with our own work tree. So we had four work trees in total, each running 16 claudes committing and pushing the files. As I said earlier, this paralyzation and prep work meant that at its peak, Claude wrote about 1,300 lines of code per minute. and every line of code was reviewed by two separate adversarial reviewers and going through a round of fixes before committing.
4:37 Still though, absolutely none of this worked yet. So, the next stage was going to be fixing the compiler errors going crate by crate. Now, for a quick bit of context here, the old Zig codebase was effectively one big compilation unit, and Jared wanted the Rust version split into 100 crates so it would compile faster. The problem with this is that crates can't have cyclical dependencies, and Zigg never cared about that, so their code was full of them. Jared actually tried to solve this himself in a PR right before the rewrite started, but it just wasn't enough. So instead, he ran one workflow to classify where all the cyclical code should live and write it down and then another workflow to actually do the refactor. Fixing these cycles then revealed about 16,000 compiler errors. As Jared puts it, that is a massive number for one human, but not actually a crazy number for 64 Claude. So another workflow was kicked off. This time looping over each crate, running cargo check once at the start, grouping the errors by file and saving them, then having Claude fix all the errors within that crate, having the two adversary reviewers again to check the changes and one fixer to apply the suggestions to stop the Claude stepping on each other again. Cargo check was only run at the very beginning and no gear until the end. This had its own full start though, as Claude actually interpreted the instruction of let's get all these crates to compile as stub out the functions that have errors. I mean, that does technically compile. It also started writing suspiciously long comments explaining why its workarounds were fine. So Jared added a rule for the adversarial reviewers saying if you need a paragraph long comment to justify why the workaround is okay the code is wrong fix the code. Honestly that is a pretty good rule of thumb. After a while that workflow had fixed all of the compiler errors. So the next step was simply getting the bun version command to run.
6:09 And this had a few issues but I guess everything worked in the end. So the next goal was simply running a test on a single file. This was yet another clawed workflow. So they loop over every bun CLI subcomand. Save each failing stack trace to a file. One claude fixes it, two adversarial reviewers, and then one fixer for those reviewers. They seem to be really confident and having good success using this workflow shape with Fable. So I might actually have to try this out one day. Once they had the CLI commands working, it meant they could now run the full Bun test suite. And this workflow actually ran a 100 random test files at a time, charted across the four work trees by folder, fixing failures with the same review loop. This was actually a massive challenge as Bun's test suite is pretty comprehensive. They have memory leak tests, integration tests that run nextdev and check that hot reloading picks up changes a 100 times. They have stress tests that exhaust every TCP socket on the machine, test the right gigabytes to disk, and tests that spawn around 10,000 processes. Even with isolation, CPU, and memory limits, the machine still ran out of disc space and crashed several times. Regardless of that though, 2 days later, the first CI run was done and the failing test list was down from 972 test files to 23. A day and a half later after that, Linux went fully green across 60 shards, followed by Mac OS and then Windows. And on May 14th, build 54,22 went green on all six platforms. The rewrite had seemingly been a success.
7:24 Next, all Jared needed to do was manually check that the tests were actually running and it hadn't skipped any silently and also run a bunch of stuff locally and then press the merge button. The final bill for all of this fable usage was 5.9 billion uncashed input tokens, 690 million output tokens, and 72 billion cache token reads. As I said, that is around 165,000 API tokens. But Jared estimates doing this by hand would have taken three engineers, the full context of the codebase for about a year during which they wouldn't be able to fix bugs or ship features. So realistically, they just never would have done it. This does actually seem like it's the first large-scale example of breaking the never rewrite rule. And as Jared says, until very recently, programming language choice was a one-way decision. You pick your language on day one and you're stuck with that for the life of the project. Now though, if AI agents can do a faithful test validated port of half a million lines for $165,000 in under two weeks, that stops being true. And yes, that number is a lot of money for you or me, but for a company, it's a fraction of what a small team costs for the year that that same port would take. So that is how Bun was rewritten in Rust. But did they actually get any benefits to this rewrite? Well, the binary is about 20% smaller thanks to better codegen and linker optimizations. It's 2 to 5% faster across the HTTP server and build benchmarks, largely from the cross language link time optimization, and memory leaks are dramatically down. For example, repeatedly calling bund.build used to leak about 3 megabytes per call, but now it's constant. They also fixed 128 known bugs in this process. At the same time, they did ship 19 new regressions from the port, but those have since been fixed. Apparently about 4% of Bun's Ross code sits inside an unsafe block as well, which is around 13,000 unsafe keywords. But 78% of those blocks are a single line, usually just a pointer coming back from C++. This is actually one of the trade-offs that they went with when they started this port.
9:07 They started off with unsafe keywords while they ported over all of the functions line by line, but then they can work on removing these over time once they're only dealing with that Rust code base. Overall, this is going to be an ongoing piece of work, but it does seem to have worked. Claude Code actually runs the Rust port of bun and they saw startups being 10% faster on Linux and basically no one noticed that they switched over which is incredibly impressive as this is a super large scale in production test. But now it's time for some drama because a few days after this bum post, Andrew Kelly, the creator of Zigg published my thoughts on the bun rust rewrite. His core argument is that this was never really a technical decision. In telling it's a relationship breakdown, and he says that Bun's codebase had become hacks on top of hacks, that features were shipped recklessly without paying down bugs, and that the Zig team had actually started to see Bun as a net liability for the language. He says along with the discomfort of Bum being the publicly presumed poster child for the Zig programming language, it was actually being the prime example of how not to write Zig code. Ouch. He then also went after that leadership culture too, showing a tweet from when Bum was hiring, saying, "If work life balance means a lot of time spent not working, it's probably not a good fit." He claims that this reputation actively hurt their recruitment, and that people in the ZIK community were steering clear of the company. And I quote here, "Jared was a stinky manager, poor communication, unrealistic expectations, low empathy, no experience, just a total shitow."
10:24 Yeah, I get the sense that he does not like Jared. He then also disputes the fact that the Rust port was necessary, pointing out that the binary size wins and link time optimizations were implementation choices that were available in Zig the entire time. Bun just never did the work. He says that the bum blog frames it that you either have to choose a style guide or a programming language feature in order to avoid these bugs, misdirecting the reader away from the main way that bugs are eliminated by dedicating engineering resources to it. You're not giving Tiger Beetle nearly enough credit. Quite simply, they put in the time to find and eliminate the bugs. They make an effort to maintain a healthy relationship with ZSF and Bundid neither of those things.
10:58 He goes on to say that the argument for shipping a million lines of unreed code is that the test suite is good enough to catch everything. But then why are you saying that you have so many annoying bugs in Zigg? Wham to the test suite being sufficient to catch everything. It's not sufficient to catch bugs in the Zig code, but it is sufficient to catch bugs in a million lines of unreed slop. After this, and yes, he is still going.
11:17 He then accuses them of lying about the advantage that Rust had to the binary size, saying, "All that engineering work had nothing to do with the rewrite. I think this is precisely why it took so long for the blog post to come out. You were doing the engineering work that you should have done in the zig codebase since the beginning. We've been trying to warn you about your comp time of use for years. We even made this time report thing specifically for projects that need to audit their usage of comp time, inline usage, and compile times." He then claims that they purposely left out the compile time comparison in the blog, saying, "I'm clocking 16 seconds to build from scratch with clean cache, followed by 90 millconds for each subsequent edit with incremental compilation enabled or the corresponding measurements of bun post rewrite." And if you've used Russ before, you can probably guess it's going to be a lot longer than 16 seconds. All of that was just his response to the blog post and we're still not done yet. He then goes on to say that the association with Anthropic brought drive by slop contributions and tasteless AI enthusiasts into the Zig community and that he's relieved that the connection is severed and he had feared Zig's identity would become known as a programming language associated with AI.
12:13 He also says the blog post is expertly written. It's almost like the marketing department of a trillion dollar company has a lot of money writing on this, implying that Enthropic in some way want to use this as the poster child of Fable's capabilities. And I mean, yeah, they probably do. The blog post that you can actually find on his site now has been rewritten. And he does admit to this, saying that he updated the conclusion section after self-reflection and chatting with friends. But he didn't actually make this nicer. In fact, quite the opposite. The original ending was, "What did we learn here today?" where he claimed that I actually don't have any personal criticisms of Jared and signed off with a shrug emoji. But that has now been replaced with a section called Moving On where he admits, "I resent Jared for making Bun into an embarrassment for Zigg and I blame him partially for some of the slop we've been dealing with, and I stand by my criticism of his leadership." He also apologizes to Zigg users who saw the languages creator trashing an exus user and worried that they would be the next target asking for grace with the line, "I hope you can give me some grace considering a trillion dollar company fired the first shot." Honestly, I think both of these posts hold some element of truth. Kelly is right that nothing about the Ross rewrite was strictly necessary.
13:14 Bun could have invested in memory safe discipline tooling and LTO in Zigg, but Jared is also right that after years of use after free bugs, we could have been more disciplined is not a valid strategy. So, a compiler that makes that bug class impossible is going to be a style guide. As for the AI side, I'd say this was not magically AI rewrote bun. It was a good engineer designing a translation pipeline with porting documentation, lifetime specs, and adversarial reviewers, plus 1.3 million test assertions. If anything, I just think this whole thing shows us how important tests can be to an AI agent.
13:43 And in fact, it's a reason why we saw a wave of projects removing their tests from the public, as AI was very good at working until those tests pass. I mean, it's what Cloudflare did with Nex.js, JS for example. So that's the full story. Bun is now written in Rust and Zig lost one of its flagship projects, but its creator seems pretty happy about it. I'd love to know what you think. Does this only work if Anthropic is paying for your token bill? Do you trust Bun now?
14:04 Let me know your thoughts in the comments down below or there. Subscribe. As always, see you in the next one.
Summary
- Bun's rewrite involved 652 commits and approximately one million lines of code added in a rapid 11-day timeframe.
- The primary motivation for the switch to Rust was to enhance stability and memory safety, addressing issues like memory leaks and bugs prevalent in the Zigg codebase.
- AI was heavily utilized in the porting process, with multiple instances of Claude managing code writing and review, achieving a peak of 1,300 lines of code per minute.
- The new Rust version is about 20% smaller and 2-5% faster than the previous Zigg version, with a significant reduction in memory leaks and 128 bugs fixed.
- Controversy arose when Andrew Kelly, Zigg's creator, criticized the decision as a failure of leadership and management at Bun, arguing that the issues could have been resolved within Zigg.
- Kelly accused Bun of shipping features recklessly and claimed that the Rust port was unnecessary, suggesting that the engineering work needed should have been done in Zigg.
- Despite the drama, the successful transition showcases the potential for AI-assisted development in large-scale projects, emphasizing the importance of testing in the process.
Questions Answered
Why did Bun rewrite its codebase from Zigg to Rust?
Bun's rewrite was primarily motivated by the need for stability, as the previous Zigg implementation had issues with manual memory management and garbage collection conflicts. The transition was executed rapidly, utilizing AI to manage the complexity of the codebase.
What were the initial reactions to Bun's code rewrite and how did the workflow evolve?
Initial reactions included skepticism and concerns about the code's functionality. Jared, the lead, had to adjust the AI workflow to prevent conflicts among agents and ensure efficient code commits.
How did Bun ensure the quality of the Rust port during testing?
Bun implemented a rigorous testing workflow, utilizing AI to loop through commands and fix errors. The comprehensive test suite included memory leak tests and stress tests, although it faced challenges with disk space.
What performance improvements were observed after the rewrite, and what controversy arose?
Post-rewrite, Bun showed performance improvements, including faster startup times and reduced memory leaks. However, controversy emerged from Andrew Kelly's critique, suggesting the rewrite was not purely technical but rather a result of mismanagement.