TheDaily Front

Since early this year, Michael Malis (of pgrust) has been showing me interesting things he's done and saying

[W]e should be more ambitious1 with the software we build

There is, understandably, a lot of skepticism about projects like pgrust because there are so many low-quality AI-generated projects out there. If I didn't know Malis and didn't look into it, I would just assume that pgrust was another vibe slopped project that's never going to work. But I know Malis well enough to trust his work without looking into it, and if you look into it a bit, you'll see that in testing pgrust, he's testing thoroughly enough that he's even uncovering real bugs in Postgres, such as this numerical bug that can cause valid SP-GiST operations to fail, this format cache bug that can cause incorrect results from JSONPath, this pgcrypto bug that can cause later operations to fail, multiple memory safety bugs in the dictionary loader2, this money overflow bug, etc.

This isn't to say that pgrust is bug free3, but it does indicate that there's a substantial amount of testing happening and, even if pgrust itself is never successful, just making the attempt will have improved Postgres in a durable way. The main pitch for pgrust is actually performance, and that's where the really interesting work is happening; if pgrust fails, it's possible that some of the optimizations they've demonstrated will make it into Postgres, Clickhouse, etc., but that's obviously less direct than filing Postgres bugs which get fixed.

That being said, I find it understandable that, every time I see pgrust discussed on a news aggregator, most people dismiss it as vibeslop garbage because almost all of these rewrite projects are slop. If I didn't know Malis, I would make the same assumption, because who has the time to look into all of these projects? Because I know Malis, I didn't bother to look at it; I just trusted him (I only found the above because I wanted to check on pgrust before using it as an example for this post).

As a consumer of software, everywhere you look, there's slow software that the creator is claiming is fast because the creator is touting some slop benchmark, e.g., here's one of multiple examples I saw yesterday, and of course low-quality software that doesn't work is more common than ever.

But, as a builder of software, it doesn't matter that almost all of these projects are vibeslop, that other people are out there shoveling out huge piles of crap and calling it gold. You can build things like pgrust or other projects that would've been too large to really be feasible in the past and, if you do a reasonable job with testing and benchmarking, that stuff can actually be good-enough quality to use.

If I were writing it, I would say "we can be more ambitious with the software we build" and not "we should", but Malis has been casually chatting with me when he's said this, so maybe he would say the same in an actual post; don't read too much into should vs. can here. What's interesting to me is how capabilities have advanced such that we can do a lot more.

Just for example, after hearing that OpenAI was going to use Cerebras for their SOTA models and not just fast-but-low-quality models like 5.3-Codex-Spark, I looked at where time on my computer would go if I was using a SOTA model, but 10x to 100x faster; some things that were already sometimes a bottleneck that would only get worse with a faster model were:

1. Ripgrep
2. Compiling Rust
3. Syscall and filesystem overhead

At the level of a "works for me" project, these are all things that can seriously be improved. I don't have much spare time at the moment but, even so, I built this regex engine that can be tweaked to be faster than other general-purpose regex libraries on any particular workload I care about, this ripgrep fork that uses the native code compiler from the regex engine and is faster for my codex ripgrep workloads (of course a text index is the better solution, but I've already worked on text indexing and this was a "just for fun" project where I wasn't trying to do the optimal thing), and a Rust that sped up builds for my projects (huge improvement for cold builds, moderate for warm, limited by wanting to avoid forking the Rust compiler front-end). For syscalls and filesystem overhead, just switching development from Mac to Linux improved this a lot, and of course this is another thing that can be sped up a lot as well, especially when agents are hammering lots of small filesystem operations (it's not uncommon to see my machine waiting on filesystem ops because the standard OS filesystem interface wasn't designed with modern disk performance in mind). With modern disks, there's no reason to use the *nix filesystem API. In the past, writing a developer toolchain that uses direct I/O instead of normal filessytem operations would've been an absurd endevaor, but that's totally doable now.

For someone like me who doesn't know anything about regexes, string matching algorithms, compilers, interpreters, etc., this set of projects would've taken a year or more to get to "works for me" quality. Instead, this was a low single-digit number of hours of my time. If I wanted to make any of these things meet a "useful product or open source project" quality bar, that would be a lot more work0, but it's still pretty incredible how much this process is sped up.

My read of Malis's meaning when he says “we should be more ambitious with the software we build”, is that it doesn't even occur to people to do these things that are now easy. For weeks, every time I logged into Twitter, I saw someone complain about how Rust compilation is slow and is limiting development velocity (I think this wasn't actually true when I saw the complaints, in that this was before 5.6 Sol and 6 Astra were available on Cerebras, and if you measured where the time actually went, Rust compilation time wasn't actually that much of a bottleneck at the time). Sure, Rust compliation is slow. You can just write a Rust interpreter as well as speed up compilation directly. There are problems like this which I see people complaining about daily that are solvable without a lot of time or effort now. As far as I can tell, these problems are problems because people are implicitly carrying around a pre-agentic cost model. Ten years ago, the kind of person who implemented a new compiler because their compiler was slow was the kind of person who started programming a new 3d game engine when they wanted to make an indie game, i.e., the kind of person who never ships anything. Most productive programmers I know still have an aversion to this kind of thing, even though it is now actually reasonable to implement this kind of thing now4.

Another project I previously mentioned was this Azul AI which is superhuman at matched time controls. When I did that (with GPT-5.1/5.2), with zero AI knowledge, it took about 5 hours of my time to get something superhuman, and about 20 hours of my time to become what appeared to be the strongest AI at the time, including being stronger against humans than this AI that was used to cheat on Board Game Arena. Gabriel Cangussu recently built an AI that demolishes my AI, and AFAICT, it took him substantially less time than my AI took me. From talking to him about it, one key difference is that Fable 5.1 and Opus 5.5 have a much better understanding of board game AIs. When I used GPT-5.1/5.2, pretty much every suggestion they had was wrong and I had to do a lot of steering to get things to be remotely reasonable. If I had GPT read a codebase or a paper to implement relevant techniques from the paper, it would fail. If I read the paper myself and named a technique, it would say it did the technique, but it would generally do something that superficially looked like the technique but wasn't the actual technique until I issued some corrections. When the model is that far from being able to do the right thing, if you have no background in the area, it's fairly time consuming to check if the model did the right thing since you have to learn enough about what it's supposed to do to be able to check that it did what was claimed.

When I did that project, I said to friends that I didn't think the state models were in could last. It seemed impossible that they wouldn't get good enough to be able to read a paper and implement the techniques soon, and it seemed like they should also be able to pick out codebases and papers to crib from as well. From my experience messing around a bit with Fable 5 and Cangussu's experience with Fable 5.1 and Opus 5.5, that seems to have happened, which is a huge time saver for someone like me who doesn't know anything about board game AIs (I know a little bit now, but still less than any real AI researcher)5.

For any particular domain, like board game AIs, you can say, "who cares? Board game AIs don't matter", but I've found this increase in capability to be true in quite a few different domains. Just for example, I did this toy eval of verification and test techniques with GPT-5.6 Sol implementing zstd on medium and xhigh. I haven't had time to write it up, but when I published that post, I also re-did the evals with GPT-6 Astra. 6 Astra was so good that I had to find different evals because models had advanced too much. For zstd, 6 Astra succeeds at passing all of the registered tests more than 99% of the time across all conditions, even the ones that significantly impair performance, like using TDD (and, BTW, it does it more cheaply as well, taking about $1 to implement zstd). I ended up using the IMAP RFC, which 5.6 Sol couldn't do effectively, and Pandoc, which is a pretty major thing to implement as it needs to be able to read and write many different formats6.

BTW, when people have said they used the new model and everything just works, software is solved, etc., I haven't found this to be true at all. It's true that there are a lot more things the newer model can one-shot, where things basically just work, but this just means that I end up working on more ambitious things where models still struggle7. I'm more productive and can work faster, but I find that, in terms of what I'm doing, I get farther from "it just works" with each more capable model because the gap between what the model can verify effectively without significant handholding and what the model can implement has been increasing8. All of that is a long-winded and less eloquent way of saying: we should be more ambitious with the software we build.

Thanks to Laurence Tratt, Yossi Kreinin, Alex Kolovsky, and Michael Malis for comments/corrections/discussion.

I've had this note on my recent posts, which also applies to this one. In order to get more posts out, I'm trying to crank posts out in half an hour and not review them much. This definitely means less well-supported posts. If I were writing this in my standard style, I would've had comprehensive benchmarks for all of my projects that I mentioned. Instead, I did a quick comparison that satisfied my curiosity and am writing this up without rigorous benchmarking, and there are maybe twenty or thirty things in this post that I'd like to do but haven't done.

I haven't even written up this Rust interpreter. If, five years ago, I'd cobbled together a Rust interpreter that got a 50x speedup on cold compiles and 2x on warm/cached builds (with much more being possible if edits to the Rust compiler were in bounds), that woudl be a super interestin thing that I'd want to write up, but even operating in this mode where I barely edit posts and just blast them out, this isn't interesting enough to write up because it's so easy to do things that would've been considered very interesting not all that long ago.

Appendix: we should be less ambitious with the software we build

Here's a comment from Yossi Kreinin, quoted (with permission, of course):

If you publish this I will publish a response explaining that for most people they should be less ambitious with the software they build with AI or they'll experience abject failure. I think a lot of the problems don't lend themselves that well to "agentic loops" since you don't have clear pass/fail criteria, and even when you do most people have trouble reasoning about giant black boxes which they can impose constraints on but can't understand the innards of [I include myself in this set.] This is a little bit like microservices I guess, where some people both need it and are very good at using it but likely for most people, they need it less and can't use it well at all. I sort of envy the ones who should and actually can be this ambitious about the software they build of course; not saying you shouldn't be more ambitious at all but moderating said ambition is a big part of handling the current situation for most I think

I have a similar thought about "there's no reason for software to be slow" BTW; in many cases, if you use AI to speed up some flow, you will add a bunch of messy code that will be overfit for the use case you tested on and will neither generalize, perf-wise, for all the use cases you should care about, nor will be maintainable upon future changes. If you know what you're doing and you understand how the system should be decomposed into parts and when it makes sense to go all-in on optimizing some part or a combination of them working together and how to properly benchmark it, etc., then things are great, but if for instance you have a manager who thinks there's no reason for software to be slow and he pushes people to use AI to make random parts of the software fast... Run for cover

My feeling is that, overall, the situation here is a bit analogous to the discussion here about why I don't like when people talk about how AI gives a 100x or 1000x productivity increase. There are some things that are sped up by a very large factor, easily over 1000000x. And some things don't currently get sped up at all. If you do exactly what you would've done five years ago, but with AI, the speedup is fairly poor because of Amdahl's law. But, in the general case, you can choose what to speed up and if you're looking for things that are productive to do, this often seems to involve things that were previously too expensive to do.

And, I think that you can often set up systems that have some kind of self-checking property even when most people wouldn't. This discussion of the implementation of Guards of Atlantis 2 is an example of that. It's a poor choice for an AI project because the official rules are a mess of ambiguous and contradictory English text (it's probably the game I've seen humans misplay at the highest rate, and the rules are confusing enough that the designer has often changed their mind about what their own rules mean despite continually maintaining that the rules are perfectly clear and one can play the game by simply reading the rules as written). If you try to produce a working set of rules in a naive way, AI won't really help with this (although it can speed up implementation once you have a clean spec), but for that, I was able to set up some kind of janky set of consistency checks between all of the inconsistent explanations of the game rules, which got fairly close to a correct set of rules (approximately as close as what someone else did with much more human effort and less of a systematic approach), which was enough to compare it to someone else's implementation and get a working set of rules. This setup did cost a lot more in tokens than a more manual approach. But it was "only" a month or two of the $200/mo OpenAI subscription with GPT-5.1/5.2, which is a lower cost than the cost in time of me sitting down and writing a clean spec for the game, i.e., a copy of the rules that is actually consistent and correct.

I feel like a lot of problems are in something like this class of problem, where it's possible to set up some kind of at least partially self-checking setup that a human can spend a small amount of time on. This also feels analogous to the situation about software testing at large, where (before coding agents) I found fuzzing and property-based testing to be highly effective. For most domains, people told me some kind of reason this kind of testing wouldn't work, but when I've tried these techniques with all of the domains people told me these techniques don't work for, they work fine. They're somewhat more work to apply, but they're still applicable and, AFAICT, more efficient and effective than writing traditional unit tests.

Part of the gist of Yossi's comment is that people aren't doing this kind of thing. Considering how many more bugs I see now than I used to, e.g., the last time I opened up the web chat for a big AI lab, I ran into a double-digit number of bugs within one minute, it's hard to say that Yossi is wrong. But, to that point, I still have my original opinion. Just because other people are pumping out vast quantities of barely working slop doesn't mean that you have to do the same thing.

Appendix: a few "random" examples

The examples in the body of the post I gave for things I've done are tooling things I never would've done. There are a lot more things like that as well since it's so easy to build "works for me" quality tooling that's intended solely for me. I do think it's possible to go further than is productive, e.g., Steve Yegge wrote a lot about Gas Town and how productive it was, but when he was building his next agentic framework, he noted that he never used Gas Town to successfully build anything other than Gas Town, people were into Ralph Loops for a while even though, when I tried comparing them, they seemed worse than just doing the naive thing I was doing at the time, normal loops with context where you sometimes clear context as appropriate instead of doing it by default at the completely arbitrary point of when a single turn is completed, etc., so I've tried to stick to things where the impact is pretty direct. Besides the obvious reason this has been effective, the dramatic change in model and harness capability over time means that what tooling it makes sense to build changes very quickly, so spending a lot of time working on any particular tool that will be suboptimal at best and quite possibly obsolete in three months doesn't really make sense.

And, for a lot of my personal tools, my assumption is that an AI lab will obsolete anything I make in short order. For a very silly example of this, I have a tool that was useful since the release of GPT-5.6 Sol that restarts codex when it finds a memory leak. Presumably, OpenAI will fix this at some point and this tool will become worthless. For a slightly less silly example, take the Rust interpreter I had some agent build. For token-budget reasons, a lot of things that ought to be done for a very high-performance interpreter haven't been done.

For example, the interpreter doesn't modify the actual Rust front-end compiler to make it easy to take updates; performance would be much better if I just made changes to the Rust front-end, but I don't have the token budget to maintain a compatible fork of the Rust compiler, but this shouldn't be a problem for OpenAI. OpenAI spent $10B+ on getting access to Cerebras while also preventing Anthropic from having access.

Funnily enough, for coding, on many tasks, if you actually use one of their SOTA models and compare it to what you get from Anthropic, the actual speedup isn't all that large because of time spent on things like compiling Rust. Given that they were willing to commit $10B+ on this, it stands to reason that they'll eventually do a better job at speeding up the relevant tools than anyone could feasibly do outside an AI lab, so there's not much point in spending too much effort on this (this could arguably be untrue if non-coding tasks are expected to completely dominate future OpenAI revenue in the short term, but even then, given what they pay for tokens vs. what I pay, it still seems like it would make sense for them to speed a lot of these things up given that I could do it outside of a lab; perhaps there's some kind of argument like, if you have access to their usage data, there are so many more important bottlenecks that this one doesn't make sense to prioritize, but it still seems like it should make sense for them at some point even if it doesn't fall into the top 100 bottlenecks for Cerebras-enabled coding agents).

Another kind of thing that one might not call ambitious, but that speaks to the kind of thing that's possible, are games that I've made. There are games I've thought about and wanted to make that I don't think are generally enjoyable enough that they make sense for a real audience, but that I've wanted to play. For example, ever since reading David Gerrold's Star Wolf books in the 90s, I've wanted to play a game that implements combat as described in the books, which is a sort of like submarine combat. At least among realistic war games, this genre is extremely niche because almost everybody finds naval combat boring, and submarine combat is a particularly boring subgenre of naval combat games, so this doesn't really make sense as a money-making game, but it's so easy to implement games that I had an agent implement the game so I could play it for an hour or two. I never would've spent 10-20 hours implementing a game so that I could play it for an hour or two, but I'd spend 5 minutes. I've also previously mentioned implementing a game with the same rules as Scout and Guards of Atlantis 2. I was never going to take the time to implement these games by hand just so I have a way to play with my friends on the occasions where we can't get together in-person, but I'm happy to have an LLM bash these things out. And, of course, for people who want to make games with a wider audience, there's a game renaissance going on right now due to coding agents. It's pretty amazing how many interesting and creative games are being made now that having programming skills and the time to put them to use is no longer a bottleneck for making games.

Data analysis is another thing where the productivity multiplier can be very high and I can do all sorts of things I never would've done. As I've mentioned in a number of my last posts, I have way too many of these projects to really be able to write them up, even when attempting to write very quickly and bash out posts in half an hour apiece.

When I've talked to people about their one-off tooling, there's a lot of fun stuff people have done. For example, here's something Peter Geoghegan mentioned a month or two back:

I had Fable implement a custom Neovim Lua function that formats code according to my very specific preferences. This is both something I don't want to maintain myself and inherently a Rube Goldberg device. Theoretically I could accomplish the same outcome in a principled way by patching clang-format, but my requirements are so exact that I think that approach would never be satisfactory.

We use a tool call pgindent, which is our fork of bsd_indent. You can approximate the same formatting rules using clang-format, but that isn't good enough. If I commit something that isn't perfectly formatted/indented according to pgindent, I get a CI failure that I'm expected to fix quickly -- something I'd really prefer to avoid ...

Practically speaking, getting slick, clang-format-style formatting that avoids nag emails about misindented code requires a total architectural mess (I also don't want to spend the time developing a principled approach, but that's really a separate issue). The function allows me to effortlessly format a given block of code from Neovim, similar to the clang-format LSP integration. I'm being unreasonably fussy here, of course, but I find that LLMs allow me to be this exacting about tooling.

The function performs multiple passes: an initial formatting pass using clang-format/clangd, and a separate formatting pass using pgindent. I need it to do both because clang-format handles some cases correctly that pgindent misses. This is brittle because I only want to format the code within a range of lines selected via block visual mode (I don't want to run clang-format on code not added by my patch). This conflicts with pgindent's design; it can only format whole C files.

Figuring out which relevant subset of the temp whole file buffer needs to be copied back into my working buffer is nontrivial. The initial line range I selected doesn't directly map to the affected C AST elements. So the function uses tree-sitter to intelligently delineate the exact subset of the temp indented C code that needs to be copied back. The rules need to be fuzzy in exactly the right way for cases where my selection straddles the same block of code. There are a bunch of edge cases.

Another complicating factor is that pgindent relies on a typedef file generated with objdump to identify types; without that file, the formatting looks wonky. The formatting function dynamically generates any extra typedefs from clangd, for that one file, so I don't have to worry about a stale typedef file (I might have created a new struct seconds before running the formatting tool, I don't want to run objdump on the whole postgres binary each time, that'd blow up latency).

This appears to work perfectly. The latency is totally acceptable. I have only a vague understanding of how all this works, though. I don't even know Lua. This will probably continue to work, even as further changes are made to our fork of bsd_indent that somewhat alter how code is formatted, which happens from time to time.

I'm skeptical of the saaspocalypse. But I do think that LLMs are writing a lot of useful software that would have been insanely uneconomical to write any other way. There's an interesting small subset of software that'd be impossibly tedious for any human to maintain, relative to the value that it creates.

I also caught with Malis while writing this and found out about all sorts of fun stuff he's been doing. It's enough that writing it all up in detail could be its own post, but one fun thing is that, the last time I taked to him about pgrust verification, he mentioned using Kani to verify some parts of the code. He's since switched to Lean and has been using Lean to verify things that are too complex to do in Kani. This would be another topic for another post, but I wonder why people have generally settled on Lean for LLM-driven proofs (as opposed to, say, ACL2). I wonder how Lean would look in an updated version of this theorem prover comparison, or if someone were to run a theorem prover eval that's analogous to this language eval.

Of course, you can think of plenty of examples out of AI labs, such as a mid-sized team Anthropic building an internal replacement for GitHub as well as some other tooling in five months

A team of seven people built both Scion and Antique (our code review tool, which we could easily write a whole other blog post about), not to mention half a dozen related systems, and fully migrated to them in five months.

I think these kinds of examples are interesting and they point to where the industry will be in the not-so-far future, but what you can do with unlimited tokens, while relying on other infrastructure and tooling built with unlimited tokens doesn't tell you much about what you can accomplish today without unlimited funds. Projects like pgrust as well as projects that people I know do in their spare time (of which I've probably mentioned something like 1%—there are just a lot of these, like David Turner creating an skipper for podcasts that's "works for me" quality that works better than anything he could find, the spare-time projects Dennis Snell mentioned here, etc.) are more representative of what you can do today.

1. In this post, we're going to use ambitious in a way that feels consistent to me, but that one could argue is overloaded. I'm using this to mean, doing more than you could've before, in a generic sense. One could say that there's a difference between building a project that's larger than was reasonably feasible before and building some kind of super-specialized tool that could've been done before but just wasn't worth the time to build. While these are very different things, I think they both fall out of the same thing (it being much easier to produce software), so I feel pretty comfortable with this arguably overloaded definition. [return]

2. Alex Kolovsky pointed out that these were not considered a bug. I see the argument from a security standpoint and I don't have enough context on Postgres to weigh on on this specifically, but I'll say that, for projects I've worked on, I've liked eliminating this kind of issue because it's a lot easier to find real bugs if you try to make your code clean from all issues like this. [return]

3. One reason I trust Malis is that he's a lot more careful about what he claims than a stereotypical founder. For example, when I joined the pgrust discord to check out what people are saying, I saw this message from Malis (dated a month ago, 2026-09-17, an eternity in agentic coding land) in response someone asking about pgrust's readiness for use as a production database:

   I would not recommend using it as your primary production database today. We are setting a production deployment for a company right now, but they are more tolerant to their production database going down than most people. We'll have good empirical evidence on how reliable pgrust is after they start running it

   Malis doesn't seem like someone who would claim something is production ready when it's clearly wildly unready, like many well-known projects have, and it turns out he's not doing that.

   I wrote the draft of this a week before publishing and didn't have time to publish it. Since then, there was this pgrust Clickhouse benchmark controversy. On looking into this, the accusations of nefarious cheating seem to be false, and the accusation that it was premature to submit the preview of pgrust to the benchmark is something Malis acknowledged. I do think this is the kind of thing that a project will never live down, in that regardless of the truth of the matter, people will be bringing this up years later. You can see this right now in discussions around that specific controversy, and as we discussed when we look at Ed Zitron's prediction track record and when we looked at futurist predictors like Kurzwil, almost nobody actually looks at what actually happened and people will repeat incorrect statements that were based on vibes indefinitely.

   I don't think I can really present all of the evidence for why I trust Malis (technically, I could do it, but it would easily be a 20k+ word post), but if you look at how he discusses pgrust vs. how typical database founders market their new DB projects, it seems dramatically different to me. Of course there are extreme examples like MongoDB, which disputed correctness issues for years despite Kyle Kingsbury repeatedly showing that the anecdotes people had about data corruption were real. For an example that's more typical, a friend of mine worked as an early employee for a database startup. Their webpage bragged about doing some huge number of QPS. On testing it, my friend found that the claim was off by multiple orders of magnitude and suggested that the webpage get changed. The founder disagreed because they were going to hit the performance numbers, so it was still a true claim. After multiple years of performance improvements, the company eventually reduced the claim by one order of magnitude and they can still only hit that number with a very contrived, unrealistic, workload. This is, unfortunately, pretty normal for a company peddling a product based on its performance. By contrast, when I look at the pgrust discord to see what Malis is saying about performance, here's his most recent message:

   Right now, the best way to help is to find bugs and performance issues in 0.4-beta. The new query engine is very fast on some workloads, but it causes performance regressions on others, and it doesn't support some more complex queries. We're trying to track down everything that's either slower than Postgres or errors out. It's not an easy thing to do given the massive space of queries you can run

   We want to fix these things before the general release of 0.4-beta. Off the top of my head, there was a big performance regression in very fast queries. This is because we primarily optimized the new query engine for large-scale OLAP-like workloads. I'm doing optimization work to fix the very slow queries. I don't think there's any fundamental reason why the new engine should be slower than the previous one for fast queries

   Even though he was a founder and exec of a successful startup prior to starting the pgrust project, he never really got into the mindset that most founders seem to, where they seem to be able to say anything and believe anything they say. You might say this is a low bar, and I'd agree, but I still think it's rare and it's why I trusted what he says about pgrust without looking into it and also why, when I looked into it, his claims seem to be borne out by reality.

   [return]

4. Another way to look at it is [this framing from Thomas Ptacek]:

   The last year in the technology industry has felt like 100 years all happening at once. Our industry is destabilized in a way nobody’s experienced since the advent of the personal computer. Every limit AI runs up against collapses within a month. Everything we do with frontier models today, in a few years we’ll be doing instantaneously and for free.

   Given the pace of change recently, I find it understandable that most people, even most programmers, aren't quite aware of what's possible right now. Speaking to non-programmer responses, one of the things I thought was interesting was how many people objected to this review of Ed Zitron's prediction accuracy because they thought that AI capabilities hadn't increased in the past two years. For most people I talk to, the idea that AI didn't improve from fall 2024 to fall 2026 is completely baffling (even if you think of programming as an exception and look at other capabilities), but there's a lot of people out there who are perhaps looking at the Google AI summary at the top of search results or using Copilot integrated into some Microsoft suite, who don't realize that capabilities have advanced. At every scale, there seems to be an information gradient like this, where very few people outside of labs have a good idea of what the most productive people are doing with models today.

   [return]

5. This comment about an increase in cheating online has come to pass. On re-reading the actual comment that I wrote, I didn't explicitly write that it seemed inevitable that there would be a large increase in cheating, although it can be inferred from connecting the statements there together, and I've been on the record in public on some discords, etc.

   I don't follow the online game scene, but I'm told that for Azul (the most popular game on BGA, the biggest online board game platform for modern board games), 3 of the top 10 places last season were taken by cheaters, including top place. Also, as I've noted elsewhere in the past, BGA doesn't seem equipped to make reasonable policy decisions on almost any level. Things like the mechanics of their Elo ranking algorithm, their structure for game seasons, their policy for counteracting collusion, their policy for countering cheating, etc., are worse than the most naive straightforward solution you can think of so, as expected, they don't have a reasonable policy to counter cheating. The top ranks of the most popular game have been overrun with cheaters and this should be expected to spread to other games. As harms from AI go, this is a fairly benign one. It just happens to be one of a double-digit number of examples I saw last week of people being more ambitious in the software they create. I'm half mentioning this because it's easily quantifiable, and half because I predicted this would happen and it seems reasonable to follow up and see what happened.

   [return]

6. Pandoc supports {html, docx, xlsx, pptx, csv, markdown, gfm, commonmark, commonmark_x, markdown_strict, markdown_mmd, markdown_phpextra, markdown_github, rtf, epub, odt, tsv, latex, ipynb, mediawiki, rst, asciidoc, bibtex, biblatex, org, typst, jira, docbook, endnotexml, ris, opml, dokuwiki, textile, man, mdoc, jats, csljson, fb2, pod, vimwiki, djot, creole, tikiwiki, twiki, t2t, haddock, muse, json, xml, native} as input formats and {pdf, html, html4, docx, plain, pptx, markdown, gfm, commonmark, commonmark_x, markdown_strict, markdown_mmd, markdown_phpextra, markdown_github, rtf, epub, epub2, odt, latex, ipynb, mediawiki, rst, asciidoc, asciidoc_legacy, bibtex, biblatex, org, typst, bbcode, bbcode_phpbb, bbcode_steam, bbcode_xenforo, bbcode_fluxbb, bbcode_hubzilla, jira, docbook, docbook4, revealjs, beamer, opml, dokuwiki, textile, man, texinfo, context, jats, jats_publishing, jats_articleauthoring, tei, icml, opendocument, csljson, fb2, ansi, chunkedhtml, djot, xwiki, zimwiki, vimdoc, haddock, markua, muse, ms, slidy, s5, dzslides, slideous, json, xml, native} as output formats (native is a serialized version of Pandoc's AST). [return]

7. BTW, this makes public evals not very meaningful for many use cases because what I and other programmers who are using models often care about is how good the model is at a task where the model still needs human guidance, not how good the model is at a task that's trivial for the model. [return]

8. This is not a claim about the future. Perhaps almost all or perhaps even all of the software skills I have will become irrelevant in the near future. But we're not there yet. At the moment, programming skills seem higher leverage than ever before. There's something a bit funny about the current moment, where it seems like having the skills that models are replacing makes one much more effective than people who are trying to use models without having those skills. I like Max Bittker's framing that models are very good at counterfeiting and the level of expertise it takes to distinguish a counterfeit from the real thing has been increasing as models improve. [return]

I’ve been in London for the last month and have heard policymakers react to the wake up call Elon Musk just delivered.

The decision of the U.S. to shut off intelligence and throttle the military version of Starlink called Starshield to Ukraine may have been a blip in the news in the US, but it has sent tremors across Europe.

For years doomsayers in the UK and Europe were worried about the U.S. having a kill switch in weapon systems and critical commercial technology that could disconnect them at will. Most people, including me, thought that it was a paranoid fantasy.

Until it happened. Seemingly on a whim of a single individual, (according to the Wall Street Journal) Elon Musk personally ordered the restrictions, and the effect was to tilt Russia’s war against Ukraine in favor of the Russians. This wasn’t an articulated U.S. foreign policy nor was it an announced act of the U.S. government. From news accounts it appears to have been a personal decision of an American CEO acting unilaterally.

Blowback

For whatever reason, whether it was to satisfy a request from the White House or some personal animosity to the Ukrainians, from Europe’s perspective, that’s almost beside the point. An American company operating critical military infrastructure was able to degrade the capability of an American partner in the middle of a war. Without warning.

The message heard loud and clear here in Europe was that the U.S. is an unreliable security and technology partner willing to throw any partner under the bus, at any time, for any reason.

The conversations I’ve heard have included, “what happens if other U.S. CEOs – Microsoft, OpenAI or Anthropic, Oracle, Palantir, or Google – decide to restrict/limit or deny access to their products? What capabilities would Europe lose if enterprise software and/or AI across Europe goes dark?” This was a laughable scenario until this week. To a lot of European decision makers, it is no longer outside the realm of possibility.

The lesson that was brought home was that no nation, Europe or elsewhere, can continue to outsource its national sovereignty and security by depending solely on US data systems or intelligence.

Unintended Consequences

Besides the political ramifications, the business ones are going to come home to roost. If every U.S. vendor is now a kill-switch risk, the allied market for that whole ecosystem shrinks. As it sinks into the European countries that the U.S, can simply shut off any technology, over time they will find and build alternatives. In the years ahead, this may cost American companies tens if not hundreds of billions of lost revenue.

Europe has been debating digital sovereignty for years. Up to this year, the idea of cloning the capabilities of the best U.S. products seemed like an insurmountable problem. And some, like low-cost access to space, still are. But today AI collapses the cost of creating sovereign software at exactly the time that geopolitical risk is increasing customers’ willingness to switch. AI makes recreating the features, functions and compatibility of Microsoft Office, Oracle, Salesforce, Palantir, Anduril, et al. within reach. AI puts it well within the capabilities of most nations.

The U.S. spent decades establishing itself as the world’s indispensable technology supplier. A substantial part of that advantage depended on customers believing American technology would remain reliable and available.

Going forward a U.S. founder selling into European defense or government now needs an answer to “who holds the switch” before the first meeting; and European founders just got sovereignty as a sellable feature.

What was done in a fit of pique will cost the U.S. both leadership and jobs.

Buckle up.

TL;DR

⚠️ Brotli is a widely used compression library from Google. On Wednesday, October 21st NodeJS disclosed CVE-2026-111497, dubbed “node2shell,” wherein a Node.js process that decompresses attacker-controlled Brotli data can be made to leak memory or run arbitrary code. On Thursday, Python disclosed that Python packages were vulnerable to CVE-2026-111497 as well. On Friday, October 23rd, it has become known that this CVE affects other major languages and service applications using HTTP, where patches are not yet fully available. Check if you are decoding Brotli as a dependency within your libraries. Stay on guard for wider announcements and be ready to upgrade as soon as possible. If there is no patch available for a library or application that may be affected, we recommend disabling Brotli decoding until patches become available. Update 4:12PM PT: Patches are available for libbrotli, Python, C++, NodeJS, bun, Envoy and more coming. Apache and Nginx do not decode brotli by default.

Update 4:15 PM PT: OpenAI shares that they discovered this vulnerability in February. While the root cause issue was fixed in the Brotli source code at the time, it wasn’t known that it gave rise to a security vulnerability. OpenAI's advisories here.

What is Brotli?

Brotli is a lossless compression algorithm and data format developed by Google. It’s a way to make data smaller for storage or sending over the internet. Software then unpacks it to recover the original data. It’s widely used for HTTP compression to compress web content such as HTML, CSS, and JavaScript.

What is CVE-2026-111497?

CVE-2026-111497 is an out-of-bounds write in the Google Brotli decoder affecting versions 1.0.3 through 1.2.0 on all platforms. This is the same Brotli decoder flaw behind Node2Shell announced a couple days ago. A remote, unauthenticated attacker can send a crafted Brotli-compressed stream to corrupt memory, leak sensitive data, crash the process, or potentially execute arbitrary code. It carries a CVSS v4.0 score of 9.2 (Critical), and Node.js is affected through its bundled Brotli library.

A bug makes the Brotli decoder keep accepting more bytes when it should reject the data. Those bytes can overflow a small working buffer and damage the program’s memory. An attacker can use that damage to control what runs, such as their own code.

OpenAI found this in February reported it as unauthenticated remote code execution that works in both directions, so a malicious client can take over an unpatched server and a malicious server can take over an unpatched client that accepts compressed responses.

The root cause was fixed in the Brotli source code in February, but it wasn't recognized as a security issue at the time and no release shipped with the fix until now. That means every project that bundles Brotli has to ship its own update.

‍You’re exposed if any environment has Brotli 1.2.0 or earlier or brotlicffi 1.2.0.x installed, since HTTP clients such as requests, httpx, and aiohttp load these packages automatically when they’re present. Upgrade to Brotli 1.2.1 or brotlicffi 1.2.1.0 and rebuild any container images that bundle them. As of October 23, advisory databases don’t map the CVE to either PyPI package, so pip-audit, Dependabot, and Safety won’t flag vulnerable versions yet.

‍You can see what we’ve tracked here in Aikido Intel.

• Python
• C++
• Libbrotil
• NodeJS

As for unaffected software, Apache’s standard mod_brotli does not decode Brotli

How to patch HTTP2Shell CVE-2026-111497

For CVE-2026-111497, update the affected runtime or Brotli library:

• Node.js: upgrade to 22.23.4, 24.22.1, or 26.12.1, or a newer supported release in the corresponding release line.
• Native Brotli / libbrotli: upgrade to 1.2.1 or later. For older release lines, use patched 1.0.10 or 1.1.1.
• Python packages: upgrade Brotli to 1.2.1 or later and brotlicffi to 1.2.1.0 or later, wherever installed.
• curl / libcurl: update its Brotli dependency to 1.2.1, a patched older release, or the vendor’s security-fixed package. Bundled or statically linked copies require an updated application build.
• Bun: upgrade to 1.4.4 or later. Rebuild compiled executables that bundle Bun.
• Envoy: upgrade to 1.40.0, 1.39.3, 1.38.6, or 1.37.8, or a newer release in a supported branch. 1.36.12 also includes the fix, but the 1.36 branch is unsupported.
• Istio: the fix is included in 1.29.9, 1.30.6, and 1.31.2. Use a newer supported release in the corresponding line; the 1.29 branch is end of life.
• Operating-system packages: install the vendor’s security update. Vendors may backport the fix without changing the upstream version number.
• Other HTTP servers, reverse proxies, API gateways, and HTTP clients: update the affected runtime or Brotli library they use. Where Brotli is bundled into the product, install the product vendor’s release that explicitly addresses this CVE. For managed services, confirm remediation with the provider.

Update dependency locks, rebuild affected applications and container images, redeploy, and restart services. Verify that running workloads use the patched components.

Fixed releases, reviewed October 23, 2026

What to do if you can't upgrade yet

These workarounds reduce exposure while you prepare the upgrade:

• Python applications: where Brotli support is optional, uninstall both Brotli and brotlicffi from affected environments. Remove dependency extras such as httpx[brotli] and any other requirements that would reinstall them, then regenerate dependency locks and rebuild/restart workloads. HTTPX, Requests, and urllib3 retain gzip and deflate support without these optional packages. However, a server that sends Brotli anyway may produce an unreadable response or an application error; automatic fallback is not guaranteed.

  • Uninstalling both Python packages and restarting workloads removes the decoding paths provided by those packages for incoming and outgoing data. It does not remove separate Brotli decoders bundled into other runtimes, libraries, or products.

• Incoming HTTP requests: reject requests whose Content-Encoding includes br at the reverse proxy, load balancer, or application middleware, before any vulnerable decoder processes the body. Match br case-insensitively, including within lists of encodings.

• Outbound HTTP requests: remove br from Accept-Encoding. Also disable Brotli response decoding or reject responses whose Content-Encoding includes br before decoding begins. Ensure this check occurs before automatic body processing; checking after the client has downloaded and decoded the response is too late.

Changing Accept-Encoding alone is insufficient: a malicious server can send a Brotli response regardless of what the client advertises, and urllib3 selects its decoder from the response’s Content-Encoding header.

Keep these controls in place until affected components are patched and verified. Review coverage as vendors confirm additional affected components or attack paths.

How Aikido helps with CVE-2026-111497

Aikido flags vulnerable versions of Brotli and brotlicffi in your repositories, container images, and dependencies, including when they arrive as transitive dependencies. Your dashboard shows which services still run a vulnerable version, so you can upgrade the ones that call outside servers or accept compressed request bodies first.

Once you've upgraded, Aikido checks if the finding is gone.

The timeline

Brotli is a widely used compression library from Google. On Wednesday, October 22nd NodeJS disclosed CVE-2026-111497, dubbed “node2shell,” wherein a Node.js process that decompresses attacker-controlled Brotli data can be made to leak memory or run arbitrary code. Users were exposed if their app or a library decompresses bad Brotli data (e.g. the built-in fetch(), body-parser).

The next day, Python released a security update that patches the same flaw; Python packages were also vulnerable to CVE-2026-111497. Brotli flaw may affect even more languages and service applications, given how widely integrated the library is downstream. It turns out OpenAI had found this back in February.

Techincal details of CVE-2026-111497

The flaw sits inDecodeBlockTypeAndLength(), which handles block switches, the points where a Brotli stream changes which Huffman tree it uses. In vulnerable versions the function returns a plain boolean, and BROTLI_FALSE can mean either "this stream is malformed" or "I need more input." Callers can't tell the two apart, and on the fast path the decoder discards the return value entirely.

A crafted stream can declare a single block type and then exhaust that block's length, which forces a switch that can't succeed. The decoder then either reports that it needs more input for a stream that's complete, or keeps decoding with the block state it already had. That second path leads to the out-of-bounds write.

The upstream fix, google/brotli commit 0ed4268, makes the function return an error code and adds a new error, BROTLI_DECODER_ERROR_FORMAT_BLOCK_SWITCH (-17). Decoding now stops on a format error on both the safe and fast paths. OSM diffed the Brotli 1.2.0 and 1.2.1 source distributions and found that 1.2.1 differs only in the version bump and the decoder changes from that commit.

Brotli vulnerabilities in Python libraries (CVE-2026-111497)

The Brotli flaw behind Node2Shell (CVE-2026-111497) affects Python. Two PyPI packages, Brotli and brotlicffi, compile the same vulnerable Brotli decoder that Node.js bundles. An official advisory was posted on GitHub. Brotli has 82.8 million downloads per month.

You're affected if any Python environment you run has Brotli 1.2.0 or earlier, or brotlicffi 1.2.0.x, installed. Popular HTTP clients such as requests and httpx load these packages automatically when they're present. Some packages automatically install Brotli as a dependency such as Scrapy, while others require explicit mention of it. An attacker can reach the vulnerable decoder by sending a malicious Brotli response to a request your app makes, or by sending your app a request body compressed with Brotli. That means an app can be exposed without importing Brotli anywhere in its own code. It’s a popular transitive dependency, so check your systems.

To fix it, upgrade to the patched releases:

• Brotli -> 1.2.1
• brotlicffi -> 1.2.1.0

The release notes for both fixed versions don't mention the CVE. As of October 23, advisory databases don't map it to either PyPI package yet, so pip-audit, Dependabot, and Safety won't flag these versions.

Who is affected by CVE-2026-111497 in Python?

You’re affected if any of your Python environments have one of these packages installed:

Your code doesn’t have to import either package. These HTTP clients advertise Accept-Encoding: br and decode Brotli responses automatically:

• urllib3 (and therefore requests) when Brotli or brotlicffi is installed
• httpx with the brotli extra
• aiohttp when Brotli is installed
• Scrapy, and anything built on the clients above

Attacker data can reach the decoder through two routes:

• Outbound requests. Your app fetches a Brotli response from a server an attacker controls, such as through a webhook delivery, URL preview, crawler or “fetch this URL” feature.
• Inbound requests. Your framework or middleware decodes a request body sent with Content-Encoding: br, which is the Python equivalent of the Express 5 case in Node.js.

How to fix CVE-2026-111497 in Python

The fix is a package upgrade, and it takes four steps:

1. Upgrade whichever package you have installed with pip install -U "Brotli>=1.2.1" or pip install -U "brotlicffi>=1.2.1.0". If you use a lockfile with Poetry, uv, or pip-tools, update the constraint and regenerate the lockfile.
2. Rebuild your container images and any wheels that bundle either package. An image built on a vulnerable version stays vulnerable until you rebuild it.
3. Check every environment you run, including laptops, CI runners, servers and containers. Many projects pull in Brotli through an HTTP client extra, so look at your full resolved dependency tree.
4. Rely on a manual check for now. pip-audit, Dependabot and Safety won't report this until the advisory databases map the CVE to the PyPI packages.

If your Python Brotli bindings came from a Linux distribution package instead of PyPI, the fix comes through your distribution, so check its security advisory as well.

What is the Python Brotli vulnerability (CVE-2026-111497)?

Python Brotli vulnerability refers to CVE-2026-111497, the Brotli decoder flaw known as Node2Shell, as it affects Python packages on PyPI. The CVE record pairs CWE-252 (unchecked return value) with CWE-787 (out-of-bounds write).

How serious is CVE-2026-111497 for Python, and is it being exploited?

The CVE record gives the flaw a CVSS 4.0 score of 9.2, and Node.js rates it high severity. OSM built Brotli 1.2.0 and 1.2.1 with AddressSanitizer and ran a crafted stream through both. Version 1.2.0 consumed all of the roughly 23 MB stream and kept reporting that it needed more input, whether the stream arrived in one piece or in chunks as small as one byte. Version 1.2.1 rejected the stream with error -17 every time.

OSM didn't pursue memory corruption or code execution in the Python extension. The decoder code is the same code that researchers exploited against an unmodified Node.js 24.22.0 build, as Imperva and others reported. For Node.js, The Hacker News reported no exploitation in the wild as of October 22.

Treat it as urgent anyway. The bug is a memory-safety flaw in a component that handles untrusted network input, and the fix is a routine version bump. Services that call outside servers or accept compressed request bodies should go first.

How to check if your Python app is exposed

1. Check installed versions with pip show Brotli brotlicffi in each environment, and with docker run --rm pip show Brotli brotlicffi for each container image.
2. Find what pulls the packages in. Run pipdeptree -r -p Brotli to see which dependencies require it, and search your lockfiles with grep -i brotli poetry.lock uv.lock requirements*.txt.
3. List the outbound HTTP calls that go to servers you don't control, and any endpoints that decode Content-Encoding: br request bodies.
4. Upgrade the services that match step 3 first, then the rest.

If you're reproducing the bug from Python, brotli.decompress() raises a generic brotli.error: decoder failed for any error, so you can't see which error code fired. A Decompressor() fed the crafted stream incrementally will never finish on 1.2.0.

FAQ

‍Which versions fix CVE-2026-111497 in Python?

Brotli 1.2.1 and brotlicffi 1.2.1.0, along with any newer releases.

Do I need to import Brotli in my code to be affected?

An installed package is enough. urllib3, requests, httpx and aiohttp use Brotli or brotlicffi automatically when either is present and decode Brotli responses from any server your app calls.

Will pip-audit or Dependabot flag this?

Not yet. As of October 23, the CVE record, OSV and the GitHub Advisory Database don't list either PyPI package. OSM is submitting PYSEC entries and a PyPI package mapping for the GitHub advisory so that tooling picks this up.

Is there a workaround?

Uninstalling Brotli and brotlicffi removes the vulnerable decoder, and HTTP clients fall back to gzip and deflate. Rejecting Brotli-encoded requests at the edge reduces inbound exposure, but it doesn't protect against Brotli responses your app fetches.

In September 2014 I published a short post announcing rqlite. The idea was simple: take SQLite, put a Raft log in front of it, and replicate every write to a cluster so that the loss of a node did not mean the loss of the data. The source went on GitHub that same weekend. As far as I can tell, that was the first open-source system to combine Raft and SQLite. Nobody else was doing it, and for several years nobody else tried.

Today “distributed SQLite” is a category with a dozen projects in it, a few of them venture-funded, two of them acquired. Since the people now writing about the category tend to start the story around 2021, I thought I should write down the version that starts in 2014. I was there for all of it.

2014: rqlite

The need was concrete. A clustered system I worked on at jut.io needed a small amount of relational storage, and the question of which node should hold it had no good answer. Replicating SQLite with a consensus protocol solved it. We went another way at work, but I kept building rqlite because the problem was general and the solution was small. The first version used goraft, the same library InfluxDB and etcd used at the time (I subsequently joined InfluxDB as a core developer around the same time). In early 2016 I replaced it with Hashicorp’s Raft implementation which rqlite still uses.

The design decision that mattered most was made then and never changed. rqlite is a CP system. Writes must go through the leader, a majority must acknowledge, and a partitioned minority refuses writes. Everything else, the HTTP API, the read-consistency levels, the snapshotting, followed from that.

Before 2014, and alongside it

Two things deserve mention for honesty’s sake. Expensify had been replicating SQLite inside its own infrastructure for years with a Paxos-style protocol; they released it as Bedrock in 2016. And ActorDB, an Erlang system that gave each “actor” its own SQLite database replicated by Raft, appeared around the same time as rqlite. Neither was widely adopted outside its origin, but both show the idea was in the air. What rqlite did was make it a standalone, general-purpose, open-source thing anyone could run.

2017: dqlite

Canonical built dqlite to get LXD and later MicroK8s an embedded replicated store without running etcd. Mark Shuttleworth actually called me once, and we talked, hoping that rqlite could be made to work, but it never quite fit. dqlite began in Go and was rewritten in C with its own Raft library. It is the project most similar to rqlite in intent, and the main difference is of one of packaging: dqlite is a library you embed, rqlite is a server you run. Canonical’s need was a database inside their products: mine was a database beside anyone’s product.

2021: Litestream, and a change of approach

Ben Johnson (whom I worked with at InfluxDB) released Litestream in early 2021. Litestream does not use consensus at all. It watches the SQLite WAL and streams it to S3, giving you disaster recovery for a single-node SQLite database at almost no cost. Ben’s argument was that most applications do not need a cluster; they use one machine and need a way to recover if it dies. He was right about a great many applications, and the “SQLite is enough” movement of the next few years, with blog posts and conference talks, owes a lot to that argument.

Litestream and rqlite answer different questions. Litestream: what if the machine dies? rqlite: what if the machine dies and I cannot wait to restore? The first question is more common, the second is the one people have when they come to rqlite.

Fly.io hired Ben in 2022 and shipped LiteFS, which replicates SQLite at filesystem level with leases from Consul. It was the first attempt to make replicated SQLite a platform feature rather than a thing you deploy. And in a final twist, Litestream inspired rqlite’s v8.0 approach to handling the SQLite WAL.

2022: forks and extensions

Two projects in 2022 took approaches that neither rqlite nor Litestream had. cr-sqlite added CRDTs to SQLite as an extension, so that multiple writers could diverge and merge without a leader. That is the AP corner of the design space, and it suits local-first applications where rqlite’s CP guarantees would be a hindrance.

libSQL forked SQLite itself. ChiselStrike, later Turso, wanted to change things inside the engine that the SQLite project, which does not accept outside contributions, and would not change. On top of the fork they built a hosted service with replicas at the edge. Turso raised venture money on that, and in 2026 Supabase acquired them.

Around the same time, SQLite Cloud appeared, offering hosted SQLite replicated with Raft. It is the project closest to rqlite’s original design, wrapped in a cloud service.

2022 onward: the platforms

Cloudflare announced D1 in 2022, SQLite as a serverless database inside their edge network, and later put SQLite storage inside Durable Objects. This is replicated SQLite as a managed product from a company that does not need to tell you how it works. Turso’s service is the same shape. The category had gone from “a thing you run” to “a thing you buy.”

What the twelve years show

Three things, I think.

The core was right. SQLite as a storage engine, with replication added on the outside, has been reinvented by at least eight teams with different needs and different protocols. Nobody has found a reason to stop doing it.

The design space has three corners, and they appeared in order. Consensus first (rqlite, dqlite): one leader, strong consistency, a cluster. Streaming second (Litestream): one writer, cheap recovery, no cluster. Merge third (cr-sqlite, and parts of the local-first movement): many writers, eventual consistency. Each is correct for a different question, and most of the arguments are arguments between their users about which question is being asked.

And the money went to the platforms. rqlite, dqlite, and Litestream are open source and free. The projects that raised the capital or got acquired are the ones that turned replicated SQLite into a service with a bill. That is the usual fate of infrastructure ideas, and I do not mind it. But it is worth recording that the idea was open source for eight years before anyone sold it.

rqlite is still here. It does the same thing it did in 2014, a good deal better, and it is still the thing you reach for when you want a fault-tolerant relational database you can run yourself, understand fully, and not pay anyone for. Twelve years in, I am content with that position in the story.

In Why Lean is Faster Than Rust, the author shares an impressive result: a DEFLATE compression library written in Lean 4, a functional programming language and proof assistant, outperforming an equivalent Rust package that crates.io says has been downloaded over a billion times. From the article:

lean-zip beats every single compression level miniz_oxide provides, i.e. we're capable of providing the same compression at higher throughput. At its fastest setting we're 15% quicker and the output is 8.8% smaller.

However, the article also notes that while they managed to surpass Rust's miniz_oxide, they were no match for the end boss of DEFLATE performance: libdeflate, written in C.

lean-zip is certainly not the best here: libdeflate unsurprisingly blows it out of the water (unsurprisingly because this is a very carefully tuned implementation using architecture-specific SIMD, that we can't touch in Lean).

Enter Roc, a functional programming language (but not a theorem prover) with an explicit focus on runtime performance.

Roc's tagline is "Fast, Friendly, Functional," and Fast comes first because both running fast and compiling fast have always been central goals. Roc is not a mature language (it's still weeks away from its first numbered release, 0.1.0), and its nascent package ecosystem has had almost no optimization work done on it. Still, between its language design, longstanding optimizations, and brand-new compiler implementation, I figured Roc already had enough of a foundation in place to take on the champ: libdeflate.

So I spent a bunch of time grinding through performance optimizations on roc-deflate, a package Niclas Ahden wrote (and uses in paid production work) for DEFLATE compression. I didn't expect to beat libdeflate's performance across the board, but I wanted to see how close it could get on the non-streaming APIs (as roc-deflate hasn't implemented streaming yet), using the classic Silesia corpus for benchmarking and verifying that it produced identical outputs.

Under 10% Slower

Here's where the benchmarks stand today, running on the 2021 M1 MacBook Pro where I did all the optimization work, as well as on an AMD Ryzen 9 3900X Ubuntu machine for an x64 comparison.

roc-deflate ran these 144 benchmarks slower on average than libdeflate, in the median case 8.9% slower on the Ryzen 3900X, and 2.6% on the M1 MacBook Pro.

Compression on Silesia corpus

So while we didn't outperform the champ on average, a median slowdown percentage in the single digits compared to such a carefully-tuned SIMD-powered C library is wild. It's frankly not a result I knew was achievable going in, since Roc is a functional language with automatic memory management racing against code that doesn't have to do bounds checks. (Bounds checks have a cost, but then again, the last time a skipped bounds check in C compression code led to a major Remote Code Execution vulnerability was yesterday.)

The graph shows more detail than just the median gap. On the worst-case inputs, roc-deflate was 18% slower than libdeflate, but on the best-case inputs, roc-deflate was slightly over 18% faster! That's not because Roc has some magical optimization that C can't access; it's just that the two implementations are different, and although on average roc-deflate is slower, for certain inputs it happens to be faster. So while it's possible to get a speedup by switching from libdeflate to roc-deflate, you shouldn't expect it on average.

To be clear, C's performance ceiling is higher than Roc's. For any input where roc-deflate currently outperforms libdeflate, C code can close that gap. The reverse is not true; Roc code is memory-safe, does automatic reference counting, checks bounds, and supports no Rust-style unsafe to opt out of any of that. These properties give Roc code stronger safety guarantees than code written in C, Zig, Rust, and other languages that allow full access to the hardware—and the price it pays for those guarantees is a lower performance ceiling.

Obviously we can't extrapolate from this one result about how close Roc can get to C in the general case, but now something has been demonstrated that nobody knew was achievable before. As far as I know, this is the most optimization effort that has been put into any Roc package to date, so how close others can get to this result is a complete unknown right now. So while I'm thrilled by this result, I'm fully aware that we won't know how well it generalizes until we've optimized a wider sample of real-world Roc code bases!

What the Performance Gap Buys

An obvious question is "Why settle for any amount slower?" If you could get C levels of performance, why not use libdeflate itself and close that remaining gap?

A pure-Roc implementation has several advantages over a pure-C one:

• Memory safety. Roc code is fully memory-safe and has no language primitives that let you opt out of it. This code is only able to approach C levels of performance because Roc's optimization pipeline determined that many bounds checks and reference counting operations were provably safe to optimize away. Compression packages written in pure Roc don't have access to the unchecked memory operation that caused yesterday's Brotli Remote Code Execution vulnerability.
• No side effects. Roc's type system infers which functions are pure and which can perform side effects. Pure functions' types use a thin arrow, like Bool -> Bool, whereas effectful functions use a thick arrow, like Bool => Bool. (As with memory safety, Roc enforces this and supports no way to opt out.) So when you see that roc-deflate's functions have only thin arrows in their annotations, you know they really are pure functions.
• Supply-Chain Security. Roc does not have an arbitrary FFI, meaning you know for sure that roc-deflate's pure functions can't even attempt to do I/O or access arbitrary memory. Given the explosion in supply chain attacks, it's valuable to know that the damage a malicious actor can do with a pure function in a compromised Roc package is limited to returning the wrong value, crashing, or wasting system resources. Roc packages also don't support build scripts, which was the vector used to successfully compromise the popular Rust arrayref package two months ago.
• Portability. Roc pure functions are designed to be extremely deterministic: they always return the same values when given the same arguments—not just when run on the same machine, but also on any target you compile that code to. (The functions may also crash or fail to terminate, but that behavior should also be deterministic across targets.) The only way a Roc pure function should be expected to behave differently when run on two different systems is if one system runs out of memory mid-execution.

If any of these guarantees don't hold in practice, it can only be due to a defect in Roc's compiler, the platform the application is running on, or the underlying hardware. So when you see in roc-deflate's docs that the entire package is 100% pure functions, you can be confident you are getting all of the above properties from everything the package has to offer. The type system enforces it!

The reason we've been working so hard on all this might become clearer if you look at the proposition in reverse: would you give up guaranteed memory safety, guaranteed lack of side effects, guaranteed deterministic behavior across all targets, and the security benefits of having no exploitable build scripts in packages...for an average speedup percentage in the single digits?

That answer can legitimately vary by use case—Roc's own compiler, for example, wants the maximum performance the hardware can deliver—but I bet a lot of people reading this can see why we're so excited to have achieved this result.

Roc-Specific Optimizations

Most of the optimizations that improved roc-deflate's numbers came from general optimization techniques that are widely beneficial across languages. (I always recommend Casey Muratori's Performance-Aware Programming Series, which I paid for and learned a lot from.) Some of the optimization techniques were Roc-specific, though, and I wanted to mention them here as a reference for others. Even if you're not writing Roc code, this can also give you an idea of how Roc achieves its performance.

Avoid Unnecessary Sharing

One of Roc's most effective optimizations is opportunistic mutation (which Koka calls "Functional but In-Place"). You can learn how it works in Roc's docs on opportunistic mutation, but from a performance optimization perspective, it's critically important to avoid unnecessary sharing so that values remain unique and therefore eligible for in-place mutation.

Hoist Bounds Checks

You can't opt out of bounds checks (e.g. getting a value out of a dynamically-sized collection) in Roc like you can in C, Zig, and Rust, but Roc's compiler can optimize them away if it knows they'd be guaranteed to pass. Bounds checks are most costly inside loops, so you can get a lot of mileage out of adding something like if collection.len() <= highest_index_used { return 0 } before the loop starts. This adds an additional check that looks redundant, but it can allow the optimizer to safely delete costlier checks inside the loop, once it can tell that failures would not be reachable after that early return.

Build Lookup Tables at Compile Time

Roc's compile-time evaluation can be used to assemble lookup tables at compile time as long as nothing depends on runtime inputs. Even better, since their lengths are known at compile time, you can do things like Ok(entry) = lookup_table.get(5) to do an exhaustive pattern match that Roc's compiler will know is valid thanks to compile-time evaluation. This avoids runtime bounds checks that non-constant tables might have to pay for.

Consider Wrapping Arithmetic

In Roc, a + b desugars to a.plus(b). For an integer like U64, U64.plus includes an overflow check, whereas plus_wrap does not. Although plus is the safer default, overflow checks have a runtime cost, as CPUs can do wrapping integer arithmetic in one instruction. When Roc's compiler can prove that overflow won't occur, both plus and plus_wrap will optimize to the same instructions, but when it can't, you may get a speedup by choosing plus_wrap.

Room for Improvement

I'm sure there are still opportunities to improve roc-deflate's performance—especially on x64 CPUs—as well as the optimization pipeline in Roc's compiler.

At this point I feel reasonably confident that all the low-hanging fruit has been picked, but if anyone wants to try it out for themselves, here's the script I ran to set up and run the benchmark suite on Linux. (Note that Claude Opus 5.5 generated this script, and I haven't vetted it for security; run it outside a sandbox at your own risk!) It pins both Roc compiler revisions and roc-deflate versions, so you'll want to adjust those commits if you're trying out potential improvements.

For my part, I'll be focusing on the upcoming Roc 0.1.0 release from now until it's out. It's been eight years in the making, so I'll be very happy for us to finally have a numbered release! If you'd like to learn more about Roc, check out roc-lang.org or come say hi on Roc Zulip!

A memory-safe systems language with no heap allocator, garbage collector, or lifetime annotations. In this repository's benchmarks, Goose runs faster than C++ and Rust while using less memory.

Tutorial · Reference · Specification · Packages · Samples · Benchmarks · Standard library

Goose has syntax familiar to C and Rust programmers, but there is no heap. Every dynamic value lives inline on a data stack managed by the compiler. Growth advances a pointer, and memory is freed when its scope ends. The language's type and lifetime rules make this memory model safe to use.

Why Goose

• Faster than C++ and than safe Rust, while memory safe. Over sixteen benchmarks Goose runs at 3.3x the speed of idiomatic C++, 1.16x hand-optimized C++ and 1.12x the best safe Rust, on 1.9x, 1.3x and 1.2x less memory (results). These gains come from Goose's data layouts and memory model.
• No allocator, no GC, no reference counting, no destructors. Memory is a handful of data stacks that the compiler assigns statically. Freeing a million-element structure is one store, however deeply it nests.
• Growth preserves references. A reference into a growing array stays valid for as long as the array does. You keep typed references where C++ must reserve and safe Rust uses u32 indices in the corresponding benchmarks.
• Memory safe with zero annotations. No lifetime syntax, no aliasing or exclusivity rules, no unsafe. The compiler infers what every reference is rooted in and rejects references that could outlive their owners.
• Nested data stays inline. A string, an array of strings, a record with variable-size fields and an array of those records are each one contiguous block with no pointer in it. A record that is 160 bytes and an allocation in C++ is 29 bytes and none in Goose.
• Enums that cost what they hold. Variable-mode ADTs give each value its own variant's size rather than the largest one's: 4x less memory and 2x the speed of a Rust enum on the benchmark that exercises it.
• Links narrower than pointers. A relative reference stores a typed, checked link as a 1, 2 or 4-byte offset. Structures built from them are position independent, so your data structure is already its file format: saving is a write, loading is a read plus a verification pass that rejects hostile bytes.
• Everything is built in place, guaranteed. A value is constructed at its final destination through any depth of calls. items.push(parse(line)) writes the parsed record straight into the array, and returning a growable array by value costs nothing.
• Returns across multiple calls. return err from load returns from a function any number of frames up, statically checked, with no unwinder, no Result type and no ? on every call.
• Threads that share nothing. A worker is compiled as a separate program with its own memory, and flat values cross typed queues as a memcpy. Data races, locks, atomics and memory orderings do not exist in the language.
• Generics and higher-order functions with no overhead. An untyped parameter is generic. Function values are compile-time entities, so xs.filter() { it > 0 } compiles to the loop it looks like and builds its result straight into its destination.
• Plain C in, plain C out. Goose compiles to one C file, so it runs wherever a C compiler does and calls C directly through extern fn. The bundled TinyCC backend compiles and runs a program in-process, with no build step.

The tutorial walks through all of this by example, the reference covers the whole language topic by topic, the specification has the exact rules, and the benchmarks have the numbers, losses included.

Features

This section focuses on what distinguishes Goose. The tutorial explains these features in more detail, and the samples show them in thirty-one complete programs.

One memory model: stacks, and scope exit is the free

A program has the native call stack, static data and N data stacks, where the compiler works out N. A data stack is a large address-space reservation with a bump pointer, and there is no other memory. At most one resizable value is live per stack and it is always on top, so growth never moves anything and never checks a capacity. All of this is proved at compile time; the runtime tracks only the bump pointers.

for round in 3 {
    var scratch: u8[>..] = [];                        // grow-only: growth is a pointer bump
    for i in 100000 { scratch.push((i % 256) as! u8); }
    print("round ", round, ": ", scratch.len, " bytes");
}                                                     // the free: one store to the stack top

Nothing moves, so references survive growth

struct Item { id: i32, weight: f32 }

var items: Item[>..] = [];
let first .= items.push(Item { 1, 0.5 });             // a reference to element 0
for i: i32 in 2..1000001 { items.push(Item { i, 0.0 }); }
first.weight = 99.5;                                  // still valid, a million pushes later

push returns a reference to the new element, so a program can link elements while building the array. A vector<T> or Vec<T> may reallocate and cannot provide this guarantee. Stable references account for several benchmark gains.

Safe references, no annotations

Every reference and slice has a static root: the variable that bounds its target's lifetime. A reference must not outlive the variable that owns its target or access that target through an incompatible type. The compiler infers roots and specializes functions for them, so no annotations are needed. There are no aliasing or exclusivity rules. When a lifetime check fails, the error names the reference's root and its destination:

fn longest(a: u8[:], b: u8[:]) -> u8[:] { if a.len >= b.len { a } else { b } }

var outer: u8[>..] = [];
var w = outer[..];
{
    var inner: u8[>..] = [];
    format(inner, "inner text");
    w = longest(outer, inner);     // error: storing a reference rooted at inner,
}                                  //        which does not outlive the destination (§9.2)

A slice T[:] lets a function accept a range from any array kind without copying its elements. For example, split returns slices into its input, and a dictionary keyed by u8[:] stores views of strings rather than copies.

Nested data stays inline

There is no single array type. There is a family that differs only in what happens to the size, and every member is [metadata][elements...], inline and packed, never a pointer to an element block:

Spelling What it is T[k] fixed size, known at compile time T[], T[varint] sized at construction, frozen after; a fixed or variable-width length T[..k], T[..] capacity inline; grows and shrinks within it T[>..] grow-only: the workhorse; references into it stay valid T[>..<] grow-shrink: stacks, queues, heaps

Strings are just u8 arrays: u8[>..] is a builder, u8[] a finished string stored inline in whatever holds it, u8[..16] a small string inside a struct, u8[:] a view. A struct may contain variable-size parts, and they sit inline in declaration order, so a record is a run of bytes with nothing indirect in it:

struct Item  { sku: u8[varint], qty: varint, cents: varint }
struct Order { id: varint, customer: u8[varint], items: Item[varint] }

var book: Order[>..] = [];
book.push(Order { id: 1001, customer: "alice",
                  items: [Item { sku: "SKU-441", qty: 2, cents: 1999 },
                          Item { sku: "SKU-7", qty: 1, cents: 500 }] });

That order is 29 bytes and 0 allocations. As C++ std::string + std::vector<Item> it is 160 bytes and 1 allocation, and as Rust String + Vec<Item> 153 bytes and 4. A whole order book is one array that streams through the cache. The price is that an array of variable-size elements is sequential: you can iterate it but not index it (unless you keep a second array of references into it as an index).

Enums in two sizes

Algebraic data types are the only dynamic polymorphism: no inheritance, no vtables. Every ADT can be stored two ways, chosen at the point of use. Fixed mode (Shape) is a tag plus room for the largest payload, indexable and overwritable. Variable mode (Shape..) gives each value exactly its variant's size; the array becomes sequential and a value never changes variant, and in exchange you may take references into a payload:

enum Shape { Circle { r: f64 }, Rect { w: f64, h: f64 }, Dot }

var packed: Shape..[>..] = [];                 // 9, 17 and 1 bytes, not 17 each
packed.push(Shape.Circle { r: 1.0 });
packed.push(Shape.Rect { w: 2.0, h: 3.0 });
packed.push(Shape.Dot);
for s in packed {
    match s { Rect &r => { r.w += 1.0; }, _ => {} }   // edits the payload in place, inside the array
}

Case functions provide another way to dispatch on a variant: define one overload per variant. Calls dispatch on the tag through a jump table, and the compiler checks that the overloads cover every variant:

fn area(s: Shape.Circle) -> f64 { 3.14159 * s.r * s.r }
fn area(s: Shape.Rect) -> f64   { s.w * s.h }
fn area(s: Shape.Dot) -> f64    { 0.0 }

for s in packed { print(s, " area ", area(s)); }

Links narrower than pointers

A T& is a machine address: eight bytes, never dangling. A relative reference is the same link stored as a narrow offset: T&<u32> is measured from the field itself to a target in the same array, so the structure is position independent, and T&<u32 in pool> from a named pool's base, so other arrays can link into the pool. T&<u32>? uses offset 0 as null. Here is a binary search tree in one grow-only array, with 12-byte nodes:

struct Node { key: i32, left: Node&<u32>?, right: Node&<u32>? }   // 12 bytes, links included

fn insert(pool: Node[>..]&, key: i32) {
    if pool.len == 0 { pool.push(Node { key: key }); return; }
    var cur .= pool[0];                             // .= binds a reference; = would copy the node
    loop {
        if key == cur.key { return; }
        let next = if key < cur.key { cur.left } else { cur.right };
        if next { cur .= next; continue; }          // narrowed by `if`: next is a Node&
        if key < cur.key { cur.left .= pool.push(Node { key: key }); }
        else             { cur.right .= pool.push(Node { key: key }); }
        return;
    }
}

References are transparent (no *, no ->), .= binds or retargets one, and Node? is a nullable reference that if, guard and assert narrow. The insert is one push and one store, and the push cannot invalidate cur. When lifetimes are not nested, a reusable pool pairs a grow-only array with a hidden freelist: alloc_ref reuses a slot or pushes, free hands one back, and since nothing is ever actually freed, a stale slot reads a different value of the correct type rather than corrupting memory.

Your data is already a file format

var image: u8[>..] = tree.to_bytes();           // a framed byte image of the whole array
var loaded, ok = from_bytes<Node[>..]>(image);  // verified before it becomes a value
image[image.len - 3] = 200;                     // tamper with a link ...
var bad, bok = from_bytes<Node[>..]>(image);    // ... and it is false and an empty array

The tree's links are self-relative, so it means the same thing wherever it sits: saving is writing its bytes and loading is reading them back, with no serializer, no schema and no pointer fixups. from_bytes checks the framing, every tag, every length and every link before the bytes become a value, so a corrupt or hostile file is a false, never a wild reference.

Built in place, always

The copy-free construction guarantee (spec §4.3) says a constructed value is always built in its final home, propagated top-down through calls. A function returning a grow-only array by value writes its elements straight into the caller's variable, or into a field of the record being built inside another array, so out-parameters mostly do not appear.

words.push(str("word", i));                    // formatted straight into the new element
let evens = xs.filter() { it % 2 == 0 };       // built straight into `evens`: no temporary
book.push(Order { id: 1001, customer: "alice", items: parse_items("SKU-441:2:1999;SKU-7:1:500") });

Returning errors across calls

struct User { name: u8[..16], age: i32 }

fn load(text: u8[:]) -> User[>..], u8[] {
    var users: User[>..] = [];
    each_split(text, '\n') { users.push(parse_user(it)); };
    return users, "";
}

fn parse_user(line: u8[:]) -> User {              // returns a User: no Result, no error parameter
    let comma = find(line, ",");
    guard comma >= 0 else { return [], str("expected name,age: ", line) from load; }
    let age, ok = parse_int(line[comma + 1..]);
    guard ok else { return [], str("bad age: ", line) from load; }
    let name = trim(line[..comma]);
    return User { name: name, age: age as i32 };
}

return E from f returns E as the result of the innermost active call of f, however many frames up, and every function in between keeps its plain signature. It is checked statically, so every call of parse_user must lie inside a load call and nothing is uncaught at runtime. It is a hidden discriminant checked per frame rather than an unwinder, and the message is built directly where load's caller wants it. The parsers in the samples use it for every syntax error.

Generics and function blocks

fn twice(x) { x + x }                              // an untyped parameter is a generic one

fn each_pair<T, F>(xs: T[:]) {                     // F is a function value: a compile-time entity
    for i in 0..xs.len - 1 { F(xs[i], xs[i + 1]); }
}

fn first_gap(xs: i64[:]) -> i64 {
    each_pair(xs) { a, b => if b - a > 1 { return a; } };   // returns from first_gap, not each_pair
    return -1;
}

let evens = xs.filter() { it % 2 == 0 };
let total = fold(xs, 0) { acc, x => acc + x };
sort(xs) { a, b => a > b };

Everything is monomorphized. Type arguments are inferred from argument types; parameters absent from those types can be supplied explicitly. Function values are passed as generic parameters: every call is direct and inlinable, they cannot escape, and there are no closure objects or function pointers, so a higher-order function compiles to exactly the loop it looks like. A block may return from its lexically enclosing function, and nested functions see the enclosing function's locals.

Threads that share nothing

thread_fn worker() {                                // a separate program with its own memory
    loop {
        let job = qget<Job>();                      // blocks on the typed queue for Job
        guard job.n >= 0 else { break; }            // -1 means stop
        qput(Result { n: job.n, digits: count_digits(job.n) });
    }
}

for i in n { ids.push(thread_spawn(worker)); }

There is no shared mutable memory. A thread_fn and everything it calls is compiled as its own program with its own data stacks and its own copies of the globals. Values cross through typed queues, one per type, and must be flat, with no references at any depth. That is cheap because a flat Goose value is contiguous: a job or a result carries real data, pixels included, and crossing is a memcpy.

A failed check or abort in a worker started with thread_spawn_contained ends only that worker. Its message goes to whoever waits for it, not to stderr, so a supervisor can log it and start another:

var why: u8[>..] = [];
while !thread_wait(thread_spawn_contained(serve, attempt), why) {
    log(why);                                       // "assert failed (serve.goose:12)"
    why.clear();
    attempt++;
}

The first thread_wait(id, msg) for a worker takes the message; a caller that doesn't ask for it loses it.

Plain C in, plain C out

extern fn hypot(x: f64, y: f64) -> f64;             // straight from libm
extern fn crc32_bytes(s: const u8[:]) -> u32;       // a slice crosses as { data, len }
extern fn stats_of(xs: i32[:], out: Stats&);        // a struct filled through a pointer

An extern fn binds a Goose signature to a C symbol. The math and os modules use this interface. Only types with supported C representations may cross the boundary; unsupported signatures are rejected at the declaration. The compiler emits one C file for the whole program. The bundled TinyCC backend can compile and run that C inside the Goose compiler's process.

Where the speed comes from

Much of the speed comes from the memory model: no heap allocation or per-object teardown, contiguous data, narrow links, and enums sized for the variant they contain. Every index is bounds-checked, and the compiler removes checks it can prove unnecessary. An assert on a slice length often lets a kernel eliminate its remaining checks and vectorize. --bce-lines reports the checks that remain. checkfree fn rejects a kernel if any bounds check survives; index types let graph edges retain validation done during construction. Benchmark measurements use whole-process wall-clock time, including teardown.

The summary explains the gains and losses; results.md has every measurement, and design.md explains what the suite tests.

What it costs you

• You think about where data lives: who owns this, and how long does its scope last. Usually the answer is "the function that builds it", and that is free; when it is not, it is a reusable pool.
• Recursive functions cannot keep growable data across a recursive call; scratch ends before the call, and what outlives it is passed in as a pool.
• Arrays of variable-size elements iterate but do not index (short of an index of references kept beside them), a fixed-mode enum cannot be pointed into, and a variable-mode one cannot be overwritten. You choose per container.
• No escaping closures, no function pointers, no dynamic dispatch beyond ADT tags, and whole-program compilation only.
• A slot handed back to a reusable pool and still named reads whatever its next owner put there: a logic bug, never memory corruption.

Tutorial §18 is the full list.

Install a release

Prebuilt compilers for Windows (x64), Linux (x64) and macOS (Apple silicon and Intel) are on the releases page, with the graphics, audio, ui and physics packages compiled in. Unpack one anywhere and put its directory on PATH: the compiler finds the standard library, TinyCC's runtime and the packages beside itself, and keeps what it builds under ~/.goose (GOOSE_HOME). The archive also holds these docs, the samples and the VS Code extension.

Running a program in-process (goose file.goose) needs nothing else on Windows. On Linux it needs the C library's headers (libc6-dev, or glibc-devel); on macOS, the SDK of the Xcode command line tools (xcode-select --install). goose build also needs a C compiler: Visual Studio 2022 or later, or clang, on Windows; gcc or clang elsewhere. The Linux compiler needs glibc 2.35 or later (Ubuntu 22.04, Debian 12), the macOS one macOS 13.3 or later. macOS keeps a downloaded archive's files from running until xattr -dr com.apple.quarantine is run on the unpacked directory.

Build and run

You need CMake 3.27 or later, a C++20 compiler (MSVC, clang or gcc) and Python 3 for the test and sample runners. TinyCC, which the in-process backend is built from, is part of the tree. The graphics, audio, ui and physics modules are packages in repositories of their own (goose_gfx and goose_physics; docs/design/packages.md): -DGOOSE_BUILTIN_LIBS=all fetches them at the pinned tags (git, at configure time) and compiles them into the compiler, so their programs run with nothing else installed. Without it the compiler builds alone, and a program importing one of them has it built into ~/.goose the first time, which needs git, cmake and a C compiler.

git clone https://github.com/aardappel/goose
cd goose
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGOOSE_BUILTIN_LIBS=all
cmake --build build --config Release

GOOSE_BUILTIN_LIBS can also name some of the packages (physics, or gfx, which carries audio and ui too). On Linux, SDL needs the X11 or Wayland development packages (third_party/SDL/docs/README-linux.md); without them the build leaves the gfx module out and says what to install.

The runner scripts prefer build/Release, then build/RelWithDebInfo, then build, then build/Debug. Pass --exe to select a specific compiler; use an explicit Debug executable when testing with compiler assertions enabled. The compiler is build/goose (build/Release/goose.exe with the Visual Studio generator), and it finds the standard library in the source tree it was built from. On Windows 10 version 1903 and later, its embedded manifest selects a UTF-8 process code page so compiler paths do not depend on the system locale. Earlier Windows versions retain their legacy code page. Generated native executables retain their own process settings.

Run a program straight from source, in-process (goose run says the same explicitly, and goose check only typechecks):

build/goose samples/01_tour.goose

Or build an executable, which lands beside the source (-o puts it elsewhere). The compiler finds a C compiler on the machine (goose toolchain says which: MSVC or clang on Windows, cc, gcc or clang elsewhere; --cc chooses) and links the generated C with the runtime and whatever native modules the program uses, graphics included. Programs that use threads need this route, since TinyCC cannot place thread-local storage.

build/goose build samples/01_tour.goose && samples/01_tour
build/goose build samples/27_gfx_cube.goose && samples/27_gfx_cube

--debug turns on the overflow, range and tag checks in the generated C, each abort naming its source line (and an overflow or a cast the values involved), and builds the C with debug information; --dry-run prints the commands a build would run, for a build system of your own; -v prints them as they run. Intermediates go under ~/.goose (GOOSE_HOME), the runtime object compiled once per C compiler. Useful flags of both forms: -O0/-O1/-O2 set the inlining level (-O2 by default), --bce-lines reports the bounds checks kept per line, and -DGS_DEBUG=1 is what --debug defines. Every resizable value lives on a data stack of 2 GB of reserved address space, committed as it is used: --stack-reserve 16G changes that for a program, --stacks reports how many data stacks the program and each of its workers can take and the cap that puts on hardware_threads(), and -DGS_STACK_STATS=1 makes a program report its actual use as it ends.

A program with C of its own names it in a sidecar beside the source, foo.build beside foo.goose: c_sources = ["impl.c"];, with c_include, c_defines, c_libs and _windows, _linux or _macos variants of each for one platform's part. Both forms build it: goose build links it in, and goose run builds it into a library the in-process run loads, so an extern fn reaches it either way. A library does the same with a sidecar beside its entry (or names the static library target of its own CMake project, cmake_target = "x";, which the compiler builds through cmake), and a program takes a library with import name from "source"; in its root file: a directory ("./lib"), or a git repository at a tag or commit ("owner/repo@v1" on GitHub, "host/owner/repo@ref"), fetched into ~/.goose and built there once per C compiler (docs/design/packages.md).

Underneath, goose -o tour.c tour.goose writes the C, which holds the program and the part of the runtime that inlines into it; the rest of the runtime (the OS layer, data stack reservation, threads, printing) is one more C file, which goose --emit-runtime goose_runtime.c writes, to be compiled once and linked with every program (cc -O2 tour.c goose_runtime.o -o tour -lm -pthread; on Windows cl /O2 tour.c goose_runtime.obj). Build the runtime with the compiler and flags the programs use; it takes no -D of its own, and a program from another version of the compiler fails to link against it. --standalone writes a C file that holds the whole runtime instead, and builds on its own. A program using audio, gfx, physics or ui also links that module's layer, which goose --audio-link msvc|cc, --gfx-link, --physics-link and --ui-link name in a response file (cc cube.c goose_runtime.o -o cube @$(goose --gfx-link cc), a ui program's before the gfx one it draws through). goose build --dry-run prints all of this for the program at hand.

A program using audio plays procedural mono/stereo PCM through SDL3 without opening a graphics window. See the audio reference.

The test suite and the samples run on Windows, macOS and Linux:

python test/run_tests.py
python samples/run_samples.py

For editing, the VS Code extension gives syntax highlighting, the compiler's language server (goose lsp: errors as you type, hover, go to definition, completion, rename, symbols) and one-key runs through the JIT:

code --install-extension vscode/goose-language.vsix

Other editors can run goose lsp themselves; editors has the configuration for Neovim, Helix and Emacs.

Documentation

• Documentation index: the map of everything below, and the root of the documentation site built from these files (how).
• Tutorial: the friendly introduction, by example. Read this first.
• Language reference: every feature, topic by topic, with a compiling example for each: the whole language, written to be read.
• Language specification: the exact rules, when you want to know why something did not compile.
• Samples: thirty-one complete programs in reading order, from a tour of the language to a JSON parser, a threaded Mandelbrot, a file tree built from two pools, a spinning cube on the GPU, thousands of boxes raining into a heap and a to-do list in windows of widgets.
• Standard library: fourteen modules, all readable Goose under stdlib/, including gfx, graphics on SDL3's GPU API (how it is built), physics, rigid body physics on Box3D (how it is built), and ui, fonts, text rendering and game HUDs, windows and widgets on Nuklear, drawn through gfx (how it is built).
• Building programs and using packages: goose build, libraries from git repositories or directories, a program's own C, and how to make a package of your own.
• Benchmarks: the numbers, with the full results and the design behind them.
• Implementation notes: how the compiler works, pass by pass, and how it is tested.

Status

Goose is new. What exists today is a whole-program compiler of about thirty thousand lines of C++, the specification, the standard library, the samples, and a test suite that CI runs on Windows, macOS and Linux with an extra sanitizer job. Deliberately out of scope for now: moves for resizable values, more than one resizable per struct, labeled break, namespace privacy, and more OS/library access. Open items are tracked in the specification's Appendix B.

License

Goose is licensed under the Apache License, Version 2.0.

Authors

Wouter van Oortmerssen: Language Design, Compiler Design, Coding standards. home page, twitter.

Claude Fable: Compiler implementation, Benchmarking, Sample & Doc writing.

History & Use of AI

This repo was produced almost entirely with AI, from a human language design. I had designed Goose several years ago, and had started to implement it, but running a game startup (which is built on another programming language of mine, Lobster) there was no time to finish it. Which was sad, because I knew Goose could do things other languages can't, and it should exist.

I had not expected AI to help with this until Fable came out. I thought it might be capable enough to do a good job. I made it essentially clone the style and structure of my other recent compiler (Lobster), which is why if you look at the code, it looks rather similar to that. My initial design had left lots of things unspecified, and lots of back and forth with Fable made me decide on all of those, and it is now a better language for it.

It is also an experiment in using AI to implement a new language, rather than reproduce an existing one.

You may wonder why I had it work in C++, if clearly I could have used any language, like Rust, or my own Lobster, or.. Goose itself (that may still happen). With Fable, it was my observation that it is almost equally capable in any language, and the strong guardrails of Rust or other languages are not as pressing as they once were. Since the compiler emits C, you likely already need to have a C/C++ compiler around, so sticking inside one ecosystem would seem to simplify deployment and adoption. I've also integrated libtcc and who knows what other C libraries in the future. And, like I said, to have the compiler code mimic my existing Lobster compiler seemed fun, at least I can read it like it is my own.

A memory-safe systems language with no heap allocator, garbage collector, or lifetime annotations. In this repository's benchmarks, Goose runs faster than C++ and Rust while using less memory.

Tutorial · Reference · Specification · Packages · Samples · Benchmarks · Standard library

Goose has syntax familiar to C and Rust programmers, but there is no heap. Every dynamic value lives inline on a data stack managed by the compiler. Growth advances a pointer, and memory is freed when its scope ends. The language's type and lifetime rules make this memory model safe to use.

Why Goose

• Faster than C++ and than safe Rust, while memory safe. Over sixteen benchmarks Goose runs at 3.3x the speed of idiomatic C++, 1.16x hand-optimized C++ and 1.12x the best safe Rust, on 1.9x, 1.3x and 1.2x less memory (results). These gains come from Goose's data layouts and memory model.
• No allocator, no GC, no reference counting, no destructors. Memory is a handful of data stacks that the compiler assigns statically. Freeing a million-element structure is one store, however deeply it nests.
• Growth preserves references. A reference into a growing array stays valid for as long as the array does. You keep typed references where C++ must reserve and safe Rust uses u32 indices in the corresponding benchmarks.
• Memory safe with zero annotations. No lifetime syntax, no aliasing or exclusivity rules, no unsafe. The compiler infers what every reference is rooted in and rejects references that could outlive their owners.
• Nested data stays inline. A string, an array of strings, a record with variable-size fields and an array of those records are each one contiguous block with no pointer in it. A record that is 160 bytes and an allocation in C++ is 29 bytes and none in Goose.
• Enums that cost what they hold. Variable-mode ADTs give each value its own variant's size rather than the largest one's: 4x less memory and 2x the speed of a Rust enum on the benchmark that exercises it.
• Links narrower than pointers. A relative reference stores a typed, checked link as a 1, 2 or 4-byte offset. Structures built from them are position independent, so your data structure is already its file format: saving is a write, loading is a read plus a verification pass that rejects hostile bytes.
• Everything is built in place, guaranteed. A value is constructed at its final destination through any depth of calls. items.push(parse(line)) writes the parsed record straight into the array, and returning a growable array by value costs nothing.
• Returns across multiple calls. return err from load returns from a function any number of frames up, statically checked, with no unwinder, no Result type and no ? on every call.
• Threads that share nothing. A worker is compiled as a separate program with its own memory, and flat values cross typed queues as a memcpy. Data races, locks, atomics and memory orderings do not exist in the language.
• Generics and higher-order functions with no overhead. An untyped parameter is generic. Function values are compile-time entities, so xs.filter() { it > 0 } compiles to the loop it looks like and builds its result straight into its destination.
• Plain C in, plain C out. Goose compiles to one C file, so it runs wherever a C compiler does and calls C directly through extern fn. The bundled TinyCC backend compiles and runs a program in-process, with no build step.

The tutorial walks through all of this by example, the reference covers the whole language topic by topic, the specification has the exact rules, and the benchmarks have the numbers, losses included.

Features

This section focuses on what distinguishes Goose. The tutorial explains these features in more detail, and the samples show them in thirty-one complete programs.

One memory model: stacks, and scope exit is the free

A program has the native call stack, static data and N data stacks, where the compiler works out N. A data stack is a large address-space reservation with a bump pointer, and there is no other memory. At most one resizable value is live per stack and it is always on top, so growth never moves anything and never checks a capacity. All of this is proved at compile time; the runtime tracks only the bump pointers.

for round in 3 {
    var scratch: u8[>..] = [];                        // grow-only: growth is a pointer bump
    for i in 100000 { scratch.push((i % 256) as! u8); }
    print("round ", round, ": ", scratch.len, " bytes");
}                                                     // the free: one store to the stack top

Nothing moves, so references survive growth

struct Item { id: i32, weight: f32 }

var items: Item[>..] = [];
let first .= items.push(Item { 1, 0.5 });             // a reference to element 0
for i: i32 in 2..1000001 { items.push(Item { i, 0.0 }); }
first.weight = 99.5;                                  // still valid, a million pushes later

push returns a reference to the new element, so a program can link elements while building the array. A vector<T> or Vec<T> may reallocate and cannot provide this guarantee. Stable references account for several benchmark gains.

Safe references, no annotations

Every reference and slice has a static root: the variable that bounds its target's lifetime. A reference must not outlive the variable that owns its target or access that target through an incompatible type. The compiler infers roots and specializes functions for them, so no annotations are needed. There are no aliasing or exclusivity rules. When a lifetime check fails, the error names the reference's root and its destination:

fn longest(a: u8[:], b: u8[:]) -> u8[:] { if a.len >= b.len { a } else { b } }

var outer: u8[>..] = [];
var w = outer[..];
{
    var inner: u8[>..] = [];
    format(inner, "inner text");
    w = longest(outer, inner);     // error: storing a reference rooted at inner,
}                                  //        which does not outlive the destination (§9.2)

A slice T[:] lets a function accept a range from any array kind without copying its elements. For example, split returns slices into its input, and a dictionary keyed by u8[:] stores views of strings rather than copies.

Nested data stays inline

There is no single array type. There is a family that differs only in what happens to the size, and every member is [metadata][elements...], inline and packed, never a pointer to an element block:

Spelling What it is T[k] fixed size, known at compile time T[], T[varint] sized at construction, frozen after; a fixed or variable-width length T[..k], T[..] capacity inline; grows and shrinks within it T[>..] grow-only: the workhorse; references into it stay valid T[>..<] grow-shrink: stacks, queues, heaps

Strings are just u8 arrays: u8[>..] is a builder, u8[] a finished string stored inline in whatever holds it, u8[..16] a small string inside a struct, u8[:] a view. A struct may contain variable-size parts, and they sit inline in declaration order, so a record is a run of bytes with nothing indirect in it:

struct Item  { sku: u8[varint], qty: varint, cents: varint }
struct Order { id: varint, customer: u8[varint], items: Item[varint] }

var book: Order[>..] = [];
book.push(Order { id: 1001, customer: "alice",
                  items: [Item { sku: "SKU-441", qty: 2, cents: 1999 },
                          Item { sku: "SKU-7", qty: 1, cents: 500 }] });

That order is 29 bytes and 0 allocations. As C++ std::string + std::vector<Item> it is 160 bytes and 1 allocation, and as Rust String + Vec<Item> 153 bytes and 4. A whole order book is one array that streams through the cache. The price is that an array of variable-size elements is sequential: you can iterate it but not index it (unless you keep a second array of references into it as an index).

Enums in two sizes

Algebraic data types are the only dynamic polymorphism: no inheritance, no vtables. Every ADT can be stored two ways, chosen at the point of use. Fixed mode (Shape) is a tag plus room for the largest payload, indexable and overwritable. Variable mode (Shape..) gives each value exactly its variant's size; the array becomes sequential and a value never changes variant, and in exchange you may take references into a payload:

enum Shape { Circle { r: f64 }, Rect { w: f64, h: f64 }, Dot }

var packed: Shape..[>..] = [];                 // 9, 17 and 1 bytes, not 17 each
packed.push(Shape.Circle { r: 1.0 });
packed.push(Shape.Rect { w: 2.0, h: 3.0 });
packed.push(Shape.Dot);
for s in packed {
    match s { Rect &r => { r.w += 1.0; }, _ => {} }   // edits the payload in place, inside the array
}

Case functions provide another way to dispatch on a variant: define one overload per variant. Calls dispatch on the tag through a jump table, and the compiler checks that the overloads cover every variant:

fn area(s: Shape.Circle) -> f64 { 3.14159 * s.r * s.r }
fn area(s: Shape.Rect) -> f64   { s.w * s.h }
fn area(s: Shape.Dot) -> f64    { 0.0 }

for s in packed { print(s, " area ", area(s)); }

Links narrower than pointers

A T& is a machine address: eight bytes, never dangling. A relative reference is the same link stored as a narrow offset: T&<u32> is measured from the field itself to a target in the same array, so the structure is position independent, and T&<u32 in pool> from a named pool's base, so other arrays can link into the pool. T&<u32>? uses offset 0 as null. Here is a binary search tree in one grow-only array, with 12-byte nodes:

struct Node { key: i32, left: Node&<u32>?, right: Node&<u32>? }   // 12 bytes, links included

fn insert(pool: Node[>..]&, key: i32) {
    if pool.len == 0 { pool.push(Node { key: key }); return; }
    var cur .= pool[0];                             // .= binds a reference; = would copy the node
    loop {
        if key == cur.key { return; }
        let next = if key < cur.key { cur.left } else { cur.right };
        if next { cur .= next; continue; }          // narrowed by `if`: next is a Node&
        if key < cur.key { cur.left .= pool.push(Node { key: key }); }
        else             { cur.right .= pool.push(Node { key: key }); }
        return;
    }
}

References are transparent (no *, no ->), .= binds or retargets one, and Node? is a nullable reference that if, guard and assert narrow. The insert is one push and one store, and the push cannot invalidate cur. When lifetimes are not nested, a reusable pool pairs a grow-only array with a hidden freelist: alloc_ref reuses a slot or pushes, free hands one back, and since nothing is ever actually freed, a stale slot reads a different value of the correct type rather than corrupting memory.

Your data is already a file format

var image: u8[>..] = tree.to_bytes();           // a framed byte image of the whole array
var loaded, ok = from_bytes<Node[>..]>(image);  // verified before it becomes a value
image[image.len - 3] = 200;                     // tamper with a link ...
var bad, bok = from_bytes<Node[>..]>(image);    // ... and it is false and an empty array

The tree's links are self-relative, so it means the same thing wherever it sits: saving is writing its bytes and loading is reading them back, with no serializer, no schema and no pointer fixups. from_bytes checks the framing, every tag, every length and every link before the bytes become a value, so a corrupt or hostile file is a false, never a wild reference.

Built in place, always

The copy-free construction guarantee (spec §4.3) says a constructed value is always built in its final home, propagated top-down through calls. A function returning a grow-only array by value writes its elements straight into the caller's variable, or into a field of the record being built inside another array, so out-parameters mostly do not appear.

words.push(str("word", i));                    // formatted straight into the new element
let evens = xs.filter() { it % 2 == 0 };       // built straight into `evens`: no temporary
book.push(Order { id: 1001, customer: "alice", items: parse_items("SKU-441:2:1999;SKU-7:1:500") });

Returning errors across calls

struct User { name: u8[..16], age: i32 }

fn load(text: u8[:]) -> User[>..], u8[] {
    var users: User[>..] = [];
    each_split(text, '\n') { users.push(parse_user(it)); };
    return users, "";
}

fn parse_user(line: u8[:]) -> User {              // returns a User: no Result, no error parameter
    let comma = find(line, ",");
    guard comma >= 0 else { return [], str("expected name,age: ", line) from load; }
    let age, ok = parse_int(line[comma + 1..]);
    guard ok else { return [], str("bad age: ", line) from load; }
    let name = trim(line[..comma]);
    return User { name: name, age: age as i32 };
}

return E from f returns E as the result of the innermost active call of f, however many frames up, and every function in between keeps its plain signature. It is checked statically, so every call of parse_user must lie inside a load call and nothing is uncaught at runtime. It is a hidden discriminant checked per frame rather than an unwinder, and the message is built directly where load's caller wants it. The parsers in the samples use it for every syntax error.

Generics and function blocks

fn twice(x) { x + x }                              // an untyped parameter is a generic one

fn each_pair<T, F>(xs: T[:]) {                     // F is a function value: a compile-time entity
    for i in 0..xs.len - 1 { F(xs[i], xs[i + 1]); }
}

fn first_gap(xs: i64[:]) -> i64 {
    each_pair(xs) { a, b => if b - a > 1 { return a; } };   // returns from first_gap, not each_pair
    return -1;
}

let evens = xs.filter() { it % 2 == 0 };
let total = fold(xs, 0) { acc, x => acc + x };
sort(xs) { a, b => a > b };

Everything is monomorphized. Type arguments are inferred from argument types; parameters absent from those types can be supplied explicitly. Function values are passed as generic parameters: every call is direct and inlinable, they cannot escape, and there are no closure objects or function pointers, so a higher-order function compiles to exactly the loop it looks like. A block may return from its lexically enclosing function, and nested functions see the enclosing function's locals.

Threads that share nothing

thread_fn worker() {                                // a separate program with its own memory
    loop {
        let job = qget<Job>();                      // blocks on the typed queue for Job
        guard job.n >= 0 else { break; }            // -1 means stop
        qput(Result { n: job.n, digits: count_digits(job.n) });
    }
}

for i in n { ids.push(thread_spawn(worker)); }

There is no shared mutable memory. A thread_fn and everything it calls is compiled as its own program with its own data stacks and its own copies of the globals. Values cross through typed queues, one per type, and must be flat, with no references at any depth. That is cheap because a flat Goose value is contiguous: a job or a result carries real data, pixels included, and crossing is a memcpy.

A failed check or abort in a worker started with thread_spawn_contained ends only that worker. Its message goes to whoever waits for it, not to stderr, so a supervisor can log it and start another:

var why: u8[>..] = [];
while !thread_wait(thread_spawn_contained(serve, attempt), why) {
    log(why);                                       // "assert failed (serve.goose:12)"
    why.clear();
    attempt++;
}

The first thread_wait(id, msg) for a worker takes the message; a caller that doesn't ask for it loses it.

Plain C in, plain C out

extern fn hypot(x: f64, y: f64) -> f64;             // straight from libm
extern fn crc32_bytes(s: const u8[:]) -> u32;       // a slice crosses as { data, len }
extern fn stats_of(xs: i32[:], out: Stats&);        // a struct filled through a pointer

An extern fn binds a Goose signature to a C symbol. The math and os modules use this interface. Only types with supported C representations may cross the boundary; unsupported signatures are rejected at the declaration. The compiler emits one C file for the whole program. The bundled TinyCC backend can compile and run that C inside the Goose compiler's process.

Where the speed comes from

Much of the speed comes from the memory model: no heap allocation or per-object teardown, contiguous data, narrow links, and enums sized for the variant they contain. Every index is bounds-checked, and the compiler removes checks it can prove unnecessary. An assert on a slice length often lets a kernel eliminate its remaining checks and vectorize. --bce-lines reports the checks that remain. checkfree fn rejects a kernel if any bounds check survives; index types let graph edges retain validation done during construction. Benchmark measurements use whole-process wall-clock time, including teardown.

The summary explains the gains and losses; results.md has every measurement, and design.md explains what the suite tests.

What it costs you

• You think about where data lives: who owns this, and how long does its scope last. Usually the answer is "the function that builds it", and that is free; when it is not, it is a reusable pool.
• Recursive functions cannot keep growable data across a recursive call; scratch ends before the call, and what outlives it is passed in as a pool.
• Arrays of variable-size elements iterate but do not index (short of an index of references kept beside them), a fixed-mode enum cannot be pointed into, and a variable-mode one cannot be overwritten. You choose per container.
• No escaping closures, no function pointers, no dynamic dispatch beyond ADT tags, and whole-program compilation only.
• A slot handed back to a reusable pool and still named reads whatever its next owner put there: a logic bug, never memory corruption.

Tutorial §18 is the full list.

Install a release

Prebuilt compilers for Windows (x64), Linux (x64) and macOS (Apple silicon and Intel) are on the releases page, with the graphics, audio, ui and physics packages compiled in. Unpack one anywhere and put its directory on PATH: the compiler finds the standard library, TinyCC's runtime and the packages beside itself, and keeps what it builds under ~/.goose (GOOSE_HOME). The archive also holds these docs, the samples and the VS Code extension.

Running a program in-process (goose file.goose) needs nothing else on Windows. On Linux it needs the C library's headers (libc6-dev, or glibc-devel); on macOS, the SDK of the Xcode command line tools (xcode-select --install). goose build also needs a C compiler: Visual Studio 2022 or later, or clang, on Windows; gcc or clang elsewhere. The Linux compiler needs glibc 2.35 or later (Ubuntu 22.04, Debian 12), the macOS one macOS 13.3 or later. macOS keeps a downloaded archive's files from running until xattr -dr com.apple.quarantine is run on the unpacked directory.

Build and run

You need CMake 3.27 or later, a C++20 compiler (MSVC, clang or gcc) and Python 3 for the test and sample runners. TinyCC, which the in-process backend is built from, is part of the tree. The graphics, audio, ui and physics modules are packages in repositories of their own (goose_gfx and goose_physics; docs/design/packages.md): -DGOOSE_BUILTIN_LIBS=all fetches them at the pinned tags (git, at configure time) and compiles them into the compiler, so their programs run with nothing else installed. Without it the compiler builds alone, and a program importing one of them has it built into ~/.goose the first time, which needs git, cmake and a C compiler.

git clone https://github.com/aardappel/goose
cd goose
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGOOSE_BUILTIN_LIBS=all
cmake --build build --config Release

GOOSE_BUILTIN_LIBS can also name some of the packages (physics, or gfx, which carries audio and ui too). On Linux, SDL needs the X11 or Wayland development packages (third_party/SDL/docs/README-linux.md); without them the build leaves the gfx module out and says what to install.

The runner scripts prefer build/Release, then build/RelWithDebInfo, then build, then build/Debug. Pass --exe to select a specific compiler; use an explicit Debug executable when testing with compiler assertions enabled. The compiler is build/goose (build/Release/goose.exe with the Visual Studio generator), and it finds the standard library in the source tree it was built from. On Windows 10 version 1903 and later, its embedded manifest selects a UTF-8 process code page so compiler paths do not depend on the system locale. Earlier Windows versions retain their legacy code page. Generated native executables retain their own process settings.

Run a program straight from source, in-process (goose run says the same explicitly, and goose check only typechecks):

build/goose samples/01_tour.goose

Or build an executable, which lands beside the source (-o puts it elsewhere). The compiler finds a C compiler on the machine (goose toolchain says which: MSVC or clang on Windows, cc, gcc or clang elsewhere; --cc chooses) and links the generated C with the runtime and whatever native modules the program uses, graphics included. Programs that use threads need this route, since TinyCC cannot place thread-local storage.

build/goose build samples/01_tour.goose && samples/01_tour
build/goose build samples/27_gfx_cube.goose && samples/27_gfx_cube

--debug turns on the overflow, range and tag checks in the generated C, each abort naming its source line (and an overflow or a cast the values involved), and builds the C with debug information; --dry-run prints the commands a build would run, for a build system of your own; -v prints them as they run. Intermediates go under ~/.goose (GOOSE_HOME), the runtime object compiled once per C compiler. Useful flags of both forms: -O0/-O1/-O2 set the inlining level (-O2 by default), --bce-lines reports the bounds checks kept per line, and -DGS_DEBUG=1 is what --debug defines. Every resizable value lives on a data stack of 2 GB of reserved address space, committed as it is used: --stack-reserve 16G changes that for a program, --stacks reports how many data stacks the program and each of its workers can take and the cap that puts on hardware_threads(), and -DGS_STACK_STATS=1 makes a program report its actual use as it ends.

A program with C of its own names it in a sidecar beside the source, foo.build beside foo.goose: c_sources = ["impl.c"];, with c_include, c_defines, c_libs and _windows, _linux or _macos variants of each for one platform's part. Both forms build it: goose build links it in, and goose run builds it into a library the in-process run loads, so an extern fn reaches it either way. A library does the same with a sidecar beside its entry (or names the static library target of its own CMake project, cmake_target = "x";, which the compiler builds through cmake), and a program takes a library with import name from "source"; in its root file: a directory ("./lib"), or a git repository at a tag or commit ("owner/repo@v1" on GitHub, "host/owner/repo@ref"), fetched into ~/.goose and built there once per C compiler (docs/design/packages.md).

Underneath, goose -o tour.c tour.goose writes the C, which holds the program and the part of the runtime that inlines into it; the rest of the runtime (the OS layer, data stack reservation, threads, printing) is one more C file, which goose --emit-runtime goose_runtime.c writes, to be compiled once and linked with every program (cc -O2 tour.c goose_runtime.o -o tour -lm -pthread; on Windows cl /O2 tour.c goose_runtime.obj). Build the runtime with the compiler and flags the programs use; it takes no -D of its own, and a program from another version of the compiler fails to link against it. --standalone writes a C file that holds the whole runtime instead, and builds on its own. A program using audio, gfx, physics or ui also links that module's layer, which goose --audio-link msvc|cc, --gfx-link, --physics-link and --ui-link name in a response file (cc cube.c goose_runtime.o -o cube @$(goose --gfx-link cc), a ui program's before the gfx one it draws through). goose build --dry-run prints all of this for the program at hand.

A program using audio plays procedural mono/stereo PCM through SDL3 without opening a graphics window. See the audio reference.

The test suite and the samples run on Windows, macOS and Linux:

python test/run_tests.py
python samples/run_samples.py

For editing, the VS Code extension gives syntax highlighting, the compiler's language server (goose lsp: errors as you type, hover, go to definition, completion, rename, symbols) and one-key runs through the JIT:

code --install-extension vscode/goose-language.vsix

Other editors can run goose lsp themselves; editors has the configuration for Neovim, Helix and Emacs.

Documentation

• Documentation index: the map of everything below, and the root of the documentation site built from these files (how).
• Tutorial: the friendly introduction, by example. Read this first.
• Language reference: every feature, topic by topic, with a compiling example for each: the whole language, written to be read.
• Language specification: the exact rules, when you want to know why something did not compile.
• Samples: thirty-one complete programs in reading order, from a tour of the language to a JSON parser, a threaded Mandelbrot, a file tree built from two pools, a spinning cube on the GPU, thousands of boxes raining into a heap and a to-do list in windows of widgets.
• Standard library: fourteen modules, all readable Goose under stdlib/, including gfx, graphics on SDL3's GPU API (how it is built), physics, rigid body physics on Box3D (how it is built), and ui, fonts, text rendering and game HUDs, windows and widgets on Nuklear, drawn through gfx (how it is built).
• Building programs and using packages: goose build, libraries from git repositories or directories, a program's own C, and how to make a package of your own.
• Benchmarks: the numbers, with the full results and the design behind them.
• Implementation notes: how the compiler works, pass by pass, and how it is tested.

Status

Goose is new. What exists today is a whole-program compiler of about thirty thousand lines of C++, the specification, the standard library, the samples, and a test suite that CI runs on Windows, macOS and Linux with an extra sanitizer job. Deliberately out of scope for now: moves for resizable values, more than one resizable per struct, labeled break, namespace privacy, and more OS/library access. Open items are tracked in the specification's Appendix B.

License

Goose is licensed under the Apache License, Version 2.0.

History & Use of AI

This repo was produced almost entirely with AI, from a human language design. I had designed Goose several years ago, and had started to implement it, but running a game startup (which is built on another programming language of mine, Lobster) there was no time to finish it. Which was sad, because I knew Goose could do things other languages can't, and it should exist.

I had not expected AI to help with this until Fable came out. I thought it might be capable enough to do a good job. I made it essentially clone the style and structure of my other recent compiler (Lobster), which is why if you look at the code, it looks rather similar to that. My initial design had left lots of things unspecified, and lots of back and forth with Fable made me decide on all of those, and it is now a better language for it.

It is also an experiment in using AI to implement a new language, rather than reproduce an existing one.

You may wonder why I had it work in C++, if clearly I could have used any language, like Rust, or my own Lobster, or.. Goose itself (that may still happen). With Fable, it was my observation that it is almost equally capable in any language, and the strong guardrails of Rust or other languages are not as pressing as they once were. Since the compiler emits C, you likely already need to have a C/C++ compiler around, so sticking inside one ecosystem would seem to simplify deployment and adoption. I've also integrated libtcc and who knows what other C libraries in the future. And, like I said, to have the compiler code mimic my existing Lobster compiler seemed fun, at least I can read it like it is my own.

My Implementing a Forth article got some great feedback, particularly around the subject of tiny Forth implementations. (And it was an excuse to list some of the tiniest Forths I’ve seen.)

I know I’m not alone in seeing the appeal of tiny Forths, tiny languages, tiny programs, and just small stuff in general. What’s up with that, anyway?

The appeal of tiny programs

Tiny programs are approachable.

If I see that something is 200 lines of code, I’m much more likely to read it than if it’s 2,000 or 20,000 lines.

What’s more, if it’s a program that does something interesting in 20 lines when I was expecting 2,000, then I’m going to be very, very interested to read those 20 lines!

When I was researching DOM diffing libraries prior to creating RetroV, one of the repos I looked at was ijk by Luke Jackson.

I was pretty used to looking at fat JavaScript libraries at that point, so when I cracked open the source file for ijk, I was really surprised. Here’s the source: index.js (githubusercontent.com).

Just 25 lines! I was instantly intrigued.

More recently, one of the responses to my aforementioned Forth article was Philippe Brochard with a 46 byte "Forth".

46 bytes!

It’s so small, I can just put the hexdump right here:

50b8 8e00 31d8 e8ff 0017 003c 0575 00ea
5000 3c00 7401 eb02 e8ee 0005 0588 eb47
b8e6 0200 d231 14cd e480 7580 c3f4

(See my update to Implementing a Forth for more about this artifact.)

Sure, very few people jump at a chance to read raw machine code. But that’s exactly my point. Something this small doesn’t feel so scary, right? I mean, 46 bytes. If you put the effort into it, you know you can figure out how it works eventually.

This next one almost serves as a counter-example because the code is so dense and frightening, but I can’t help mentioning it anyway.

One of my favorite software stories is Roger K.W. Hui’s Appendix A "Incunabulum" from An Implementation of J in which he describes Arthur Whitney writing an array-based language interpreter,

"…​on one page and in one afternoon — an interpreter fragment on the AT&T 3B1 computer. I studied this interpreter for about a week for its organization and programming style…​"

Hui can list the entire interpreter in the appendix because it’s just that single page, 122 lines of incredibly cryptic C, which you can view here:

https://code.jsoftware.com/wiki/Essays/Incunabulum

I haven’t personally tackled that puzzle yet, but it’s in my unwritten list of dense computer things that I would print out and take with me if I suddenly found myself cast out of society and forced to go live alone up on a mountain or something.

(By the way, J is in the family of array languages which include APL. If you enjoy brevity, cryptic programming puzzles, and high performance numerical computing, these languages have got you covered!)

Update July 2026: Speaking of Arthur Whitney, Alex (alexalejandre.com) sent me a link that I knew I’d have to add here. Check this out: https://fall.tr/kos

It’s a transcript of a Geo Carncross talk describing an operating system, kOS, featuring Whitney’s k array language, kdb database, and a GUI and window manager in a 90 Kb bootable image. Individual source code stats are just as impressive (such as 60 lines of C for the aforementioned GUI and window manager). And apparently kdb is 100 lines of C? Is that right? Even Whitney C, that’s still impressive. Now I’m even more intrigued by kdb than ever! And if any of that seems interesting, you’ll definitely enjoy the article Impending kOS (vector.org.uk).

Something else that would certainly need to be in the collection I take to that lonely mountain would have to be The Most Beautiful Program Ever Written, according to the title of a talk by William Byrd. Spoiler alert: It’s this Lisp interpreter in Lisp:

(define eval-expr
  (lambda (expr env)
    (pmatch expr
      [`,x (guard (symbol? x))
        (env x)]
      [`(lambda (,x) ,body)
        (lambda (arg)
          (eval-expr body (lambda (y)
                            (if (eq? x y)
                                arg
                                (env y)))))]
      [`(,rator ,rand)
       ((eval-expr rator env)
        (eval-expr rand env))])))

(The above is from Alberto Zaccagni’s transcription of the code (github.com) shown in Byrd’s slide.)

Recreation or fundamental truths?

Now, we all know that code golfing (wikipedia.org) is a programmer’s recreational game, and just for fun, right?

But is any of this actually useful?

I say yes. And I’ll go one step further: They are meaningful.

When you see a ray tracer that fits on a business card (fabiensanglard.net), it makes you realize that a ray tracer is something that can fit on a business card!

That’s an actual fact about the minimum complexity of ray tracers.

Kolmogorov complexity (wikipedia.org) is:

"…​the length of a shortest computer program (in a predetermined programming language) that produces the object as output."

(I learned about Kolmogorov complexity in a book about Kurt Gödel and it was my favorite thing in what was otherwise a pretty dry and depressing book.)

The meaning I get from this is that code golfers, for the benefit of the whole human race, produce versions of programs that represent the minimum complexity of those programs. Way to go code golfers!

What do we know about the fundamental complexity of the Lisp programming language?

Thanks to Justine Tunney, we know that a LISP with garbage collection can be as small as 436 bytes (justine.lol).

That’s a fact about Lisps with garbage collection. When you strip everything else away, you can get the essentials down to 436 bytes!

I take comfort in this because that means the concept can be boiled down to this hand-held size. And that means I can probably fit it in my head.

Thanks to Frank Force, we know that a 3D browser game can fit in 2Kb of (insanely compressed) JavaScript source: A 3D game in 2Kb of JS (frankforce.com)

Martin Kleppe’s whole site (aem1k.com) is filled with wonders you can run right in your browser.

And we haven’t even scraped the surface of what the demoscene has to offer for computers going back to the 1980s. Here’s a good place to start: https://demoscene.assembly.org/

Mini-things are fun and useful and meaningful.

My point here is that you don’t have to actually study the intricate clockwork puzzle of a miniaturized/golfed program to benefit from it. The miniature simply serves to prove that the concept can fit on a business card and it can fit in your head. Go ahead and find a readable version to study instead. (The miniature will likely have many things to teach, but some of them will be about how to make miniature programs.)

Small languages

Small programs are interesting. But I’m also interested in small programming languages.

Generally speaking, the smaller the language, the less expressive it is.

One of the most well-known, least expressive languages is assembly.

Assembly languages are very syntactically and conceptually simple. There’s not much language at all: it’s mostly a rigid sequence of opcode mnemonics. Even with macros, writing in assembly is mostly about understanding the puzzle of a particular CPU’s instruction set architecture, or ISA. (You can prove this by comparing the syntax of assembly for CISC and RISC processors. The puzzle is totally different, while the language remains practically unchanged.)

An honorable mention must be made here for SNOBOL, which is kind of like assembly language both for the rigid syntactic reasons listed above and also because it presents a similar control flow challenge. Like a CPU, it only understands jumps and call/return. (In every other way, SNOBOL is wildly different, being a string matching and replacing language. It’s weird and fun and highly effective.)

Moving on to syntactically tiny "high-level" languages, these three are brilliant:

• Forths win just about any "smallness" contest. The only syntactical construct is the space character. Forths are mind-bendingly flexible and powerful in a way that is downright disturbing.
• Lisps, (especially Scheme?), have small core languages with very simple syntax and enormous expressive power and flexibility.
• Tcl deserves to be mentioned with Forth and Lisp. Tcl is wild because everything is a string in a way that is hard to appreciate at first. You can construct your own language in Tcl just as you can with Forth or Lisp.

Forth, Lisp, and Tcl only require you to understand a few core ideas and give you limitless power in return. You could fit the syntactic core of these languages on an index card or less.

And yet, to wield these tools effectively requires a massive shift in thinking if you’re used to "normal" Algol-like procedural languages.

So how about something that is both small and simple and doesn’t require a blood sacrifice?

I think the Lua programming language fits the bill. Check out how small the core language is:

https://lua.org/manual/5.1/

I’ve got the printed book version of this document and it’s a thin book! The Lua language is covered in the beginning and it is just 27 pages. 27 pages! I have game manuals longer than that.

The C language is pretty small. I don’t think it’s super easy to program effectively and safely with C, but there’s no denying it’s compact and pretty expressive for its size.

Newcomer Zig is C-like. But I think it’s actually quite large and you need to know a lot of it to use it effectively. (I’m also a fan of Zig and think it’s totally worth it. The size is there for a reason.)

JavaScript has a core language that is quite small. Probably on par with Lua, actually. Functions, arrays, and objects (used as dictionaries) are pretty much all the JS I use and I like it that way!

As with the natural human languages, the trade-off with a bigger programming language is, hopefully, greater expressivity.

Small or expressive. Which way should you lean?

In a wonderful talk by David Ungar (wikipedia.org) titled Self and Self: Whys and Wherefores, he laments the mistake of Self’s multiple inheritance:

"It’s far better to make a system a little too simple so a person has to write extra stuff, even write it twice with a comment that says, 'Look over here,' than to have a system that acts in strange ways and nobody can understand why. So, simplicity trumps expressiveness."

Got that? Here’s the bullet from the slide to help you remember:

snippet of a slide that reads Simplicity trumps expressiveness

I think small languages can also be fairly expressive, but only with very, very careful design. Recall Forth, Lisp, and Tcl.

Microworlds: The idea of small, self-contained, learnable programming environments also has a David Ungar connection I discovered while watching his talk. See "ARK" on my Microworlds card. (Smith and Ungar co-created the Self language (selflanguage.org), which is a notable milestone in programming and still under active development to this day.)

Languages which come with full-featured interactive IDEs are obvious potential microworlds.

But I also think that traditional text file-based programming can give you the microworld experience as well. For that, a language needs high quality documentation that installs alongside the compiler/interpreter. And it should strive to have error output that can be understood by beginners, including explanations of what went wrong and hints for how to fix it or or where to find more information.

Libraries: Another easily overlooked aspect of a programming language’s apparent size is its "standard library" of functions, data structures, and methods.

I think function libraries are subject to the exact same size versus expressiveness challenge.

For example, learning the Ramda (ramdajs.com) functional programming library for JavaScript was very much like learning a new programming language and I never managed to learn all of it. The composability of Ramda and the consistency of its curried functions was extremely eye-opening to me. My advice: A little bit of Ramda goes a long way!

The feels: Concision in our tools has a certain quality that goes beyond utility. Little things just spark joy, you know?

Why are small things so delightful?

The appeal of tiny things in general

I’ve always had a fascination with miniatures, dioramas, and tiny scale models.

I know I’m not alone!

Stephanie M. Langin-Hooper, author of Figurines in Hellenistic Babylonia writes in Tiny Things and Why We Love Them (cambridgeblog.org):

"…​the act of shrinking a life-size thing to a smaller scale has an influential effect. The object is now cute, more accessible, more delicate, more 'squee'. It is also more helpless, and thus controllable – comforting and completely non-threatening due to its diminutive size.

Which is exactly the reaction I have when I see a tiny bit of code that does something useful.

Also, I’m super okay with "controllable" in the context of a computer, but the feeling of that paragraph changes a bit if we’re talking about, say, dolls.

Small scale helps make "big" things approachable:

"What might be threatening to think about in the real-scale world can be more manageable and more conceivable in miniature."

Small things, "allow low-stakes experimentation," which is why we often build tiny models of things before we scale them up.

Simon Garfield, author of In Miniature: How Small Things Illuminate the World has many of the same answers. He writes in Small world: Why we love tiny things (theguardian.com):

"The answer lies in our desire for mastery and elucidation. The ability to enhance a life by bringing scaled-down order and illumination to an otherwise chaotic world – a world over which we may otherwise feel we have little control – cannot be overvalued. […​] At its simplest, the miniature shows us how to see, learn and appreciate more with less."

I think this is a big part of what I get out of Assembly Nights with the small scale and orderly logic and internal consistency. Unlike the vast, fathomless Real World, an offline computer is a fantasy place where I can believe I could understand everything if I just tried hard enough for long enough.

"The satisfaction of observing small things becomes a desire to make small things, and both stages address the human need for comprehension and order. We live in a huge and doomy universe, and controlling just a tiny scaled-down part of it restores our sense of order and worth."

I’ve been studying computer history for a little while and I’ve only scratched the surface. But one of the things I’ve been finding is that a lot of the "big" stuff in our field are just accumulations of lots of "little" victories as individuals have solved "little" problems in new and different ways.

"We create small universes in which we may bury ourselves to the exclusion of all else. Blocking out real life for a while – always the prerequisite of the dedicated domestic hobby, from doll’s house modeller to jigsaw enthusiast to adult book colourist – may be contemplative, meditative, blinkered and essential. The people crouching over tiny details as if the world depended on it are only doing it because their world does depend on it."

Back to pl-design.

Today, we're releasing Dream-1, a World Action Model that controls robots it has never seen, performs tasks it was never shown, and runs from $3/hour.

Give Dream-1 a new bimanual robot with parallel grippers, connect it to our API, and it can start performing tasks out of the box. No fine-tuning, no demonstrations from that robot, and no machine learning expertise required.

This is an important step toward general-purpose robot intelligence.

Today, Dream Labs is also coming out of stealth with $90M in seed funding and making Dream-1 available to people all around the world through a public API.

Dream-1 controls robots it has never seen, on tasks it was never shown, in places it has never been, from $3/hour.

Breaking the chicken-and-egg problem

Robotics' chicken-and-egg problem.

For decades, robotics has been stuck in a chicken-and-egg problem: a robot needs to be useful before people want to buy it, but making it useful requires data from that exact robot, and that data only exists once enough people start using it. Dream-1 is our first step toward breaking that loop.

Self-driving found a way around this cold-start problem. Cars were useful from day one with zero autonomy, so millions of people drove them every day and generated the data needed to train increasingly capable models. Most other robots never had that advantage.

Two things are changing that.

First, robot hardware is getting cheap enough to reach far more people. A capable bimanual arm setup can now cost as little as $2k1.

Second, a robot may no longer need a large dataset of its own before it can be useful. If a model can generalize zero-shot to unseen environments, tasks, and even robots, a new robot can start with useful capabilities.

Put those together and the flywheel can finally start. Cheaper hardware puts more robots into the world. Generalizable models make those robots useful earlier. Useful robots generate more data, and that data makes the models better for the next robot.

The Cambrian explosion of cheap, affordable robots

Robot hardware is getting cheap faster than anyone expected. See the robots above, where 7 semi-humanoid robots cost less than $15k, a price tier that barely existed a year ago. At the far end of the price curve, Hugging Face's $399 Microduck sold more than 10,000 units within 4 days of its launch in late August2. We believe the robotics take-off will start with robots like these.

That's because their buyers are consumers, prosumers, and small teams, not enterprises. An enterprise buyer is making a spreadsheet decision. If the robot doesn't beat a person on cost, speed, or reliability, it doesn't get bought; that's a high bar to clear. Consumers are different. 10,000+ people just paid $399 within 4 days for a robot whose main job is to be delightful and cute: it walks, waddles, and roller-skates. Somewhere between that and a fully capable home robot is a sweet spot, a robot affordable enough to be an easy yes and useful enough to earn its place in the home. Once robots hit that point, we think adoption starts to compound, and every robot in a home becomes a source of new training data. Imagine two Microducks folding your laundry, using their beaks as grippers.

The result that started Dream Labs

In February 2026, our team released DreamZero, the paper that introduced World Action Models3. Since then, the community has adopted WAMs as one of the most promising paradigms in robot learning. At that time, the most surprising result for us was when after post-training DreamZero on just 30 minutes of YAM robot data, we saw it generalize zero-shot to tasks it had never seen. That kind of generalization used to take hundreds or even thousands of hours of embodiment-specific data. If 30 minutes of data from a given robot was enough to transfer general-purpose task knowledge to that robot, breaking the chicken-and-egg loop no longer looked impossible. Combined with the hardware wave above, the timing felt right. That's when we decided to start Dream Labs.

After six months of building, we're excited to share a new research breakthrough that was only made possible through world action models: early signs of zero-shot embodiment generalization.

Introducing Dream-1

Dream-1 is our first World Action Model available through a public API that can be used all around the world. Out of the box, it can control unseen robots for unseen tasks in unseen environments.

Unseen Tasks Dream-1 was able to do. Autonomous. 8x speed at the beta testers’ sites, 4x in-house.

Get started in minutes

We want generalizable robotics to be accessible to anyone with a robot, not just large labs. So we designed Dream-1 to be as close to plug-and-play as possible. Connect your robot, point it at the API, and run. No machine learning background is needed to get started. Every week, we run internal hackathons to keep time-to-deploy as short as possible. Getting started tutorial

import dream

client = dream.DreamClient(
    api_key=<API_KEY>,
)
client.policy({"checkpoint": "dreamlabs/dream-1"})
client.set_observation(observation)
action = client.get_action()
client.close()

Condensed client side code

Zero-shot on robots it has never seen

Dream-1 can zero-shot generalize to many bimanual, parallel-gripper robots. Here are some we've tested. It can even control the original PR2, a general-purpose bimanual research robot released 18 years ago. Note that no embodiment-specific data was used to get these rollouts.

Unseen Robots Dream-1 was able to control. 4x speed on real robots, 3x in simulation.

This is one of the first reported zero-shot generalizations to completely unseen robot embodiments.4 If you have a bimanual parallel-gripper robot, try prompting Dream-1 and let us know how it goes. Zero-shot performance depends partly on your robot's inverse kinematics (IK) solver, so we've written a guide to help you get the most out of it. IK guide. Performance also varies across robots. Below, we break down zero-shot performance in simulation and on real robots.

Dream-1 zero-shot performance on 27 unseen robots in simulation and 4 real robots, relative to YAM, the robot Dream-1 was actually trained on. The real robots ran earlier Dream-1 checkpoints that never trained on them, each compared with the same checkpoint on YAM; OpenYAM was tested on unseen tasks only. Simulated evaluations are conducted by doing 300 rollouts per embodiment: 150 rollouts of 50 tasks quantifying language following and 150 rollouts of 150 tasks quantifying unseen task generalization.

Below, we explain how zero-shot embodiment generalization works and where it falls short. We then show how one-shot embodiment learning closes that gap.

How zero-shot embodiment generalization works

Two ways to control a robot: joints or end-effectors. Robot policies commonly output joint angles, which ties them to one specific robot's body. Dream-1 can do the same for the embodiments it supports natively, but it can also predict end-effector (EEF) poses: where each gripper should be and whether it should be open or closed. Every robot with two arms and parallel grippers shares that EEF action space, and its own IK solver turns the poses into joint commands. That shared interface gives Dream-1 a common way to command robots it has never seen, but on its own it isn't enough to make zero-shot transfer work.

Why does this work? Dream-1 sees the world through three cameras: a head view and a wrist view on each arm. Prior work on the Universal Manipulation Interface (UMI) showed that policies can transfer zero-shot across robot arms when every arm uses the same gripper and the policy relies only on wrist cameras5. From the wrist's point of view, the robots look identical. But that shortcut has a hard limit. It only works when the gripper hardware matches exactly, and it gives up the head camera's view of the whole scene.

World Action Models remove that limit. A WAM learns to predict video and actions together, so controlling a robot and imagining what that robot will do are the same problem. And every robot shows up the same way: as pixels. The head camera sees a UMI gripper, a two-finger gripper or an arm Dream-1 has never met as nothing more than shapes moving through a scene, which makes video the one representation every robot type shares. If Dream-1 can predict what a robot will look like as it completes a task, it can produce the actions that get it there. That’s why it can control robots with grippers that look nothing like UMI’s, and even entirely new bodies seen through the head camera.

How a World Action Model works.

Limitations

Predicting EEF trades some control for generality. Compared with joint-angle prediction with per-embodiment adaptation:

• Performance depends on the quality of each robot's IK solver.
• Dream-1 doesn't control how the arm reaches a pose, only where the gripper ends up, which can matter around obstacles or near joint limits.
• Robots whose workspace or kinematics / dynamics differ sharply from what Dream-1 has seen may predict future states that are infeasible.

What Dream-1 can and can't do

Robocurve’s independent report on Dream-1: mean score per task attempt on RoboDojo-RC Tier 1.

Ahead of launch, Robocurve6, a Public Benefit Corporation that measures and reports frontier robotics capabilities to the public, evaluated Dream-1 independently. Their report, above, shows Dream-1 is better, faster and cheaper: a new Pareto frontier for robot control. It scores above the next best models, Opus 5.5 and Gemini 4 Argon, at 16x lower cost per task attempt than Gemini 4 Argon and 1.7x faster than the fastest LLM, GPT-6 Astra. While current LLMs excel at planning, robots still require a fast, general System 1 to act, and that is what Dream-1 is built to be.

How much does it cost to run Dream-1? We want you to know exactly what to expect before you try it. Running the model costs just $3 per hour for Dream-1, or $2 per hour for Dream-1-flash. Each call to the model returns 1.6 seconds of actions, so an hour is 2,250 calls: about $0.0013 an action for Dream-1, and $0.0009 for Dream-1-flash.

Across tasks. We break Dream-1’s performance down into eight task categories, from what it handles well to where it still struggles.

Fine-tune for your task with FaaS. If Dream-1 falls short on a task you care about, you don’t have to wait for our next release. Collect some data on your own robot, run it through FaaS, and get a custom model built for that task.

We want anyone with a robot to work like a forward-deployed engineer: find a task someone needs done, collect a bit of data, adapt Dream-1 to it, and deploy. If people will pay for what your robot does, you keep everything above the hourly cost.

Dream-1 is a research beta

It will make mistakes, and it has not been validated for safety-critical use. Please test it in controlled settings, and take extra care whenever a robot is running near people. If you see unsafe behavior, please report it to safety@dreamlabs.ai.

One-shot embodiment learning

When zero-shot isn't enough, we fine-tune Dream-1 for your robot. We call this Fine-tuning-as-a-Service (FaaS).

Here's what that looks like in practice: timelapses of data collection on two of our partners' robots, 5 minutes each.

Five minutes of data collection on two robots at 8x: reBot B601-DM on the left, Maker Arm on the right.

After adapting on just those 5 minutes, the difference shows side-by-side with the zero-shot run. Dream-1 now understands a bit better the affordance and kinematics of the robot.

Zero-shot and after FaaS on the same instruction, “go ahead and put the banana in the bowl”, with five minutes of the robot’s own data in between: reBot B601-DM in its simulated twin, Maker Arm on the real robot. 3x speed.

Some embodiments sit much further from what Dream-1 already knows, and closing a gap that large takes more of the robot’s own data. Dexmate with Sharpa dexterous hands grasps with hands in place of grippers, and took 77 minutes of its data. On YAM, we swapped in different end effectors for this task, a compliant gripper and a modified knife gripper, and it took 98 minutes of data.

After FaaS on embodiments far from Dream-1’s training data, on the real robots. 5x speed.

Across twelve robots from our FaaS beta partners, fine-tuning lifts the language-following score on ten of them:

Zero-shot versus after FaaS on the same twelve robots, in simulation, relative to YAM.

One-shot adaptation still outperforms zero-shot, so a gap remains. Closing it is one of our main goals: the better zero-shot gets, the closer any new robot comes to working out of the box.

Looking forward

Dream-1 today is a manipulation policy for bimanual robots with parallel grippers. It’s a starting point. We believe a strong, general robot foundation model can extend to mobile manipulators, dexterous hands, and bodies we haven't imagined yet, opening up many more of the things people actually want robots to do. Robotics won't be solved on day one, and Dream-1 has real limits, which we've tried to show transparently above.

Robotics is still stuck in a difficult loop: every new robot or task demands new data and a custom policy, often with limited generalization. With Dream-1 as a robotics API, we want to break that loop and put robot intelligence into more people's hands, so everyday users, not only researchers, can discover new applications for general-purpose robot intelligence.

Notes and references

1. MakerMods, Maker Arm: a bimanual arm setup from about $2k.
2. Yahoo Tech, Hugging Face's $399 Microduck robot has already sold more than 10,000 units, 1 September 2026: more than 10,000 units since pre-orders opened on 28 August.
3. DreamZero: the paper that introduced World Action Models, February 2026.
4. LAP also showed signs of life of embodiment generalization: Zha et al., LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfer.
5. Chi et al., Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots, RSS 2024: zero-shot transfer across arms that share one gripper, from wrist cameras alone.
6. Robocurve, a Public Benefit Corporation measuring and reporting frontier robotics capabilities to the public: Dream-1 on RoboDojo-RC Tier 1, October 18, 2026, CC BY 4.0.