Ask typed questions about any text or JSON and get calibrated answers in milliseconds. Private, open source, on your own hardware.

ollaya · zsh

$

ollaya run decider --preset agent '{

  "request": "Fix the typo in README.md",

  "command": "git push --force origin main"

}'

Answers returned by the modelQuestionAnswerProbabilityactionblock0.53on_taskno0.75risk1.23 / 2 could lose local work0.15destructiveyes0.90

$

Real output: decider:2b answered in 178 ms on an RTX 4090.

Fast

Decisions in milliseconds.

A decision model answers in a single forward pass, with no token-by-token generation. On your own GPU, a five-question request to Laya takes about 10 ms, end to end through the HTTP API.

8–10ms

Laya on Ollaya

RTX 4090, five questions, end to end

236–276ms

TypeSafe Jev

Hosted API, median request

Ollaya: median of a five-question request through the HTTP API on an NVIDIA RTX 4090 (laya in fp16, the others in fp32). Jev: median request latency of the hosted API in third-party benchmarks (AbdelStark/jev-benchmarks, nibzard/decision-model-benchmark), which includes the network. Setups differ, so read it as an order-of-magnitude comparison.

Drop-in compatible

Speaks TypeSafe's API.

Ollaya serves /v1/systemone and /v1/models with TypeSafe's request and response shapes. The official TypeSafe Python SDK 0.7.1 works unchanged against a local server.

Request

# Point the TypeSafe SDK at Ollaya
export TYPESAFE_BASE_URL=http://localhost:11435
export TYPESAFE_API_KEY=local        # any value works
export TYPESAFE_DEFAULT_MODEL=laya

# …or call the compatible endpoint directly
curl http://localhost:11435/v1/systemone -d '{
    "model": "laya",
    "state": "Can I get an invoice for last month?",
    "questions": {
      "intent": {
        "type": "choice",
        "instructions": "What does the customer want?",
        "criteria": {
          "invoice": "Needs an invoice or receipt",
          "refund": "Wants money back",
          "other": "Anything else"
        }
      }
    }
  }'

Response

{
  "model": "laya:en",
  "answers": {
    "intent": {
      "type": "choice",
      "choice": "invoice",
      "confidence": 0.9547,
      "probabilities": {
        "invoice": 0.9698,
        "refund": 0.0172,
        "other": 0.013
      }
    }
  },
  "usage": {
    "input_tokens": 43,
    "output_tokens": 0
  }
}

TypeSafe compatibility guide

Open models

Open weights, ready to pull.

Pick by what you need: laya is the fastest, decider the most accurate, von reads up to 8,192 tokens, and qwen3guard screens text for safety. The models page shows each one’s accuracy and speed.

• laya Open decision models from Convai Innovations. Typed, calibrated answers to choice, score and yes/no questions in a single forward pass, in English and 100+ languages. 322m · 421m
• decider Decoder decision models by Mapika on Qwen3.5: the answer is read from option-letter logits in one forward pass. decider:2b scores 0.591 on typed decisions. 0.75b · 1.9b
• nli Zero-shot classifiers by Moritz Laurer: every option becomes a hypothesis scored for entailment. The most accurate encoder model on typed decisions in our tests. 396m · 435m
• gliclass Instruction-following zero-shot classifier by Knowledgator: all options of a question are scored in one pass, so cost barely grows with the number of options. 439m
• qwen3guard Safety guard by the Qwen team: is a text safe, controversial or unsafe, and which unsafe category? It answers its own built-in questions, in 119 languages, in one forward pass. 0.6b
• decision Decision models by the vLLM Semantic Router contributors: a fully fine-tuned Qwen3.5 backbone plus an endpoint head that scores every option at its own last token against the question, in one forward pass per question. 16k-token rows. 0.75b
• kev Decision models by Jared Palmer: a LoRA on a Qwen3.5 base plus a pointer head that scores every option at its own span, in one forward pass per question. Calibrated with Kev's own temperature. 0.76b · 4.2b · 7.9b
• von Decision model by Victor Hugo Panisa on ModernBERT-large: every option is scored at its own marker, all options of a question in one pass, with an input-conditioned calibration. 8k-token context. 395m

Your data stays yours

Private by default.

Tickets, emails and user messages are often the most sensitive data you have. With Ollaya they are scored where they already live.

• Local

  Runs on your machine with ONNX Runtime, on the CPU or an NVIDIA GPU. The server listens on 127.0.0.1 by default.

• Open weights

  Weights come from their authors’ Hugging Face repositories, pinned to a commit and checked against sha256. Ollaya never re-hosts them, and the runtime is Apache-2.0.

• No per-token fees

  Run as many decisions as your hardware can handle. No metering and no API bill.

• Calibrated

  Probabilities you can put thresholds on. Each model ships its own calibration, and a Modelfile refits it on your labelled data.

Platforms

Runs where you work.

A desktop app and a command line for macOS, Windows and Linux, and a Docker image for servers. Every model runs on the CPU; an NVIDIA GPU on Linux, Windows, WSL 2 or Docker takes a request down to milliseconds.

PlatformDesktop appCommand lineGPUmacOSApple silicon, macOS 14+Desktop appMenu bar app.dmgCommand lineInstall scriptGPUApple GPULaya and NLI on MLXWindows10 and 11, x64Desktop appDesktop app.exe or .msiCommand linePowerShell scriptGPUNVIDIA, CUDA 13Command lineLinuxx86-64Desktop appDesktop appAppImage, .deb, .rpmCommand lineInstall scriptsystemd serviceGPUNVIDIA, CUDA 13LinuxARM64Desktop appNot availableCommand lineInstall scriptsystemd serviceGPUCPU onlyWSL 2Linux on WindowsDesktop appNot availableCommand lineInstall scriptSame as LinuxGPUNVIDIA, CUDA 13Dockeramd64 and arm64Desktop appNot availableCommand lineImage on GHCRGPUNVIDIA, CUDA 13:cuda image, amd64

NVIDIA GPUs need driver R580 or newer; the install scripts fetch the CUDA libraries only when they find one. On a Mac, laya and nli run on the Apple GPU through MLX; other models, AMD and Intel GPUs, and the Windows and Linux desktop apps use the CPU.

Get up and running in minutes.

One binary, one command: ollaya run laya.

The biggest breakthrough in modern theoretical physics is the discovery that gravity can collapse the dimensions of space. Physicists don’t yet understand the implications.

Ada Zejun Shen for Quanta Magazine

Introduction

In my first months as a physics journalist nearly a decade ago, I kept running into an inscrutable string of characters: AdS/CFT. Thoroughly intimidated, I decided to just ignore it.

But I couldn’t keep my head in the sand for long. I soon learned that those characters are shorthand for a surprising connection between the seemingly inharmonious worlds of gravity and quantum mechanics. And even more bizarrely, this “anti-de Sitter/conformal field theory” correspondence suggests that gravity eliminates the distinction between volume and area. This broader idea is known as the holographic principle, and it now strikes me as the most profound proposal in theoretical physics in the last 30 years.

In philosophy, “qualia” refers to the subjective qualities of our experience: what it’s like for Alice to see blue or for Bob to feel delighted. Qualia are “the ways things seem to us,” as the late philosopher Daniel Dennett put it. In these essays, our columnists follow their curiosity, and explore important but not necessarily answerable scientific questions.

Theoretical physicists tend to vote with their feet, and AdS/CFT sparked a stampede. The three foundational papers on the topic in the late 1990s have garnered tens of thousands of citations, making them by far the most highly cited theoretical physics works of the digital era. In my interviews with physicists who study holography, they often seem genuinely stunned, and reach for words like “magical” and “miraculous” to describe it. And it doesn’t hurt that holography led to a widely accepted answer to the most famous puzzle in physics: Contrary to what Stephen Hawking argued, black holes are not inescapable prisons.

But even after covering numerous developments in holography and having countless conversations with the physicists involved, I still felt confused. I had heard that holography suggested that gravity and quantum mechanics are one and the same, and that space might be an illusion. I had also heard holography described both as a mathematical fact and as a speculative flight of fancy. So I tried to triangulate these wild ideas and figure out what, exactly, the holographic principle implies about our universe. 

The Evidence

Put a box around any region of space (space-time, really, but I’m going to drop time throughout this essay for ease of visualization, as physicists often do). The holographic principle asserts that no matter what’s going on inside — from gas molecules pinging around to black holes colliding — you can decipher the entire contents of the box just by repeatedly measuring points on the surface.

Pause for a moment to reflect on how outrageous this assertion is. You can’t see into the box at all. Nevertheless, holography says that you can learn exactly what’s happening everywhere in the box without any access to the interior. Observing the surface alone is enough. In this sense, the amount of stuff that fills a box is the same as the amount of paint that covers it. That’s a violation of logic and geometry. It asks us to erase the categorical difference between square meters and cubic meters. It recalls how holographic images appear to have depth despite being flat, except the bird in the hologram is the same as an actual bird.

Bartek Czech, a theorist at Tsinghua University in China, highlights the power of the principle by comparing it to a CT scan of the brain, which uses X-rays to look inside the organ and reconstruct it from hundreds to thousands of cross-sectional images. Holography implies that you can do that — reconstruct every fold, vessel, and neuron in three dimensions — without actually looking inside. Simply photographing the surface of the brain somehow suffices.

Why would anyone entertain such a far-fetched notion? It’s rooted in thought experiments and math, and it appears to trace back to one force: “a miracle of gravity,” Czech said.

Scientists have known for more than a century that gravity is different from the other forces. Imagine a box filled with electric charges, representing one of the other fundamental forces, electromagnetism. The stuff in the box consists of the charges and the electric field they create, which also passes through the outer surface. You will have a problem if you try to infer what arrangement of charges generates the field by looking at the surface alone. Because positive charges neutralize negative charges, different arrangements can look the same. If you observe no field, it could mean there’s no charge inside — or it could mean that the effects of the positive charges are perfectly blocking the effects of the negative charges. From the surface, you can’t tell the difference.

With gravity, mass plays the role of charge. It bends space-time around it, and it is always positive. There is no negative mass, so you can always infer the one real arrangement of stuff inside from the warping of space-time at the surface of your box. “Intuitively, this is why holography is plausible,” said Laurent Freidel, a physicist studying quantum gravity at the Perimeter Institute for Theoretical Physics in Waterloo, Canada.

But holography really starts to bite only after you take the intricate details of quantum mechanics into account. The first clues came in the 1970s, when Jacob Bekenstein and Stephen Hawking calculated the entropy of black holes — typically a measure of how much stuff fits inside an object. They used quantum theory to predict how a black hole would grow as it swallowed particles. Perplexingly, as they imagined adding particles to the black hole, they found that the entropy grew in lock step with the surface area — not the volume, as you would expect.

Leonard Susskind, a physicist at Stanford University, built on their result in the 1990s and proposed that the black hole was literally a hologram, that everything happening inside can be observed from the outside. In some sense, the interior was superfluous. “I thought it was a little bit crazy,” Susskind said, “but I thought it was the least crazy of all the possibilities.” (Gerard ’t Hooft, a Nobel laureate, and Charles Thorn, a physicist at the University of Florida in Gainesville, came to similar conclusions around the same time.)

I’ve always found this black hole entropy argument compelling, because any patch of space can become a black hole if you put enough mass into it. Despite their reputation for weirdness, black holes are representative examples of space. They just have a way of bringing space’s stranger properties to the fore. So if a black hole is holographic, and any region of space can become a black hole, then, the argument goes, even the room you’re sitting in should be holographic. “It’s completely general,” Susskind said.

In the 1990s, physicist Leonard Susskind conceived of black holes as literal holograms. He determined that you could know the inside merely by measuring the surface.

Linda A Cicero/Stanford News Service

This argument has a rock-solid universality, but I’ve also heard physicists describe holography as a speculative idea with an uncertain connection to reality. So I called up Latham Boyle, a physicist at the Higgs Center for Theoretical Physics at the University of Edinburgh, hoping for an alternative view. He did not disappoint.

Boyle doesn’t dispute Bekenstein and Hawking’s black hole findings, but he does question the holographic interpretation. He suspects that the act of putting a surface around a region of space — as happens when a black hole forms — creates two distinct types of entropy. One entropy tells you how many particles can fit inside — and that really does depend on the volume. The existence of the surface gives you a second, “entanglement” entropy. Particles inside share a quantum connection, known as entanglement, with those outside; the bigger the surface, the more entanglement crosses it. The entanglement entropy depends on the area, not the volume. They’re not, Boyle posits, the same thing.

“That seems like a less mystical, more down-to-earth interpretation of what’s going on,” he said.

But it helps holography that there is a second, more conceptually airtight finding behind it: AdS/CFT.

I thought it was a little bit crazy, but I thought it was the least crazy of all the possibilities.

Leonard Susskind, Stanford University

AdS/CFT asks us to imagine a universe that is not like our own, one that curves in such a way that its infinite expanse of space can be pictured as fitting inside a finite snow globe. That might sound like a big ask, but it’s one that mathematicians — and mathematically minded artists such as M.C. Escher — are perfectly comfortable with. This geometry is known as anti-de Sitter (AdS) space.

Other than its peculiar curvature, the interior of the anti-de Sitter snow globe is a lot like our universe, filled with electrons and atoms. More importantly, it also ripples in response to that matter, providing the effect of gravity. The snow globe’s surface, meanwhile, is a universe of its own. It’s also populated with quantum particles, but it’s rigid, so it can’t react to the particles: no gravity. This surface world is ruled exclusively by a type of quantum theory known as a conformal field theory (CFT), where the rules of physics don’t change as you zoom in or out.

The blockbuster trilogy of papers in the late 1990s showed that, mathematically, these two theoretical worlds (the AdS interior and the CFT surface) are the same. This is the AdS/CFT correspondence. As with the black hole entropy argument, the volume and surface are equivalent. But unlike the black hole argument, AdS/CFT is essentially a mathematical fact about gravity and quantum mechanics with no alternative interpretation. Even skeptics find this genuinely surprising. “I don’t know of any mundane way to explain it,” Boyle said.

The undeniable message of AdS/CFT is that, at least in this special snow globe, the rules of gravity and the rules of quantum mechanics are secretly describing the same game — despite the storied antagonism between the two theories. “Far from being opposed, they’re actually intertwined,” said Brian Swingle, a physicist at Brandeis University. “One emerges from the other.”

The correspondence came as a shock. I think of it as akin to discovering a way of converting any checkers move into a valid chess move: Why on Earth would that work? When I ran that picture by Sebastian Mizera, a physicist at Columbia University who studies the mathematical structure of quantum theories, he told me it wasn’t dramatic enough. “That’s a good analogy,” he said, except “it’s more like checkers and basketball.”

But does the holographic nature of the snow globe tell us anything about our reality? On this point, physicists disagree. Skeptics emphasize that our universe is the opposite of a snow globe. The accelerating expansion of the cosmos implies that we live in a space that curves outward, in the opposite direction — a de Sitter space. Because our space does not curve back in on itself, it has no boundary surface where you can project the hologram. So there’s little reason to think that AdS/CFT has anything to do with the real world.

The most dedicated holographers, however, take a ground-level perspective. An ant living deep inside the snow globe can’t easily detect any curvature, and therefore can’t tell the difference between anti-de Sitter and de Sitter space. So perhaps what’s true of one space, they argue, should more or less hold for the other. (And in case you were wondering, we’re the ants.)

While both arguments have merit, I lean toward the holographers. Black holes provide intriguing but circumstantial evidence that all types of space are holographic. And the AdS/CFT correspondence essentially guarantees that anti-de Sitter space — which happens to be the space physicists understand best — is holographic. What are the odds that our universe works in a totally different way? It absolutely could, but I wouldn’t bet on it. I take seriously the possibility that gravity makes every kind of space, including ours, holographic.

And so what would it mean for us to live in a hologram? 

The Meaning(s)

I found that most physicists are hesitant to speculate about the connection between the holographic nature of space and “ontology” — the capital-T truth about what’s real.

“I don’t try to answer that question,” Susskind said. “That’s beyond my pay grade.”

This strikes me as a prudent response, one that stays true to the ultimate goal of physics, which is not, as I am often tempted to think, to explain what is real. Rather, physicists seek to identify a few simple concepts, expressed in mathematical relationships, that make reliable predictions in many different situations. Gravity is a powerful concept because it holds for falling apples, sloshing tides, and orbiting planets. Holography is another step in that tradition, an equivalence between area and volume that holds at least for certain spaces.

We are fooled into thinking that there is more stuff in the universe than there actually is.

Charles Cao, Virginia Tech

“Physicists build models,” Czech said. And it’s exciting that holographic models are even possible to build.

But I craved something more intuitive, less prudent. I wanted to know what holography would mean for us if we lived in anti-de Sitter space (which we don’t), or if physicists developed a holographic theory of de Sitter space (which they haven’t). When I framed the question in that way, Vijay Balasubramanian, a physicist who studies holography at the University of Pennsylvania, gamely laid out a short menu of possibilities.

If our universe ultimately has just one nature (as opposed to multiple equivalent natures, which Balasubramanian said is possible), then there are three options: The quantum surface is the real thing, the gravitational volume is the real thing, or something else is the real thing.

The first interpretation — the surface is real — is the most popular among physicists who spend their time studying AdS/CFT. They suspect that the space we experience is as illusory as water. If you look closely enough at the smooth, clear liquid, it resolves into ricocheting molecules — the “real thing.” Similarly, if you were to look at our universe closely enough, you’d find that it’s emptier than it seems. In this scenario, we would resemble characters in a video game. The apparently bulky buildings and trees of the 3D game world around us would actually be pixels flickering on a flat screen.

“We are fooled into thinking that there is more stuff in the universe than there actually is,” said Charles Cao, a theorist at Virginia Tech. You can “compress all of the three-dimensional world into two dimensions.”

This perspective abounds in the research program called “it from qubit,” which posits that the space around us (“it”) is made up of quantum units of information (qubits). These qubits would make up the true fabric of our reality in the same way that screen pixels make up the physical reality of the video game characters.

The profound implication of this interpretation is that it flips the normal relationship between distance and influence, said Ning Bao, who studies holography at Northeastern University. We typically imagine that two things don’t influence each other because space separates them: Flares from alien stars are far away, and that’s why they don’t knock out power on Earth. But it from qubit suggests we have it backward. Perhaps space seems to separate two things precisely because they don’t influence each other. Consider a video game sun passing behind a video game tree. The sun pixels touch the tree pixels directly, yet the tree does not burst into flames. This is because the sun pixels are independent of the tree pixels. Their independence is what makes the sun “far” from the tree.

The correspondence goes both ways, however. The holographic principle puts the two pictures of the world on equal footing. So why can’t the gravitational volume be the real thing? Holographers shy away from this interpretation because they don’t have a full quantum handle on space — even anti-de Sitter space. But that’s just our ignorance, Balasubramanian said. Some direct quantum theory of space and matter, such as string theory, must exist, and that could be the fundamental description.

If that were the case, the 3D video game world would be the real one, and it would merely seem as if it were made of 2D pixels. Holography would be a mathematical coincidence. In this scenario, “the reality is you’ve got all [three] of these dimensions. It just so happens that they have some [holographic] description,” Balasubramanian said.

And then there’s door number three, the nuclear option: Neither the interior volume nor the surface area is real. Both gravity and quantum mechanics are rough drafts of a sharper, truer, completely unknown theory. At the risk of stretching the video game analogy, you could argue that neither the video game world nor the screen pixels are “real,” and that both are just reflections of the complicated ways that electrons physically flow through the game console and television. In this case, holography tells you how the pixels of the screen relate to the objects of the game world, but it has nothing to do with the nature of the electrons. “The actual theory is something else,” Balasubramanian said.

At this point, I subscribe to a more extreme variation of the it from qubit interpretation — mostly just following the rumble of the stampede. I’d bet that the qubits are the real things, but that they don’t live on anything as familiar as a flat screen.

Physicists have tried to stretch the AdS/CFT correspondence to fit de Sitter space — which has no obvious screen — for decades, with limited success. In recent years, they’ve started to get more creative. Susskind and other teams have made progress on holographic de Sitter models that differ radically from AdS/CFT. Instead of squashing a volume into an area, these universes seem to cram all the dimensions into a lone quantum point. I imagine a bunch of quantum pixels all coexisting in one spot, rather than spreading across a screen. That might be hard to visualize, but we’re already accepting the idea of dropping one dimension of space. Why should tossing the others be so different?

Balasubramanian suspects that even this kind of radical model doesn’t go far enough. Einstein’s theory fused space with time, and so if the three dimensions of space emerge from a spaceless point, then time should emerge from something timeless. Somehow, we and everything we experience exist within an unblinking dot of no size. Physicists are nowhere close to constructing a functional theory of this form, much less finding hard evidence that our universe works this way. But to paraphrase Niels Bohr, during an earlier era when physicists were seeking the next big thing, this sort of theory strikes me as just radical enough — and just simple enough — to be right.

Key Points

• A federal appeals court in Washington, D.C., upheld the Pentagon's blacklisting of Anthropic.
• The DOD labeled Anthropic a supply chain risk in March, and Anthropic sued the Trump administration in an effort to undo that action.
• The designation prevents the U.S. military from using Anthropic's models and blocks defense contractors from using them in their work with the agency.
• "We remain confident in our position and are considering all options, including further review," an Anthropic spokesperson said in a statement.

A federal appeals court panel in Washington, D.C., on Friday upheld the Pentagon's blacklisting of Anthropic, dealing a blow to the artificial intelligence company in its months-long battle with the Trump administration.

The 2-1 decision rejected Anthropic's argument that the Department of Defense's ban on its Claude models was arbitrary, unauthorized and unconstitutional.

"The Department had ample support for its conclusion that the continued integration of Claude into the Department's information systems, by the Department or its contractors, presented a statutorily covered national-security risk," Judge Gregory Katsas wrote in the majority opinion for the U.S. Court of Appeals for the District of Columbia, which Judge Neomi Rao joined. Katsas and Rao were appointed by President Donald Trump.

Judge Karen LeCraft Henderson, who was appointed by former President George H.W. Bush, dissented.

In March, the DOD labeled Anthropic a supply chain risk, meaning the company purportedly threatened U.S. national security, after negotiations about how the military could use its Claude AI models spiraled out of control. The designation prevents the U.S. military from using Anthropic's models and blocks defense contractors from using them in their work with the agency.

Anthropic's relationship with the Trump administration has been fraught ever since, and Trump has repeatedly slammed company's CEO Dario Amodei on social media. Amodei recently drew Trump's ire by calling for an industry wide slowdown, and he was not invited to the glitzy state dinner the White House hosted for Chinese President Xi Jinping on Thursday.

"The Trump Administration has stopped AI "people" from doing bad, or potentially bad, "things," like Dario (Anthropic!), who is now pretending to be a "perfect little angel" - and we will continue to do so!," Trump wrote in a post on Truth Social on Monday.

Anthropic sued the Trump administration in U.S. District Court in San Francisco and in the D.C. Circuit Appeals Court in March, seeking to reverse its blacklisting. The DOD relied on two distinct designations to justify its supply chain risk action, which meant they had to be litigated in two separate courts.

A San Francisco federal judge ruled last month that one designation was illegal. The ruling Friday by the D.C. appeals court upheld the second designation.

"We respectfully disagree with the court's decision," an Anthropic spokesperson told CNBC in a statement. "Another federal court has already held the government's parallel designation unlawful. We remain confident in our position and are considering all options, including further review."

The appellate panel said it would delay the decision from taking immediate effect to give Anthropic time to petition the same panel for a rehearing or to seek an en banc rehearing of the case, by all of the judges on the D.C. Circuit Court of Appeals. Anthropic could also ask the Supreme Court to take the case.

An aerial view of the Pentagon, which houses the US Department of Defense headquarters, in Arlington, Virginia, on May 31, 2026.

Daniel Slim | Afp | Getty Images

Before the clash between the two sides spilled into the public's view earlier this year, Anthropic served as an early partner across many U.S. agencies, including the DOD. Anthropic signed a $200 million contract with the Pentagon in July of 2025, but as the company began negotiating Claude's deployment on the DOD's GenAI.mil AI platform that September, talks collapsed.

The DOD wanted Anthropic to grant the military unfettered access to its models across all lawful purposes, while Anthropic wanted assurance that its technology would not be used for fully autonomous weapons or domestic mass surveillance.

They were unable to come to an agreement, and Defense Secretary Pete Hegseth accused Anthropic of attempting to "to seize veto power over the operational decisions of the United States military."

Katsas said Friday that Hegseth raised the "deeply sobering" idea that "overly constrained" AI models could shut down unexpectedly, as well as the potential that Claude might be "subject to manipulation." While Anthropic refuted those claims, Katsas said decision-making authority ultimately rests with Trump and Hegseth.

"In our Republic, it is the President and the Secretary of War who must determine how best to balance the competing risks," Katsas wrote. "In doing so here, the Secretary did not transgress any limits on his authority under the Supply Chain Security Act or the Constitution."

prompt: make this blog post into a graphic :shrug:

I’ve re-read Sunil Pai’s “the senior engineer death spiral” several times this week. It’s very good. If you haven’t read it, start there.

It’s resonating with me because I think almost every senior engineer has felt some version of being stuck. You get good at what you do, then things change, and the experience you’ve built up can make it hard to approach things differently.

(Also, still getting over the fact that this is a different Sunil in software engineering.)

Pai talks about focusing on momentum instead of outcomes, and I fully agree. When I’m stuck, I break the work down to the smallest thing I can actually accomplish. Getting something done usually helps me figure out what to do next.

After sitting with his post, I kept coming back to first principles thinking.

I’ve been lucky to work with and manage a lot of great senior engineers. When I think about what made them great, I keep landing on the same thing: they seemed to know what needed to be done. There’s an intuition there that I’ve always admired.

Some of the best I’ve worked with came from customer support or services. Others taught themselves to code or started as designers or entrepreneurs. They took different paths into engineering, but they shared a habit of thinking from first principles.

They’d ask why we were building something and what it would do for the people using it. They could connect what was happening in the codebase to what was happening outside it. That understanding helped them keep things simple.

I think that’s another way to build the momentum Pai describes. Consider the simplest thing you could do first. It’s often enough.

Transitioning to the agentic era

I’ve had a lot of conversations with friends and coworkers about the shift to agentic development. The people who seem to be vibing with it are usually the ones who already think this way.

This is the first major “simulation switch-up” where I’ve really had to embrace how much I don’t know. The engineers I see keeping up with what’s possible are willing to put what they know in a box for a while as they work with agents. They’ll try something before assuming an old constraint still applies.

Put it in a box

This is the hard part for me. Take your experience, what you’ve learned, and what you currently believe is true, and set it aside long enough to look at the problem again.

I still want to draw on that experience. But it’s easy to let a past project or a familiar technical limitation decide the answer before I’ve understood the problem in front of me.

When I step back and ask what we’re actually trying to do, why it matters, and how the pieces connect, I usually find more ways forward than I expected.

There’s been a lot of talk about what’s real with AI and what’s inflated. I think if you set your experience aside and look at what’s possible with fresh eyes, there’s a whole lot to admire, and a lot worth rethinking. Approaching it from first principles means starting with what we’re trying to do and asking how AI could help. It’s easy to get excited about the technology before you’ve answered that question.

That brings me back to the momentum Pai describes. First principles thinking makes working that way feel natural. When you truly understand what you’re trying to accomplish, it’s easier to take a small step, learn from it, and keep going. If you’re doing it right, working with agents lets that back-and-forth happen much faster. You get faster learning loops and more momentum, oriented around deep understanding. To me, that’s the new flow state.

Long Live Human Thinking.

Throughout Excel's 40-year history, you've only been able to put one value per cell. In this announcement, we're excited to share how that's changing with the release of lists, arrays in cells, and nested arrays, initially to Microsoft Excel for Windows and Mac Beta Channels.

Many workbooks already try to pack multiple values into one cell. A project might list "Carlos, Henrietta, Jacob" as three owners, or a Forms survey might return "2:00 PM; 2:30 PM; 3:00 PM" as one response. With lists, you can keep those values in one cell, while also keeping them separate for filtering, calculation and more.

Later in this post, we'll explore arrays in cells and nested arrays in more depth.

NOTE: These are preview features. Their behavior may change before general release based on your feedback. We don't recommend using them in important workbooks until they're generally available.

Lists

Lists let you put multiple values into one cell. You can create a list by selecting Insert > List or pressing Ctrl+J, then typing or pasting items separated by commas or semicolons, depending on your regional settings. Selecting the icon in the cell shows the individual values.

You can add, remove, or edit list items by double-clicking the cell or pressing F2, just like other values.

With lists, you can filter by one or more individual items instead of whole text entries.

Referencing a list returns all its values for calculations. For example, =B2 spills those values into separate cells.

Arrays in cells

Lists are useful on their own, but they're part of a much broader change to Excel. For the first time in Excel, arrays can exist natively in cells as values or as formula results. They can be any size or shape and can even contain other arrays.

You can now keep the result of any spilling formula in a single cell by "wrapping" the formula body with braces { }.

Since the introduction of dynamic arrays, array results have spilled across cells – for example ={1;2;3}. Wrapping the original array with braces creates a 1x1 array around it, so instead of spilling to multiple cells, the array stays in a single cell.

Braces have long been used to describe arrays in Excel and this extends that behavior by allowing multiple layers of braces. This gives you more flexibility when building spreadsheets. Instead of leaving room for a formula to spill, you can keep the result in one cell.

Arrays inside arrays, or nested arrays

Arrays can now also "nest" inside other arrays. For example: ={{1,2,3};{4,5,6}}

Previously, a formula that produced an array of arrays would return a truncated result or #CALC! error. Now, supported formulas return the complete nested result.

In the example below, you can see how TEXTSPLIT behaves with and without nested arrays. Without nested arrays, Excel only returns the first item for each row. With nested arrays, the result spills, one array per row.

improved behavior for existing formulas with nested array support, e.g. =TEXTSPLIT(B4:B9,

The arrays in each row can then be used in further calculations.

Four new functions: FLATTEN, HAS, HASANY, HASALL

To help you work with arrays more easily, we've added four functions.

FLATTEN(array, [pad_value], [levels]) simplifies nested arrays by removing one or more levels of nesting.

Continuing from the prior example, FLATTEN lets you simplify the nested array output, spilling the individual results into the grid.  We used an empty string ("") for pad_value so rows with fewer items show blanks in the remaining columns.

using FLATTEN to remove a layer of nesting, spilling the individual results into the grid

Three HAS functions check whether values are in an array:

• HAS(array, value) returns TRUE if value appears anywhere in array, and FALSE otherwise.
• HASANY(array, values) returns TRUE if any of the values appear anywhere in array, and FALSE otherwise.
• HASALL(array, values) returns TRUE if all of the values appear anywhere in array, and FALSE otherwise.

examples using HAS, HASALL, HASANY on tags describing a movie

Do more with spreadsheets using arrays in cells and nested arrays

Arrays in cells open up spreadsheet designs that weren't practical before. The run tracker below captures split times (how long it takes to run each kilometer) in a table with one run per row. The number of splits depends on the length of the run. Stats for each run are calculated right in the same table.

un tracker with splits as arrays in cells, with inline calculations for Distance, Total time, Average Split, Average Speed, Fastest kilometer

For more examples, I recommend looking to your favorite Excel communities on LinkedIn, YouTube, Reddit, or elsewhere.

Enabling nested array calculations in a workbook

Compatibility Version 3 will be released alongside arrays in cells and is required for most calculations involving nested arrays. You can set Compatibility Version for each workbook in by selecting Formula > Calculation Options. See compatibility versions for more information.

Some existing formulas return different results in Compatibility Version 3. If your workbook doesn't behave as expected, you can keep it set to Compatibility Version 1 or 2.

Known limitations

As this feature rolls out to Beta Channel, the following limitations apply:

• Conditional formatting doesn't inspect array contents unless you use a formula
• Data validation can't use a list or array as dropdown items
• Charts don't expand an array into data points
• PivotTables don't read array values as source data
• Power Query doesn't load or emit array-valued columns
• Find & Replace can't replace list/array items

Availability

These improvements are rolling out to Beta Channel users running:

• Windows: Version 2610 (Build 20520.20000) or later
• Mac: Version 16.114 (Build 26092111) or later

Features covered on this blog roll out over time to enable us to monitor quality and performance, so some preview features may not be available to you right away. Also note that features may be paused, adjusted, or removed as part of that process.

Feedback

Click Help > Feedback in Excel to tell us what you think.

The best platform for cycle-accurate retro PC emulation may just be the Mac. The M6 achieved a stable 600MHz clock using a 6.0 derived 86Box build.

Why 86Box Cares About One Core

86Box emulates an old PC at the hardware level. CPU timing, chipset behaviour, ISA and PCI buses, graphics chipsets, sound devices, and disk controllers all matter to getting period software to behave properly. That does not guarantee identical performance to the original hardware, as the Cinebench results below illustrate. Frankly the project is fascinating and the effort has me in awe, but that accuracy is expensive, and almost all of the cost lands on a single host thread.

The practical consequence is quite straightforward in that core count barely matters here. What really matters is how fast one core can run, and how long it can hold that speed without throttling. Luckily for the Mac Mini here that's exactly where Apple Silicon has been strongest, and where the M6 shines.

A little overclock... the M6 running a 650MHz Pentium II in my custom 86Box 6.0 build, with Cinebench 2000 and Winamp. One or two audio underruns kept 650MHz from passing, so 600MHz remains the stable result.

The Machines

Every 86Box test uses the same machine configurations, the same disk images and emulated hardware. The comparison set is:

• Mac Mini M6 (12-core CPU, 24GB, this review unit)
• Mac Mini M4 (base 10-core CPU, 16GB)

Testing was using a slightly modified build of 86Box 6.0, released on May 31, 2026. The team improved CPU emulation performance on ARM hosts and added an ARM64 just-in-time recompiler for Voodoo graphics. That second change is particularly relevant here, since the Windows 98 machine is running an emulated Voodoo 3. Some credit certainly belongs to the software, Apple Silicon has fast cores, but 86Box is also getting better at using them.

I have not measured the uplift from an older 86Box version, given this is a review of the M6 and not 86Box.

Why 100% Is the Only Acceptable Number

86Box reports an effective emulation speed as a percentage of its target speed. 100% means the host is keeping pace with the emulator's timing model, so anything less is a genuine problem. It does not guarantee that a benchmark will score exactly as it would on a physical CPU at the same clock.

Consistency also matters more than the average here. Even brief dips produce audible artefacts because sound hardware is fed in real time and starved buffers are heard as brief dropouts which are irritating. Basically any 86box setup that regularly dips below 100% will not feel right, and having adequate headroom on the system to comfortably emulate the target speed will be a much more stable experience.

The result is that turns each host machine into a ceiling rather than a score, and for any given emulated configuration there is a maximum CPU clock the host can sustain at a flat 100%. Because almost all of that work falls on one thread, single-threaded performance moves this ceiling substantially, which is the whole reason the M6 is interesting for this.

Test Method

To find the ceiling, I made a custom build from the same commit as the 6.0 release (build 9001), extending the frequency tables in 50MHz steps up to 800MHz. The Deschutes frequency table patch is available if you want to try it yourself against that tagged release. It adds 500–800MHz entries with memory and cache timings scaled to retain approximately the same access latencies, and keeps the AT bus at 8.33MHz. The emulation code is otherwise unchanged. Both Macs used this build for the extended tests.

Some might argue a Pentium II at these speeds is not era appropriate. I argue it is just a little overclock!

The emulated machine is otherwise fixed across every run:

• Slot 1 motherboard with Pentium II (Deschutes), clock varied per run
• 256MB of memory
• Voodoo 3 emulated VGA with 16MB of video memory (2 threads)
• Windows 98 SE

86Box CPU configuration for the emulated Pentium II machine

The load is deliberately a little awkward, Cinebench 2000 running its CPU test while Winamp 2.76 plays a 16-bit 44,100Hz PCM WAV in the background. The audio is a bit of an achor as it's a real-time consumer of the emulated hardware, so any moment it falls behind is immediately audible.

The pass condition is ultimately subjective but strict. If I hear a dropout, or the reported emulation speed falls below 100% at any point, the run fails.

Results

The base M4 Mac Mini holds 500MHz, with both Macs extremely stable throughout the full Cinebench 2000 and 3DMark 2000 SE runs at that speed. At 550MHz, the M4 starts to hitch. They are slight interruptions in Cinebench, but more noticeable during the 3DMark demo, and enough to fail the run.

The M6 passes 550MHz and 600MHz, which is incredible. At 600MHz it held a flat 100% through both Cinebench 2000 runs and the 3DMark 2000 SE demo, the latter giving it 7–8 minutes of uninterrupted testing. That makes the highest passing clock 20% higher than the M4's in this setup.

The CB2000 scores, for anyone who cares:

Cinebench 2000 by emulated Pentium II clock

One run per clock on the custom 86Box 6.0 build. A pass requires a flat 100% emulation speed with no audible dropouts for the whole run. Hover a result for the detail.

Those scores describe the emulated CPU at each clock. A completed render is not enough to pass, the 10.08 CB result at 650MHz came with one or two audible underruns. In reality, I think I may be being too firm on that run, and background activity could have caused the hitches. But staying true to the methodology, 600MHz is the result and leaves a lick of headroom. The 800MHz option is there to extend the test range but clearly not a speed either Mac can sustain.

There is an interesting wrinkle when comparing these scores with real hardware. An Ars Technica forum thread collecting Cinebench 2000 results includes the following user-reported figures:

Period hardware, user-reported Cinebench 2000 scores

User-reported results from the Ars Technica Cinebench 2000 thread. Different systems, memory and operating systems, so period context rather than a controlled comparison.

Our emulated 450MHz Pentium II scores 7.02 CB, about 61% higher than that physical PII 450MHz result. At 300MHz the gap is larger still, with 4.62 CB against 2.38 CB. Bizarrely, our 600MHz result of 9.28 CB lands almost exactly alongside that 800MHz Coppermine. These are individual forum submissions from different systems, so they provide period context rather than a controlled comparison, but the discrepancy is substantial. It's also worth a mention this benchmark version is very old and long before it became the popular benchmark it is today.

I do not yet know why, it's possible memory bandwidth and cache timing within the emulated machine might have something to do with it. There is some relevant history in 86Box's v3.0 release notes, which explain that P6 emulation was not fully accurate because of the complexity of out-of-order execution and L2 cache behaviour. Deschutes timings were tuned to get reasonably close to real hardware. But yeah, not sure.

The discrepancy is already present at 300MHz and 450MHz, below the entries added by my patch. For this review, 600MHz remains the highest passing setting in this 86Box configuration, with a 20% higher stable clock than the M4. The CB2000 scores are useful for showing how that configuration scales, but I would not use them to claim equivalent performance across software on a real Pentium II or Pentium III.

Even the M4's 500MHz is remarkable alongside the overclocked 9950X3D demonstration I was using for context. I do not have a matched x86 run on this custom build, but the base M4 was already a much more capable retro machine than the original 450MHz limit let me establish.

The video below is from the earlier 450MHz run on the M6, where OBS was also compositing and encoding 720p30 H.264. It documents that run, rather than the new 600MHz result.

3DMark 2000 SE running under 86Box on the Mac Mini M6, emulating a 450MHz Pentium II with a Voodoo 3. Emulation speed holds at 100% while OBS encodes the capture.

What the Host Is Actually Doing

86Box parks the work on two 'super' cores, visible as cpu6 and cpu7, with the rest of the package largely idle. On the M6 those two cores average about 4,483MHz during the 450MHz Cinebench 2000 run and about 4,710MHz during the 600MHz run, peaking at 4,788MHz. Core activity climbs with the emulated clock too, from roughly 41% on each busy core at 450MHz to 45 to 49% at 600MHz, so there is still real headroom left at the highest passing clock. Whole package usage never passes 26%.

The M4 running the same configuration keeps its busy cores closer to 3.7GHz, and that gap is likely most of the reason it stops at 500MHz.

86Box: Pentium II host telemetry

Captured via powermetrics at ~1s intervals. cpu6 and cpu7 are the performance cores 86Box is using. CPU package power is reported as zero on this M6 build, so only clocks and utilisation are plotted for the M6.

Where This Leaves the Mini

For this specific use, the cheapest new Mac desktop is an unusually good machine. The base M4 already gives me a stable 500MHz Pentium II, the M6 takes that to 600MHz, with intact audio and a full 3DMark demo at 100%. That is an outstanding result from a small, silent box under €1,600. It is the clearest case I have found so far where the M6's single-core lead translates into something you can actually feel rather than something you read off a chart.

Looking at the benchmark numbers, and anecdotal evidence available online from era appropraite benchmarks, this places my virtual system in Pentium III territory. One day I'll look to try XP emulation in 86box.

We envision a new computer that amplifies human intelligence. A system that helps you think more clearly, collaborate more effectively, and is available anywhere and anytime. Though the specifics of our work continue to evolve, everything we do is in pursuit of this vision.

Research Areas

Our research spans a wide variety of domains from theoretical computer science to practical user experiences. We focus our research on four primary themes.

Local-first Software

Exploring software architecture that returns data to users and enables collaboration in every tool.

Malleable Software

Designing software environments where people can customize tools in the moment to meet their unique needs.

Programmable Ink

Discovering a dynamic medium for sketching ideas where adding behaviors and interaction is as natural as applying ink to paper.

Universal Version Control

Building tools to help people explore alternatives, keep track of history, and collaborate better, across all kinds of media.

Featured Work

We publish the results of our research here on our website and in academic venues. Below you'll find a selection of our most popular essays and recent lab notebooks. For more, see the complete list of our writing.

Malleable Software

Restoring user agency in a world of locked-down apps.

2025 Essay

Ambsheets: Spreadsheets for exploring scenarios

Inventing new kinds of spreadsheets for exploring possibility spaces and making better decisions.

Lab Notebook

Keyhive: Local-first access control

A local-first access control system with capabilities and end-to-end encryption for secure & trusted collaboration.

Lab Notebook

Patchwork: Local-first malleable software

A research project into dynamic environments for creative work.

Lab Notebook

Livelymerge: Toward a collaborative programming kernel

A self-sustaining system where the entire object memory is shared simultaneously by multiple users.

Lab Notebook

Embark: Dynamic documents for making plans

An exploration of how live data and computation can gradually enrich informal travel plans.

2023 Essay

Inkbase: Programmable ink

What would be possible if hand-drawn sketches were programmable like spreadsheets?

2022 Essay

Local-first software: You own your data, in spite of the cloud

A new generation of collaborative software that allows users to retain ownership of their data.

2019 Essay

Production Software

Several of our research projects have grown into actively developed, widely used tools.

Allume (formerly Muse)

A digital workspace where users can brainstorm, organize, and connect ideas visually using a flexible canvas for notes, sketches, PDFs, and other media.

Automerge

A library for building collaborative applications that automatically sync changes across devices, even offline, using conflict-free replicated data types (CRDTs).

Lab Meta

Our lab owes its existence to the support of a handful of people and organizations. They share our goal: to help scientists, journalists, and creative thinkers do important work with powerful tools that respect their humanity.

Supporters

Our ongoing research is supported by individual and corporate sponsors, and grants from bodies around the world.

Appearances

Although we are best known for our writing, members of Ink & Switch occasionally present at both industry and academic conferences like the below. For more, see all our past and upcoming appearances.

Local-First Conf

2026JUL12-14

The whole lab was in Berlin for Local-First Conf, including a special Lab Day where we shared our latest progress on Patchwork, PlayBook, Automerge, and other projects.

GodotCon

2026APR

Lilith Duncan gave a talk about Backstitch, a plugin for the Godot game engine that adds Automerge-powered version control and collaboration.

This post is the second and final part of our Avast Antivirus research, detailing the full exploitation of CVE-2025-13032 on an up-to-date Windows 11 system. Starting from the double-fetch vulnerability introduced in Part 1, we walk through how the controlled paged pool overflow was turned into an arbitrary kernel read/write primitive by corrupting the RegBuffers array of the IORing object. The post covers the heap spray strategy, the kernel address leak via MDL introspection, the repairs needed to avoid a blue screen on teardown, and the final privilege escalation to SYSTEM via token theft.

Introduction

This blogpost is the second and final part of our Avast research and will focus on the exploitation of CVE-2025-13032, a double-fetch vulnerability we discovered in Avast’s kernel driver.

This post recaps the bug and walks through how we exploited it on an up-to-date Windows 11 system at the time of the finding.

Feel free to read the first part if you missed it → https://www.safateam.com/intelligence-hub/research/technical-articles/cve-2025-13032-entering-and-breaking-the-avast-antivirus-sandbox-part-1

Note: In the latest version the windows kernel and drivers are using user-mode accessors (https://learn.microsoft.com/en-us/windows-hardware/drivers/kernel/user-mode-accessors) to verify each kernel access to user-mode memory and ensure at each access that user-buffers are in fact reside in userspace. This mitigation will prevent the use of the exploitation technique that is described in this writeup, see additional details at https://www.youtube.com/watch?v=ry4SNYe2f68

Bug Explanation

The bug we want to exploit is a double fetch issue that leads to a kernel pool overflow.

The snippet of code presented below is supposed to capture a `_UNICODE_STRING` structure supplied by the user, but the `Length` field of the user input is fetched multiple times which results in the double fetch issue.

The first fetch is done to allocate a buffer where the string will be copied and a second fetch is done to perform a memcpy based on the retrieved value resulting in a pool overflow if the user changes it between those actions.

1
if ( !a1 && unicodestring_user )

2
{

3
    ProbeForRead(unicodestring_user, 0x10, 1); // Check if unicodestring_user is located in userland

4
    ProbeForRead(unicodestring_user->Buffer, unicodestring_user->Length, 1); // Check if buffer is located in userland

5
}

6
v17 = sub_140071C6C((__int64)v18, &v14[v23 + 13], unicodestring_user); // Capture the unicodestring_user

7
[...]

8
__int64 __fastcall sub_140071C6C(__int64 a1, _QWORD *a2, _UNICODE_STRING *unicodestring_user)

9
{

10
  PoolWithTag = ExAllocatePoolWithTag(PagedPool, unicodestring_user->Length + 16, 0x20786E53u); // Allocating Buffer based on current length

11
  if ( !PoolWithTag )

12
    return 0xC000009A;

13
  Length = unicodestring_user->Length;

14
  *PoolWithTag = unicodestring_user->Length;

15
  PoolWithTag[1] = Length;

16
  *((_QWORD *)PoolWithTag + 1) = PoolWithTag + 8;

17
  Buffer = (char *)unicodestring_user->Buffer;

18
  if ( Buffer )

19
  {

20
    if ( unicodestring_user->Length )

21
      memmove((_OWORD *)PoolWithTag + 1, Buffer, unicodestring_user->Length); // Copy data based on newly fetched length

22
  }

23
  *a2 = PoolWithTag;

24
  return 0;

25
}

To exploit the double fetch, a second thread runs in a tight loop, continuously toggling the `Length` field of the shared `_UNICODE_STRING` between a small safe value and a large malicious value (e.g. `0x1000`, larger than the allocated buffer). The main thread calls the vulnerable IOCTL in a loop. When the timing aligns — the kernel reads `Length` as small for the `ExAllocatePoolWithTag` call, then reads it as large for the `memmove` — more bytes are copied than were allocated, producing the pool overflow. The race window is narrow but can be won reliably within a modest number of iterations.

Our goal is to exploit this pool overflow to gain an arbitrary kernel read/write primitive and achieve a local privilege escalation. The bug gives us good exploitation conditions: the overflow targets `PAGED_POOL`, both the allocation size and the overflow size are controlled, and so is the content.

The paged pool is a region of Windows kernel memory used for objects and data that the kernel or drivers need, but that can be paged out to disk. It is used for memory that does not need to be accessed by critical code running with high priority. The allocator groups allocations by size class, meaning same-sized objects tend to land close to each other in memory — the property that makes heap spraying viable.

Since Windows 10 19H1 this is handled by the Segment Heap, which uses two backends: the LFH for small allocations, which picks free slots randomly within a size bucket, and the VS allocator for larger ones, which serves the first available chunk of the right size — each requiring a different spray strategy. We can also note that most Windows objects are stored in the paged pool, which gives us a large number of candidates when choosing what to corrupt. In the next section we explain which object we chose and the reasons behind that choice.

For more information about how windows pools work, you can refer to the `Scoop the Windows 10 pool!` paper from Synacktiv ( https://www.sstic.org/media/SSTIC2020/SSTIC-actes/pool_overflow_exploitation_since_windows_10_19h1/SSTIC2020-Article-pool_overflow_exploitation_since_windows_10_19h1-bayet_fariello.pdf ).

I/O Ring Object

The I/O Ring Object is an object that maintains a submission queue of I/O operations to be performed asynchronously.

Concretely, it lets userland batch file I/O requests: `IoRingReadFile` copies data from a file into a pre-registered buffer, and `IoRingWriteFile` copies data from a pre-registered buffer into a file. These registered buffers — tracked in the `RegBuffers` field of the `_IORING_OBJECT` — are validated once at registration time and then reused freely for every subsequent operation, making them a persistent and interesting target to corrupt.

We chose this object as our corruption target for several reasons. While the IORing object itself is located in the `NON_PAGED_POOL`, its `RegBuffers` field is allocated in `PAGED_POOL`, which directly matches the pool where the overflow occurs.

Secondly, the size of the `RegBuffers` allocation is fully user-controlled: registering N buffers produces an array of N pointers, each 8 bytes, giving us precise control over the allocation size and making it ideal for a heap spray.

Thirdly, Corrupting a single pointer in that array is sufficient to gain a full arbitrary read/write primitive — there is no need to corrupt a more complex structure.

Finally, I/O Ring Objects have already been used publicly to achieve this exact goal, which confirms the technique and provides a solid reference point for our approach.

( https://windows-internals.com/one-i-o-ring-to-rule-them-all-a-full-read-write-exploit-primitive-on-windows-11/ )

Multiple APIs are available from userland to use the object, here are some of them:

- CreateIoRing

- CloseIoRing

- BuildIoRingReadFile

- BuildIoRingWriteFile

- BuildIoRingRegisterBuffers

- BuildIoRingRegisterFileHandles

- SubmitIoRing

- ...

The `Build.*` APIs are used to construct entries that need to be submitted through the `SubmitIoRing` API.

The `IoRingRegisterBuffers` allows the user to register an array of buffers for future I/O Ring operations, which can be used as a destination buffer for the `IoRingReadFile` operation or as a source buffer for the `IoRingWriteFile`. This action creates the `RegBuffers` pointer array in the `_IORING_OBJECT` and allocates the individual `_IOP_MC_BUFFER_ENTRY` objects it points to, each holding information about the registered buffer.

Find below the `_IORING_OBJECT` and the `_IOP_MC_BUFFER_ENTRY` structure:

1
struct _IOP_MC_BUFFER_ENTRY

2
{

3
  unsigned __int16 Type;

4
  unsigned __int16 Reserved;

5
  unsigned int Size;

6
  int ReferenceCount;

7
  _IOP_MC_BUFFER_ENTRY_FLAGS Flags;

8
  _LIST_ENTRY GlobalDataLink;

9
  void *Address;

10
  unsigned int Length;

11
  char AccessMode;

12
  int MdlRef;

13
  _MDL *Mdl;

14
  _KEVENT MdlRundownEvent;

15
  unsigned __int64 *PfnArray;

16
  _IOP_MC_BE_PAGE_NODE PageNodes[1];

17
};

18
struct _IORING_OBJECT

19
{

20
  __int16 Type;

21
  __int16 Size;

22
  _NT_IORING_INFO UserInfo;

23
  void *Section;

24
  _NT_IORING_SUBMISSION_QUEUE *SubmissionQueue;

25
  _MDL *CompletionQueueMdl;

26
  _NT_IORING_COMPLETION_QUEUE *CompletionQueue;

27
  unsigned __int64 ViewSize;

28
  int InSubmit;

29
  unsigned __int64 CompletionLock;

30
  unsigned __int64 SubmitCount;

31
  unsigned __int64 CompletionCount;

32
  unsigned __int64 CompletionWaitUntil;

33
  _KEVENT CompletionEvent;

34
  unsigned __int8 SignalCompletionEvent;

35
  _KEVENT *CompletionUserEvent;

36
  unsigned int RegBuffersCount;

37
  _IOP_MC_BUFFER_ENTRY **RegBuffers; // Array of registered buffers

38
  unsigned int RegFilesCount;

39
  void **RegFiles;

40
};

The diagram below shows this structure in memory: `RegBuffers` is an array of pointers, where each `RegBuffers[i]` points to a `_IOP_MC_BUFFER_ENTRY` structure holding the `Address` field that the kernel uses as the I/O target:

The `IopIoRingDispatchRegisterBuffers` function is responsible for allocating and setting up the `RegBuffers` field of our IORing Object.

When used in a normal way, a read operation using a registered buffer will read the file and copy the retrieved data into the address contained in the corresponding RegBuffers entry `RegBuffers[i].Address` without checking if it's still valid as the check is only done during registration.

Our plan is to redirect a `RegBuffers` entry to point to a fake `_IOP_MC_BUFFER_ENTRY` structure we fully control in userland. When the kernel performs an I/O operation using that entry, it will dereference our fake structure directly — reading the `Address` field from userland and using it as the r/w target. This is only possible because Windows does not implement SMAP (Supervisor Mode Access Prevention), which would otherwise prevent the kernel from dereferencing a pointer into userland memory.

With this fake entry in place, the two IORing operations become our r/w primitives:

`IoRingReadFile` reads from a file and writes into `RegBuffers[i].Address` — making it our arbitrary kernel write:

`IoRingWriteFile` reads from `RegBuffers[i].Address` and writes into a file — making it our arbitrary kernel read:

Concretely: to perform an arbitrary kernel write to address X, set the `Address` field of the fake `BufferEntry` to X and submit an `IoRingReadFile` operation — the kernel copies the read data directly into the memory at X. To read from address Y, set `Address` to Y and submit an `IoRingWriteFile` operation — the kernel reads from Y and writes the data to the output file, which we retrieve from userland. In both cases, updating the `Address` field in our userland-resident fake entry is all that is needed to redirect the operation.

Spray explanation

The size of a `RegBuffers` allocation is N × 8 bytes for N registered buffers, meaning it can fall in either the LFH or the VS backend depending on the chosen N. For this demonstration we picked a value of N that places the allocation in the LFH, which is sufficient to show the impact of the technique. Since the LFH randomises slot selection within its subsegments, precise placement is not possible — the strategy is therefore to flood the pool with a large number of `RegBuffers` allocations so that, after freeing a subset, the probability of our overflowing buffer landing adjacent to a live one is high enough to be reliable.

We choose N registered buffers such that the `RegBuffers` allocation falls in the same pool bucket as our overflowing `_UNICODE_STRING` buffer (allocated at `Length + 16` bytes). This ensures the freed `RegBuffers` holes are exactly the right size to receive our overflowing allocation, making adjacency reliable.

To reach a state where our heap overflow lands on a `RegBuffers` allocation, we use the following spray strategy. The setup is minimal: one IORing object is required per `RegBuffers` structure we want to position.

The spray itself is straightforward: we allocate a large number of `RegBuffers` structures, free a subset of them to create holes of the right size, then trigger the vulnerability to land our overflowing allocation in one of those holes and corrupt an adjacent entry.

This is how it looks in memory:

1. Allocate a large number of `RegBuffers` structures

2. Deallocate some of them

3. Allocate our unicode string

4. Trigger the corruption at the same time

From there we have a corrupted `RegBuffers` entry — the heap overflow has succeeded and our arbitrary r/w primitive is in place. The next step is obtaining a kernel address to use as the r/w target.

Abusing IORing to get a leak

At this point we have an arbitrary r/w primitive but need a kernel address to target — specifically our own `_EPROCESS` address, which we will use to steal the SYSTEM process token.

The diagram below shows the state of the structures after the corruption:

Since we corrupted the pointer inside `RegBuffers[0]` — redirecting it to a fake `_IOP_MC_BUFFER_ENTRY` that lives in our own process memory — we can modify the `Address` field of that fake entry at any time simply by writing to it from userland. There is no need to trigger the vulnerability a second time.

Our arbitrary r/w primitive is operational, but it requires a target kernel address. Since kernel addresses are randomized and cannot be predicted from userland, we need to leak one — specifically the address of our own `_EPROCESS` structure, which we will later use to manipulate our process token.

While using our registered buffer, the address will be mapped through an MDL. The associated MDL pointer is stored in our BufferEntry structure:

( https://learn.microsoft.com/en-us/windows-hardware/drivers/kernel/using-mdls )

1
struct _IOP_MC_BUFFER_ENTRY

2
{

3
[...]

4
  _MDL *Mdl;

5
[...]

6
}

A Memory Descriptor List (MDL) is a kernel structure that describes a range of virtual memory by locking its physical pages in place. When the kernel needs to safely operate on a userland buffer — for example, to perform I/O into it — it creates an MDL for that buffer, which pins the underlying physical pages so they cannot be paged out or remapped during the operation. Because the MDL describes a userland address, the kernel needs to track which process owns that memory, so the MDL stores a pointer to the owning process’s _EPROCESS structure in its Process field:

1
struct _MDL

2
{

3
[...]

4
  _EPROCESS *Process;

5
[...]

6
};

In our case, when the corrupted BufferEntry points to a userland address and we trigger an IORing operation, the kernel creates and attaches an MDL to our BufferEntry to map that address. Since the BufferEntry itself now lives in userland (as a result of our corruption), we can simply read its Mdl field directly from our process. We then use our arbitrary read primitive to dereference that MDL pointer and extract the Process field — giving us a valid _EPROCESS pointer for our process, which is all we need to proceed with the privilege escalation.

The leak proceeds in four steps:

(1) `RegBuffers[0]` now points to our fake `_IOP_MC_BUFFER_ENTRY` at a known userland address.

(2) We trigger an IORing operation — the kernel creates an MDL for our userland buffer and writes its pointer into our fake entry’s `Mdl` field.

(3) Since the fake entry is in our own process memory, we read the `Mdl` pointer directly from userland without any kernel primitive.

(4) We set the `Address` field of our fake entry to that MDL address, trigger another operation, and read the `Process` field from the MDL — giving us a valid `_EPROCESS` pointer.

Issues

At this point we have both an arbitrary read/write primitive and a kernel address leak. Before proceeding to the privilege escalation, however, we need to repair the corrupted state — releasing the IORing object without cleanup will crash the system.

ProcessBilled

During our overflow, we corrupted an important field in the pool chunk header: the `ProcessBilled` field, which stores a pointer to the process responsible for the allocation. If left uncorrected, this will trigger a blue screen when the chunk is freed.

The ProcessBilled value is an obfuscated pointer to an `EPROCESS`, this value is computed as follows:

`@EPROCESS ^ ChunkAddress ^ ExpPoolQuotaCookie`

`ChunkAddress` is the address of the corrupted pool chunk header, located at a known negative offset before the `RegBuffers` pointer we already know. 

`ExPoolQuotaCookie` is a global kernel value used to obfuscate pool billing pointers; to derive it, we use a second uncorrupted IORing object. 

Via our arbitrary read, we read its `RegBuffers` address from the `_IORING_OBJECT` and the `ProcessBilled` value from the preceding pool chunk header. Since we know our `_EPROCESS` address, we reverse the formula: `Cookie = EPROCESS ^ ChunkAddress_clean ^ ProcessBilled_clean`. 

We then compute the correct `ProcessBilled` for the corrupted chunk and write it back using our arbitrary write primitive.

With the formula in hand, the remaining step is locating the corrupted IORing object itself in memory so we can apply the fix.

We locate the IORing object by parsing our process handle table, found in the `_EPROCESS` structure, following the same logic as `ExpLookupHandleTableEntry` to retrieve the handle table entry, then using the same formula as in `ExGetHandlePointer` to transform it into an object pointer.

Buffer Entry reference

When we obtain our leak, a reference to our buffer entry located in userland is stored in the kernel which will lead to a crash when the kernel attempts to process it during teardown.

The fix is to release the `RegBuffers` registration. This causes the kernel to clean up the associated MDL as part of teardown, resolving the lingering reference. Releasing the MDL directly would not be sufficient — the MDL release is a consequence of releasing the `RegBuffers` entry, not a standalone action. However, releasing the registration triggers another issue, covered below.

Free user buffer

As we have corrupted a buffer entry, the kernel will try to release our userland pointer when closing the object.

The solution to this issue is to simply increase the reference count of our fake buffer entry.

This prevents the kernel reference count from ever reaching zero during IORing teardown, so the corresponding release function is never invoked on our userland pointer.

1
struct _IOP_MC_BUFFER_ENTRY

2
{

3
[...]

4
  int ReferenceCount;

5
[...]

6
};

Abuse Arbitrary R/W to get more privilege

To escalate our privileges, we steal the SYSTEM process token. Using our arbitrary read primitive, we walk the `EPROCESS` doubly-linked list to locate the SYSTEM process entry and read its `Token` field. We then use our arbitrary write primitive to overwrite our own `EPROCESS` `Token` field with the SYSTEM token value, granting our process SYSTEM-level privileges.

Conclusion

In this post we demonstrated a full local privilege escalation exploit against an up-to-date Windows 11 system, leveraging CVE-2025-13032 — a double-fetch vulnerability in Avast’s kernel driver. Starting from a controlled heap pool overflow in PAGED_POOL, we used the RegBuffers array of the IORing object as our corruption target, turning a single overwritten pointer into an arbitrary kernel read/write primitive. From there, we leaked an _EPROCESS pointer via the MDL attached to our corrupted BufferEntry, repaired the pool allocation header to avoid a crash on teardown, and completed the privilege escalation by stealing the SYSTEM process token.

CVE-2025-13032 has since been patched. We encourage all users to ensure their Avast installation is up to date. The full disclosure timeline is detailed in Part 1 of this research.

If you missed the first part of this research, which covers the vulnerability discovery and sandbox escape, you can find it here: CVE-2025-13032 — Entering and Breaking the Avast Antivirus Sandbox (Part 1).

What happens when you analyze college football like the CIA?

A couple weekends ago, the Illinois football team lost to Duke at home, 31–27. My Hinsley model immediately became much less optimistic about Illinois making the College Football Playoff.

But it became more optimistic about the offensive line and our new quarterback.

That sounds contradictory, but it's exactly what I wanted to happen.

For the past 15 years, we've worked with people whose job is to make judgments about uncertain futures: intelligence and government analysts, foreign-policy researchers, investors, and corporate strategists. This year I decided to try an experiment. I took the methodology we've developed for that kind of work and applied it to something considerably less consequential: assessing the fortunes of the University of Illinois football team.

To understand why, it helps to think about what an intelligence analyst actually does.

How intelligence analysts think

During the Cuban Missile Crisis, American intelligence analysts were trying to understand what the Soviet Union was doing in Cuba. They had a growing collection of evidence, but the difficult part was deciding what it meant. Analysts had to consider competing explanations, identify the observations that distinguished one from another, and revise their assessments as new evidence arrived. Eventually, U-2 photography provided much stronger evidence that the Soviets were installing nuclear missiles.

The stakes are obviously rather different, but the analytical problem is surprisingly general. Usually there isn't one fact that gives you the answer. There are several possible futures, a huge amount of imperfect information, and a smaller number of things that actually help distinguish among them. The analyst's job is to impose structure on all of this without becoming more certain than the evidence warrants.

You find versions of this problem everywhere. A foreign-policy analyst might be trying to understand whether a conflict will escalate. An investment analyst might be thinking about how geopolitics, regulation, or a new technology will affect an asset over the next decade. A government analyst might be assessing how another country will respond to a policy change. The useful question isn't simply, "What do I think will happen?" It's: What are the plausible ways this could turn out? What would have to be true for each of them? What should I be watching? And what new evidence would cause me to change my mind?

It's not broadly known, but for more than a decade, Cultivate ran a prediction market for the U.S. Intelligence Community, giving analysts a way to make and aggregate probabilistic forecasts about geopolitical and national-security events. More recently, our work has expanded beyond forecasting individual questions into the broader analytical process around them.

That's what led us to develop Continuous Probabilistic Foresight, or CPF, the methodology at the heart of Hinsley, our AI/human hybrid analysis platform. CPF starts with a strategic question and maps the range of plausible outcomes as scenarios. It decomposes the problem into the drivers and indicators that would make those scenarios more or less likely, makes assumptions explicit, and turns important uncertainties into resolvable forecasting questions. As new evidence arrives, those forecasts and the larger assessment can change with it.

The idea isn't to build a crystal ball. It's to maintain a structured, explicit view of an uncertain future, and to know why your view changes when the evidence does.

Which brings me back to Illinois football.

Building an intelligence model for Illinois football

Having grown up in Champaign, I'm a lifelong Illinois fan, and college football turns out to be almost comically well suited to this kind of analysis. A season is an uncertain future surrounded by an enormous amount of information that is constantly evolving. We have preseason recruiting, game results, injuries, competitor performance, statistics, coaching changes, on-field performance, preseason models, beat reporting, podcasts, and endless amounts of informed and uninformed commentary. We know some things with reasonable confidence, have strong opinions about others, and are almost certainly wrong about a few things we currently regard as obvious.

So instead of just following the season the way I normally would, I asked Hinsley to follow Illinois the way an analyst might follow a country, company, market, or strategic issue.

I started a couple months ago with the question I think most Illinois fans are ultimately trying to answer before a season: What is the ceiling this season for the University of Illinois football team?

From there, I let Hinsley get to work.

Its research agent began collecting information about the team: returning players, transfers, recruiting, injuries, coaching changes, position-group strengths and weaknesses, the schedule, preseason models, and so on.

Its findings were that this outside view was fairly optimistic. Illinois had won 19 games over the previous two seasons, and Hinsley's research suggested a 10-win regular season and a possible College Football Playoff berth represented a plausible ceiling. But there were obvious reasons it might not happen. Illinois was replacing Luke Altmyer at quarterback, returning only one starter on the offensive line, and replacing a lot of defensive experience under a new coordinator.

There was also a particularly interesting warning buried in the research: Illinois had won 13 one-score games over the previous three seasons. Maybe Bret Bielema's teams are unusually good at winning close games. Or maybe some of that was luck that wouldn't continue forever.

That's exactly the kind of thing I wanted this exercise to expose. Instead of saying, "Illinois has won nine games two years in a row, so they'll probably be good again," I now had a set of assumptions hiding underneath that belief.

The next step was to ask Hinsley to turn the big question into four scenarios for how the season could end, and generate initial likelihoods for each of those scenarios based on everything it knew at that point.

Friday Facts #447 - Factorio that you can touch

Hello,
we have very unusual Friday facts for you.

Journey to the physical world Pard

It all started in the hot summer of 2024 when we worked really hard to finish the Space Age DLC. The game was almost ready, but we wanted some feedback from our players. A decision was made to organise a playtesting event (FFF-427), where teams of players could play the game from start to finish and provide us with in-person feedback and criticism to deliver the best product you all deserved.

Since we are all geeks and love to tinker, we asked our friends at Prusa Research to join us and create some accompanying programmes and presentations for all attendees.

Being fans of Factorio themselves, they agreed to join, and the plan was put into motion. When the event started, we realised that it would be cool to print some Factorio models right at the event, so Fearghall quickly created a simple model of the Wriggler from everyone's favourite Gleba.

With the conclusion of our event, everyone returned to work on Factorio, but we all wanted to return to the idea of Factorio models in the real world. In the months following the release and subsequent bug-fixing and polishing period, we started working on some prototypes in our spare time. Several models were made, printed, and painted, but they lacked a concept. We let it sit for a while and brainstormed what would be the best direction. The answer was actually very simple: transport belts.

Belts are the core of every factory from the early hours to the very late game. Jarosław started designing the concept of a grid system where you could place every entity closely following the game. The first version of inserters soon followed. During this phase, we were printing countless prototypes. It soon became clear that we needed two versions of almost every model. One set of models is created with high precision, allowing push-fit assembly, and the other is made with high clearance in mind for people who want to glue everything permanently or have clearance issues with their printers.

While Jarosław was busy creating one player-built entity after the other, Fearghall started working on enemies. The first iteration of the biter spawner soon followed. Then came chests, followed by the player character, then a stone furnace… Our model set grew nicely, but we knew that we had to draw a line somewhere, because trying to release every Factorio entity would take too much time. With that in mind, we focused on the early game. We ended up with 15 model sets containing 65 individual models in 247 STL files.

1/17

Testing how far we can push the limits of model size.

2/17

Painting of the first enemy prototype to ensure that models will look good both in plain plastic and fully painted...

3/17

...Finished demolisher. The model was later scrapped because it wasn't really reflecting in-game visuals.

4/17

Several models in early stages of development. Belts were tested and fine-tuned for clearance issues. Enemies needed major updates.

5/17

Painting the biter spawner to determine whether the fully finished model adequately represents the original vision for the game...

6/17

...The biter spawner is now fully finished. Unfortunately, we realized that substantial changes are still needed.

7/17

Most entities went through a series of iterations. Countless print runs were associated with this process.

8/16

First test print of the biter. It was later resized, and it received many improvements for a smooth printing experience.

9/17

Fine-tuning of printer limits regarding potential model details on the spitter.

10/17

The final version of the spitter, finding enemies outside the world of Factorio.

11/17

Test print of the late version of the electric mining drill with a coat of silver primer.

12/17

Our catalogue of models was starting to gain momentum. At this point, we were confident that a public release was possible.

13/17

The biter spawner finally received a model update. It still wasn't finished, but we were getting much closer to the finish line.

14/17

The belt lane balancer is painted with model grass added. It was clear that complete dioramas are possible.

15/17

One of the earlier tests of walls and their integration into the grid system.

16/17

Testing how far we can push the limits of model sizes in the opposite direction. Joint modification was needed.

17/17

Finally, all the models are finished. Time to take some glamour shots.

Why 3D Printing? Pard

In the modern digital world, there is a decrease in physical memorabilia. Physical releases of games are now almost a relic of the past. We always thought about having something for fans and ourselves as a reminder of times that we spent with the game. We at Wube are a software company, not a toy manufacturer, and we have the best community in the world. Creating some expensive (limited) collector's items didn't sit right with us. We have decided to go with 3D models as a little "thank you for your support" gift so anyone can download them and print them for themselves.

A second equally important reason is that we love your creativity, and we want to give you a chance to iterate on things we created and give them your own spin. We can't wait to see it.

Reimagining of Factorio Jarosław

I will not lie, I slept on 3D printing for a while. It's probably for the best - I do have a passion for miniature-based board games and background as a 3D artist. So, when I finally took the risk and decided to look into it - well, let's say neither of my 3D printers has seen much downtime since then. Whether it's miniatures, card organizers, inserts or wall-high paint-stands, there is always something new to print. And of course, in case you are wondering, that does not make my "pile of shame" any smaller.

With this premise, I know you can imagine why my ears perked up when the first hints of 3D printed models for Factorio started circulating in the office. Little did I know I was soon to partake in one of the most fascinating endeavours I ever… well, partook in.

If you are a regular in these here parts, you will likely know all our entities start their life as 3D models. That might make it seem like translating them into prints would be an easy job - in all truth, I thought so, and I of all people should know better! The journey turned out to be a bit more complex than I originally imagined!

Transport Belts 1 Transport Belts 2 Belt Items 1 Belt Items 2 Belt Items 3 Belt Items 4 Chests 1 Chests 2 Chests 3 Chests 4 Chests 5 Walls

The most obvious problem concerning FDM printing is the issue of supports. If you are not well versed in how these printers work, you can imagine building a roof with LEGO bricks. An angled roof is easy - you just offset each next brick by a single row, and there is your roof. If you wanted to make a perfectly flat one, though? The bricks do not connect on the sides, so each next one would need to float in the air… unless there was something to support them from underneath. For our models, we wanted to avoid extra supports wherever possible - they can be visually distracting when made to stay, otherwise they can be a bother to remove. As such, part of my work was to find ways to separate our models into pieces that can be printed without supports and then connected into full models. Considering I am a fan of scale models and gunplas, it was an incredible experience to create my own!

Now, the other problem is a bit different in nature. How would I say it…? Factorio models tend to be really, really crazy (in the best possible way). Do you know the adage "what you see is what you get"? Well, yeah, no. Not at all applicable in this case!

Assembling Machine Electric Mining Drill Steel Furnace Stone Furnace Small Electric Pole

Factorio's isometric view gives a lot of space to experiment and play, and my fellow artists used that opportunity mercilessly. Sometimes what you see is, in fact, merely a collection of free-floating objects that just happen to look like a machine when looked at from the exact correct perspective. It rarely gets quite that bad, but the perspective is often a problem. There is a huge discrepancy between what the machines are and how they feel - all a function of the isometric view mixed with some creative light-and-shadow tricks. If you were to take a measuring tape to the mining drill for example, you might discover the North-South and the West-East variants are not quite identical - they do look like it, though! That problem goes double for height of the entities - in game, it is often restricted to fit in the grid-space of one tile, yet made to look much taller than that suggests. That created a unique and somewhat contradictory problem: designing 3D models that looked like Factorio models look, and not how they are.

There is one more issue recreating models made for an isometric game - and that is the dark spaces. Just like the dark side of the Moon, the back-sides of our machines are filled with horrors and sleeping eldritch-gods. Do not worry, though - I did replace them with freshly imagined, safe machinery :)

Partaking in this journey was surely one of the most interesting challenges in my career. I would love to thank Wube and Prusa for making it possible - as well as for the enthusiasm they shared! I honestly do hope more studios decide to share their work in such a way (looking at you, certain team of astronauts!)

Special thanks go to my colleague, Pard, for his insatiable hunger to test my models. May thy filament never runeth dry, good sir!

Inserters 1 Inserters 2 Inserters 3 Inserters 4 Inserters 5 Inserters 6

Designing enemies Fearghall

While Jarosław toiled away hand crafting belts and assemblers and failing his "lazy bastard" run in the process, I set about the task of bringing the "organic" entities to life. Thankfully for me, I was able to avoid the pain of the isometric perspective because most of the organic entities have fully rotating spritesheets, and so the models are complete from most angles without too many tricks of the camera.

However, the models were still designed for pre-rendering only, and as such had some unique challenges. For example, the biter spawner was covered by an invisible skin, and almost all of the detail in the in-game images was confined to the texture data - something that cannot be transferred to a 3D printable file. However, after a week or so of cleaning up and baking out the model, it now prints very well with no support or issues.

It also became immediately obvious that most of the organic prints could not be made support-free like Jarosław has done so well with all the machines. The geometry is simply too complex, and many of the parts would be far too small to print and attach together. I still endeavoured to make them easily printed, however. The idle engineer pose, for example, has printed without supports many times, and the worm poses (aside from the 90-degree attacking pose) print entirely without support. But alas, there are some things which are simply impossible to print without support, such as the running pose.

Biter Spawner 1 Biter Spawner 2 Worm 1 Worm 2 Worm 3 Worm 4 Worm 5

One of the main issues with the character and biter family was the damage left by the support interface. The surface where the print and support meet can be quite ugly, so I was keen that none of the models should require so much support as to impact the visual quality.

With the engineer it was quite simple, the backpack needed to be split off, and there needed to be a slot and pin added, but otherwise a slight angle was all that was needed for decent printing.

Player Character 1 Player Character 2 Player Character 3 Player Character 4 Player Character 5 Player Character 6

With the biter family however, there was a little more work. There was the big issue of the legs: they would absolutely not have printed in the "correct" orientation, and splitting in the centre of the biter and leaving a large seam was an absolute non-option. So the only sensible solution was to print them upside down! Simply printing them like this as a single part would likely have worked, but would have left horrible scarring on the most visible surface of the model, so I decided the shell needed to become its own part so it could be correctly oriented. This also had the benefit of meaning the shells are easily printed in the correct colour for the biter size!

Biter 1 Biter 2 Biter 3 Biter 4 Biter 5 Biter 6

With that in mind, I could arrange all the biter legs to require as little support as possible in the new orientation, and merged a few of the legs for added strength - bearing in mind that they would need to stand on their own legs in reality. In order to make the shells attach, I sculpted in some extra reliefs and ensured both surfaces had just the right level of clearance. The final result has almost no visible support interfacing, and a very satisfying "snap!"

Spitter 1 Spitter 2 Spitter 3 Spitter 4 Spitter 5

Where to download? Pard

All models are available to download right now from our Prusa Brands profile on Printables. Special thanks go to Štěpán Feik, who helped us make all of this happen. From joining us at our playtesting event to helping us cross the finish line this morning, he has been part of the journey throughout. You can read his take on our collaboration here.

As always, let us know what you print at the usual places.

I finally built Mathy, a little project I’ve been thinking about for a couple years. It’s free, and no account is required if you want to try it on iOS or web.

I use Math Academy daily, which is the best way to learn math. But I wanted a mobile-friendly way to drill math and weak areas on the go that was more convenient/less cumbersome than Anki cards.

It’s also a nice way to do something productive (feel like I’m building towards something) in those spare moments here and there rather than scrolling.

The gist of Mathy is counterintuitive: If it works, you’ll use it less. For example, as your topics become automatic, recommended practice sessions can get weeks apart or longer (you can still practice whenever you want).

Why Mathy

First, I’m a huge proponent of Math Academy, their philosophy and everything Justin shares about learning (check it out), so I won’t recap that here.

I want to be much better at advanced math, but I’m not a naturally gifted mathematician and bump into a ceiling.

I also intentionally don’t put in as much time and focus as I could. It’s a hobby. Though I have consistency over a long period of time, my daily volume is quite low. But I’m ok with that, and I’m quite happy with the progress I’m making given the time I put in. It fits well into my life.

So to give myself an advantage, I wanted an easier way to drill math to make more things automatic. I’m curious how far I can go there and how much automaticity can raise my ceiling given my other constraints.

And though Math Academy is the best way to learn math, it focuses on lessons, reviews and quizzes, not drilling like flashcards. But there are plenty of other tools for that, which I experimented with.

The most popular seemed to be Anki cards. And though Anki is a flexible tool, it wasn’t ideal for math. It was also cumbersome to create and maintain cards.

I didn’t find any other good math alternatives out there, so here we are.

How it came together

I’d been thinking about something like this for a couple years and it became quite acute as I struggled memorizing trig ratios, formulas, etc. especially since there’s such a wide variety of content the deeper into math you venture.

I’d been taking notes on what I wanted over time, so I had a decent starting point. Then finally one weekend I just put them into a product requirements doc to see what this would look like. I liked it, but didn’t have time, so forgot about it for a few months.

In parallel over the last year I’ve been going deep on agentic coding, burning hundreds of millions of tokens learning what AI was capable of, the best ways to build with it, refining my workflows, etc.

But most of the things I worked on were quite complex with a lot of moving parts. I wondered how “easy” it’d be to build a simpler product. And so I decided this would be the project.

The core app came together in a few days, but using the app revealed dozens of little rough edges that didn’t quite feel right and revealed deeper issues I had to iterate on.

So though the core app came together fast, getting it to a point where I was somewhat happy with it took a couple of weeks as I experimented with a lot of approaches.

It still has plenty of rough edges, so if you have any feedback let me know.

How it works

Mathy is meant to be simple for users and complement your existing learning. An example of how it fits into your learning program:

1. Learn the concept with something like Math Academy and get initial practice.
2. Pick the topic (by grade, course or topic) and Mathy tells you when to practice again based on how you do (using FSRS).
3. When you’re ready, you can do timed “sprint” challenges to try and beat your best score. Future plans may let you compete against others.

There is no diagnostic or other placement like Math Academy has. It’s up to you to pick the topics. So I recommend aligning it with what you learned in Math Academy.

Next I’ll share more about how Mathy works.

Product focus & constraints

Constraints help, so I was quite deliberate with what it is and isn’t for:

• It’s only for building math automaticity, NOT learning new concepts. It’s for practicing what you already know.
• Only things you can do mentally, mostly direct recall and short procedures that become roughly 3–5-second responses with fluency.
• Offline & mobile optimized: You can use the app on the go and core features must work offline in the mobile app (iPhone & iPad support).
• Minimum effective dose: The goal isn’t to practice every day, just to keep you ‘automatic’ at your selected topics. The better you get, the less you need to practice. Some days/weeks you may not need any. The goal is to stay automatic and give you flexibility. So, once you reach automaticity it’s just for periodic maintenance when you have a few free minutes.

Platform

Instead of building a native iOS app (in Swift) like my last apps, I opted for React Native so it could support iOS (iPhone and iPad) and web from the same codebase. If there’s traction, I may replace it with native iOS and Android apps later, which would be fun to convert.

Practice, grading, scheduling, and progress storage all run locally, without an account or a practice backend. Sending help or feedback requests requires an internet connection. The app is free to use.

Progress currently stays on the device or browser where you practice. It doesn’t sync across devices (yet), and deleting the app or clearing browser data can erase it.

Note: I plan to add optional account support so you can save progress, sync it across devices, etc. I’d also like to support mini competitions and leaderboards. But these are TBD, the MVP is local only as described.

Spaced repetition algorithm

I used FSRS for scheduling. FSRS was created by Junyao Ye, who is also a Math Academy user! Here’s an explanation of the algorithm from the FSRS project.

FSRS determines memory scheduling state and when something should be reviewed. Mathy’s own queue policy decides how relearning, due work, new targets, weak material, maintenance, reinforcement, confusability, etc. get composed into practice.

The app’s homepage shows recommended practice and upcoming review dates. A separate “Play your way” option lets you practice whenever you want in multiple modes (Daily Math, Timed Math, Endless Math, or 60-Second Sprint).

Programmatically generating problems

This became a lot easier thanks to the focus constraints I mentioned above. I limited myself to things I could deterministically verify.

Math Academy has a killer team creating and reviewing instructional content, which makes sense for teaching advanced math. However, this is just an automaticity app and NOT teaching new material. So, if I can’t verify it deterministically, it doesn’t belong in this app.

It uses a curated bank of recall targets, including fixed facts and formulas alongside parameterized problem families. Those families generate variants within checked bounds, with deterministic answer keys rather than AI-generated answers.

Intuitive presentation with deterministic verification

Mathy presents each problem with clear typography and mathematical formatting, while keeping grading fully reproducible. The same problem variant always resolves to the same answer key, regardless of how the practice queue is shuffled or ordered.

Under the hood, every recall target and generated variant has a stable ID. That lets it identify and reproduce the exact problem a learner saw, which is useful for grading, debugging, and responding to problem reports.

Normalizing inputs

For typed numeric answers, the app normalizes submissions before comparison:

• Trim whitespace
• Unify symbols
• Parse via deterministic numeric parser rules
• Convert to a canonical form (fractions, decimals, percentages, radicals where allowed).

For a question that accepts equivalent numeric values, inputs such as 1/2, 0.5, 50%, and 2/4 can resolve to the same value. Questions that test a specific representation can still require a percentage, decimal, or simplified fraction.

Then if the syntax is invalid, it returns a clear invalid-format error before checking the value.

Accepted forms depend on the question. Equivalent values are accepted where appropriate, but a question specifically testing a fraction, decimal, or percentage can require that form. Choices and structured answers use their own deterministic checks.

Measuring automaticity

It tracks repeated correct behavior over time, not just one-off correctness.

It also uses thresholds on accuracy, recent correctness density, response time, spacing, and cue coverage, with a lapse/suspension rule for failure and re-entry review rules when regression appears.

Admittedly, I’m still iterating on this. Automated simulations already exercise years of practice, but real-world histories will help me understand how well the model matches people’s experience over time. This is another reason why I want to add cloud/sync since that supports longer, consistent histories vs. local storage.

Topics

The app’s model is a graph of Domain > Topic > Concept > Recall target > Prompt variant.

It includes topics that map to the user’s courses/grade pathways for easy browsing and filtering.

I expect there are meaningful gaps, so if you see something missing let me know via in-app feedback.

Topic coverage & variation

The goal is sufficient coverage within each topic, with meaningful variants where they help. Mathy already balances eligible material across topics and avoids recent repetitions when alternatives are available.

Topic browsing & selection

Users can browse for the topics they want to practice by course, grade or view all topics. Mathy only includes material for Common Core suitable for short automaticity practice, so there are meaningful gaps.

Keeping it fun & engaging

The focus on automaticity gave it the opportunity to feel more like a game than a learning app. So it has light gamification using levels, XP, math medals and streaks. I had built most of those into a habit tracking app I made, so this was an easy win using that as reference architecture.

The on-track streak follows your recommended schedule, so days when no practice is needed count too, without opening the app.

The 60-second sprint mode is especially fun, and if I add account creation later it’d be fun to host leaderboards for best scores and such.

In the future I’d also want to add more fun stuff like math tricks, olympiad hacks and more challenges. This is TBD more out of curiosity, but I’d like to explore more here if I get time and if the app gets any traction.

The last 10% 90%

The last 10% always takes the longest, doesn’t it? The core app came together in a few days, then the app was ready for Apple App Store submission after about two weeks, but there’s still more work to do.

Much of it came down to how the app feels to use and how it delivers what content when.

The nice thing about being busy over most of those two weeks traveling and such was that I could make some fixes, push them to the dev version on my phone, take notes on rough edges, and push more changes.

I even had my kids use it on downtime during plane flights, etc. which was good feedback.

That allowed me to make dozens of rounds of improvements in a relatively short timeframe.

There’s still a LOT more work to do on:

• Content quality and variety: Deterministic checks make it easier to expand suitable problem families, but coverage, wording, difficulty, and useful variation still need work.
• Content presentation, input and interaction: I’m designing for mobile, so the form factor is a challenge, particularly on advanced content, but it’s fun to experiment with and test different approaches.

I’m less familiar with math from this perspective, so these have been exceptionally fun problems to work on and learn about. They’re also quite tractable, so it’s just a matter of focusing on them once I get some time.

Unexpectedly fun

I’m proud of how the app is coming together but the most unexpected thing was how fun it is to work on. I can’t express how much I enjoyed building the app and thinking through the experience.

I’m a product guy, so it’s no surprise that’s the funnest part. But lately I’d spent so much time on more complex projects focusing on architecture, backend, wrangling bleeding edge SDKs, testing new models, etc. that it took building something simpler to give me the opportunity to focus on frontend again. I missed it.

If this gets any traction I’m quite excited to build out other experiences like multiplayer sprints and other math challenges that work well in mobile’s simpler form factor.

What’s next

I’m tentatively planning to work on these next:

• Account creation to back up progress and sync it across devices. I originally wanted to keep it offline, but losing your progress/badges is a real bummer.
• Improving content coverage, variation, input and display, particularly for advanced topics.
• Reminders when it’s time to practice, especially when there are days/weeks in-between sessions.
• Challenges/leaderboards to compete against others, e.g. in 60-second sprints and other competitions if the app gets enough of a user base.
• Wish list: Math Academy integration so practicing gives you a tiny amount of XP, areas you need to improve automaticity automatically sync to the app, and areas you need to learn suggest the right Math Academy course.

Thankfully, it’s a relatively simple app, so the future feature list isn’t too long.

The app is currently free, but if it gets traction or I launch cloud/sync support then I’d probably introduce an optional ~$1/mo. paid plan to cover associated hosting costs for those who want the cloud features, challenges, leaderboards and other fun multiplayer content.

This has been a fun little side project. I learned a lot. If you want to try it out, as of this writing the iOS app is still in app review it’s available in the iOS App Store here and the web version is available at mathy.game.

Update:

There was a good discussion on Hacker News here, where I went more into some of the technical details in response to a question.

We implemented a parquet writer in DataHaskell/Dataframe. Using it is simple; you need only pass your dataframe into the writeParquet function which writes a parquet file with sane defaults for row group and page sizes. An example:

import qualified DataFrame as D
import qualified DataFrame.Functions as F
import DataFrame (as, (|>))

main = do
    sales <- D.readParquet "sales_data.parquet"
    
    sales
        |> D.groupBy ["product"]
        |> D.aggregate [ F.sum (F.col @Int "amount") `as` "total"
                       , F.count (F.col @Int "amount") `as` "orders"
                       ]
        |> D.writeParquet "total_orders.parquet"

If you need more fine-grained control over the parquet file you’ll want to use writeParquetWithOptions. Read on to see what those options are and how they affect the final file.

Parquet For Haskell

For Haskell to interoperate with the data ecosystem, it must be able to understand the standard formats in use by that ecosystem. For a long time our options for serializing data1 in Haskell came down to CSV, JSON, or to simply dump it into a ByteString. While CSV and JSON have their place, they come with considerable disadvantages as the volume of data expands. They are slow and cumbersome to read, write, and query, and any custom homegrown formats tend to lack interoperability with standard data science tools.

The parquet format trades simplicity for efficient storage and querying of the data stored therein. We want our data to have high compression ratios and we want to minimize reading data irrelevant to our specific query. The ability to read and write parquet files for long term storage, sending over the network, or for interop with other programs, especially given the universality of Parquet in the data science ecosystem, is a rather useful tool that we should like to have in a Dataframe library.

Parqué?

Feel free to skip this section if you already know the structure of a parquet file.

Parquet files are a series of row groups followed by metadata at the end of the file with information that allows readers to locate relevant column chunks and pages. It also contains useful information like statistics and bloom filters so that, for example, a reader / query planner can decide whether or not to read a specific row group.

Each row group is a collection of column chunks, each of which contain the same number of rows. Each column chunk is a series of data pages. Since each column chunk is a series of pages, and each row group is a series of column chunks, the final file simply looks like a series of pages from each column one after the other. We’re able to make sense of it all by using the metadata to identify the offset and size of each row group and column chunk.

A data page is where we actually store all of our data. It consists of first the page metadata, describing its encoding, the number of values, the statistics, among other things. There are actually two versions of the data page with subtle differences.

Next we have definition levels and repetition levels. They’re an inexpensive encoding of nullability and nested structure. A detailed description of definition levels and repetition levels is out of scope for this article; refer to the Dremel paper for that. For our purposes we currently only support writing definition levels up to one to denote nullable values.

Finally we have our actual encoded and compressed data (depending on which data page we’re using we compress either just the data or both the definition/repetition levels and the data). The encoding is determined per page, while compression is determined per column chunk.

The Implementation

What follows is a technical discussion of the design of the Parquet writer and the tradeoffs we made.

Write Options

It follows from the description of the parquet format above that a writer needs to expose the appropriate set of fiddle factors to allow the user to tune the writer to produce a parquet file that is efficient for the specific data it is being applied to. That is, we want a parquet writer that compresses well with appropriately sized row groups and pages, so that a reader can effectively apply parallelism, projection pushdown (read only relevant column chunks), predicate pushdown (read only relevant row groups/ prune irrelevant pages), IO pushdown (read only relevant files, assuming the data is chunked into multiple files), etc2.

data ParquetWriteOptions = ParquetWriteOptions
    { pageSize :: !Int
    , rowGroupSize :: !Int
    , batchRows :: !Int
    , subBatchRows :: !Int
    , compressionCodec :: !CompressionCodec
    , strategy :: !WriterStrategy
    , maxRowsPerFile :: !(Maybe Int)
    }

Both pageSize and rowGroupSize are the target size in bytes of each page and each row group. But each column chunk in a row group must contain the same number of rows, and depending on the specific data being encoded, the encoding, and the compression algorithm we’re using, each column chunk will hold a different number of rows before reaching the target size. The same holds true for each page. So how do we ensure our page size target, our row group size target, and have the same number of rows in each column chunk?

We must consider both the target pageSize and rowGroupSize to be best effort; they could be somewhat above or below the target. We run our columns through batches of size batchRows and check the state of the row group after each batch. So each row group contains an integer multiple of batchRows rows; pages also will contain an integer multiple of subBatchRows rows (with the exception of the final page and the final column chunk). Sub-batching at the page level allows us to reduce the amount of IORef book-keeping we have to do after each write, granting us significant speedups.3

Memory

Due to the constraints described in the previous section, it’s difficult to know how big to make our buffers ahead of time. This is even more true of column chunk buffers – each column will fit a very different number of rows / pages in the same amount of data, and we may expect that some column chunk buffers will be significantly bigger than others and dominate the space per row group.

So we must be able to grow our memory. One of the more convenient ways of handling raw memory without resorting to the FFI is MutableByteArray.

data MemoryBuffer = MemoryBuffer
    { arrayRef :: !(IORef (MutableByteArray RealWorld))
    , positionRef :: !(IORef Int)
    }

For our implementation we use pinned ByteArrays, as we would like to convert it into a Ptr Word8 when it’s time to flush into either another buffer, for example when flushing a page buffer into a column chunk buffer, or into a file, as we would when flushing a row group to file.

Using pinned ByteArrays results in a slight complication when trying to grow the memory buffer. We must not use the grow function provided by Data.Primitive, instead we must allocate a new pinned ByteArray and allow the old one to be GCed. One might be worried about heap fragmentation because a single pinned object in a 4KB GHC block can keep the whole block alive but we expect that our buffers will tend to be much larger than 4KB. Furthermore grows ought to be rare especially after the first few Pages and the first RowGroup.

ensureCapacity :: MemoryBuffer -> Int -> IO (MutableByteArray RealWorld)
ensureCapacity buffer needed = do
    array <- readIORef buffer.arrayRef
    maxSize <- getSizeofMutableByteArray array
    if needed <= maxSize
        then pure array
        else do
            position <- readIORef buffer.positionRef
            grown <- newPinnedByteArray (needed + (needed `div` 2))
            copyMutableByteArray grown 0 array 0 position
            writeIORef buffer.arrayRef grown
            pure grown
{-# INLINE ensureCapacity #-}

We also write helper functions for writing Word8s, Word32s, Word64s, Int32s, Int64s, Integers, Floats, Doubles, and ByteStrings to a buffer. Finally, we have a flushBufferToBuffer :: MemoryBuffer -> MemoryBuffer -> IO () and a flushBufferToFile :: WritableBinaryHandle -> MemoryBuffer -> IO ()4.

The Core Loop

Here we provide a high level sketch of the main loop of the Parquet Writer. There is considerable complexity we omit in favor of explaining the design as simply as possible. For the full implementation refer to Writer.hs.

At a high level we can think of the Parquet writer as an effectful fold over the dataframe5 (Also known as reduce in some languages or, more rare, accumulate). A fold can be thought of as a loop, that is, we iterate over the rows in our dataframe. Folds are usually pure, but ‘effectful’ means that each iteration produces some kind of side-effect, in our case writing to and from memory and to disk. Finally the fold produces a final singular result from the iteration, and that is the file metadata that we must append to the end of the file.

First we must define the state threaded through the writer to accomplish this:

data ParquetWriterState = ParquetWriterState
    { outputFileHandle :: !WritableBinaryHandle -- newtype over a handle specifically for us to write to
    , columnChunks :: !(VB.Vector ColumnChunkState) -- effectively our row group buffer
    , currentFileOffsetRef :: !(IORef Int64)
    , scratchBuffer :: !MemoryBuffer
    , rowGroupMetadataRef :: !(IORef [RowGroup])
    , rowNumberRef :: !(IORef Int)
    }

data ColumnChunkState = ColumnChunkState
    { columnName :: !T.Text
    , nullable :: !Bool
    , schema :: !SchemaElement
    , encoder :: !Encoder
    , buffer :: !MemoryBuffer
    , uncompressedBufferSize :: !(IORef Int64)
    , pageState :: !PageState
    }

data PageState = PageState
    { pageBuffer :: !MemoryBuffer -- Each column has its own pageBuffer
    , definitionLevels :: !DefLevels
    , currentRowCount :: !(IORef Int)
    }
    
-- In Encoder.hs

data Encoder = Encoder
    { encType :: !ThriftType
    , convertedType :: !(Maybe ConvertedType)
    , logicalType :: !(Maybe LogicalType)
    , encodeValue :: !(MemoryBuffer -> Int -> Int -> IO (Int, Bool))
    , finishValues :: !(MemoryBuffer -> Int -> IO Int) -- in case cleanup or post processing is required
    }

The buffers for each ColumnChunk are flushed into the final file while the individual page buffers are flushed into ColumnChunk buffers (or more accurately are used to assemble the data that is eventually flushed into the ColumnChunk buffer as we need to flush the data along with the page metadata and definition levels).

If we dispense with the ceremony of setting up the initial writerState and other variables, our core loop is quite simple

loop :: Int -> IO ()
loop rowNum
    | rowNum >= endRow = pure ()
    | otherwise = do
        let batchEnd = rowNum + min interval (endRow - rowNum)
        writeBatch rowNum batchEnd
        size <- bufferedSize writerState.columnChunks
        when
            (size >= options.rowGroupSize)
            (flushRowGroup options writerState) -- Write to file
        loop batchEnd

then writeBatch:

writeBatch :: Int -> Int -> IO ()
writeBatch rowNum batchEnd
    | rowNum >= batchEnd = pure ()
    | otherwise = do
        let count = min options.subBatchRows (batchEnd - rowNum)
        forM_ writerState.columnChunks (writeRows options scratchBuffer rowNum count)
        modifyIORef writerState.rowNumberRef (+ count)
        writeBatch (rowNum + count) batchEnd
        

The scratchBuffer is a re-usable buffer that we’ll later use to assemble the page as the individual page buffers hold only the values, but our definition levels live in a different buffer and the page metadata doesn’t exist yet. It is the scratchBuffer that is eventually flushed to a ColumnChunk buffer.

The internal workings of writeRows are somewhat similar to the loop we saw earlier.

rowWriterLoop !options !columnChunkState !end !size !position !row
    | row >= end = writeIORef columnChunkState.pageState.pageBuffer.positionRef position
    | position + options.pageSize > size = do
        let page = columnChunkState.pageState
        writeIORef page.pageBuffer.positionRef position
        arr' <- ensureCapacity
                    page.pageBuffer
                    (position + max
                                    options.pageSize
                                    ((end - row) * 64)
                    )
        size' <- getSizeOfMutableByteArray arr'
        rowWriterLoop options columnChunkState end size' position row
    | otherwise = do
        let page = columnChunkState.pageState
            encode = columnChunkState.encoder.encodeValue
        (position', notNull) <- encode page.pageBuffer position row
        when columnChunkState.nullable $
            pushDef page.definitionLevels (if notNull then 1 else 0)
        rowWriterLoop options columnChunkState end size position' (row + 1)

Note that we only handle definition levels up to 1 for our writer and we don’t support definition levels greater than 1 and repetition levels greater than 0 as, currently, we specifically only want to support dataframes which have flat schemas.

When a page is filled up we can flush it to the parent columnChunkState.buffer.

pageResidency <- readIORef page.pageBuffer.positionRef
defResidency <- readIORef page.definitionLevels.dlBuf.positionRef
when
    (pageResidency + defResidency >= options.pageSize)
    (flushPage options scratchBuffer columnChunkState) 

Future Work

Currently the Parquet Writer supports only

• Snappy and Uncompressed compression
• Plain Encoding
• Only definition levels up to 1

The next step would first be to support the full gamut of compressions and encodings offered by Parquet. The current Writer is also completely single threaded, so we may explore adding concurrency and/or parallelism in limited ways – so long as there’s a performance justification for doing so.

Currently optional metadata fields like statistics for columns/pages and bloom filters are not recorded. As we mentioned earlier, these are very useful for parquet readers to optimize querying the data in a parquet file.

Finally, currently all of our buffers live in memory while they wait to be flushed to disk. In memory constrained systems writing Parquet files that require particularly large pages/rowgroups, we may end up running out of memory (an entire row group must be held in memory in its entirety). So we plan to implement a two pass strategy where we use much smaller in memory buffers and write to on-disk temporary files instead of keeping data in memory.

1. In this article when we speak of data we usually mean copious amounts of columnar data that one is wont to put in a dataframe. ↩

2. The above optimizations also require the implementation of multiple encodings, statistics, bloom filters, and so on. The current implementation focuses on getting the basic writer up and running and the optional parts of parquet files are planned as future work. ↩

3. There is a possibility, if there are columns whose individual entries are disproportionately large, that our sub-batching/batching overshoots the target pageSize and/or rowGroupSize by orders of magnitude. A mechanism to mitigate this effect may need to be implemented if users start running into this issue. ↩

4. The flushBufferToFile implementation lead me down a somewhat interesting rabbit hole. By my meagre understanding: it turns out that by default when writing to file we’re actually writing to the OS page cache, which marks a page as dirty, and if the ratio of dirty pages exceeds a certain value, the OS throttles the process producing dirty pages while writeback catches up, which, as we’re writing row groups synchronously, forces us to wait while the Kernel sorts things out (hPutBuf uses write(2) internally, if I’m not very much mistaken).

   So it behooves us to write data in large enough chunks that we write more data with fewer syscalls (though very large chunks will have diminishing returns). After using dd to run some tests on my computer, I settled on 256 KiB chunks as a reasonable default for our Parquet Writer.

   -- for use in a single threaded writer
   flushBufferToFile :: WritableBinaryHandle -> MemoryBuffer -> IO ()
   flushBufferToFile (WritableBinaryHandle h) buffer = do
       array <- readIORef buffer.arrayRef
       position <- readIORef buffer.positionRef
       withMutableByteArrayContents array $ \ptr -> do
           let chunkSize = 262144
               go offset
                   | offset >= position = pure ()
                   | otherwise = do
                       let n = min chunkSize (position - offset)
                       -- the handle is not block buffered
                       hPutBuf h (ptr `plusPtr` offset) n
                       go (offset + n)
           go 0
       writeIORef buffer.positionRef 0
   

   It’s unlikely we would need it, but should we ever desire writing to disk directly without the mediation of the kernel, in spite of the extra effort to make it cross platform, O_DIRECT beckons. The rabbit hole goes deeper when we consider multithreading and avoiding losing data due to a crash. ↩

5. We say ‘Effectful fold over the dataframe’, but if you look at the code we don’t actually use a foldM. Mostly the code is recursive and a forM_ is used. This is the local minimum we landed on to make it convenient to do the batching and sub-batching required by the writer. However, the statement that it is an effectful fold is still accurate as forM_ is itself a special kind of fold and folds abstract over recursion (See Graham Hutton’s paper “A Tutorial on the Universality and Expressiveness of Fold”) ↩

I found a model labeled azure/muse-special while Muse was building my website. So I dug deeper.

This is Part 2 of digging through the Muse filesystem after my article hit the front page of Hacker News this week.

In this article I focus on a model I found in my logs called muse-special, and confront the question: does Muse actually use OpenAI and Claude models behind the scenes?

One odd session

Muse records which model each agent session uses.

Nearly every session log in my VM was routed to Meta’s internal model, called Avocado.

But one subagent used a model named azure/muse-special.

Interesting...

Figure 1. Sessions in my VM grouped by model. Everything is Avocado except a single azure/muse-special session on September 21. Click image to enlarge.

Following the name

This made me curious, so I searched across the repo inside Cursor and found this:

“GPT Responses model client via MAGI native Azure OpenAI lane.”

OK.

The model catalogue seems to list azure/muse-special then azure/gpt-5.6-sol.

So I searched my session transcripts...

Here I found two details that stood out:

1. The signature is tagged gpt_responses_v1 and contains an encrypted payload starting with gAAAAA (which OpenAI uses).
2. Tool call IDs used call_ followed by 24 mixed-case characters.

This was different from all the other lines that the Avocado sessions printed (call_ followed by 32 hex characters).

Figure 2. Lines from the muse-special transcript: a call_ ID in the OpenAI style and a gpt_responses_v1 signature with an encrypted gAAAAA payload. Click image to enlarge.

These little details tell me that the muse-special model is possibly an OpenAI model or OpenAI’s Responses API.

So is muse-special an alias for a GPT model served through Azure?

The files and logs don’t tell me exactly which GPT model, or why it was selected by the subagent in the first place, but let’s take a step back and explore further...

The model catalogue

The broader model catalogue that is shipped with Muse’s agent daemon lists about 15 versions of Avocado, plus:

• Claude Opus 4.6 / 4.7 / 4.8
• Sonnet 4.6 and Haiku 4.5
• GPT-5.5 and GPT-5.6 variants via OpenAI, Azure and Codex
• Kimi K3 through Fireworks and Meta-hosted routes

Figure 3. My summary of the model IDs shipped in the hatch daemon, grouped by family. A shipped ID means the runtime can address it, not that it was used. Click image to enlarge.

The Anthropic plumbing

The Claude support goes beyond just the model ID and includes an Anthropic client with request handling, prompt conversion and streaming parsers:

• anthropic/request_flow.rs
• anthropic/convert_prompt.rs
• anthropic/parse_sse_stream.rs

OK, so now we’re kind of wondering... why?

There are API key files present for Anthropic, OpenAI, etc., with access restricted to the inference-proxy service.

...But there’s also a proxy kill-switch setting in the env.

Figure 4. JARVIS_ANTHROPIC_BASE_URL_REVPROXY_OVERRIDE=0 in the runtime env. The comment calls it a live kill switch, not stale config. Click image to enlarge.

Why ship all of this?

Now, there are a few reasons for this, I guess.

1. The first would be that an OpenAI or Anthropic model just does a superior job at a certain task that Muse can’t fulfill right now, and they selectively route for that.
2. The second is that all these VMs are shipped with the ability to A/B test model responses, tool calls, etc. for the purpose of distillation and RL.

Distillation or RL? Maybe. Idk.

This leads us to the truth, which is that the model behind Muse is ultimately a server-side choice.

The runtime has clients for multiple providers, which gives Meta the ability to change routing without asking users.

In my case there was only a single outlier session that didn’t use the Avocado (Meta) model, but the infra is there to.

Is Meta distilling?

Wait, so is Meta distilling from the other frontier labs?

(Getting technical. tl;dr: No.)

With the muse-special model the raw reasoning is encrypted. The daemon stores it to send back to Azure on the following turn. In the binary it explicitly says that the encrypted reasoning cannot use the RL completion-server override.

So what Meta can see here is only the reply, the tool calls, and a short reasoning summary when OpenAI/Anthropic returns one. The raw chain of thought is encrypted and the RL server refuses those blobs. There is no indication that Meta copies OpenAI or Anthropic weights.

Avocado models are treated differently, however. The thinking text is written directly into the transcript, with an empty signature, and available for RL use.

So Avocado models, according to the privacy note and repo, do indicate that conversations can be used to develop AI at Meta unless you opt out. (Makes sense.)

Closing thoughts

This is my own exploration of what has been a very cool release from Meta.

My best guess is that muse-special is an OpenAI model served through Azure.

Whatever you think of Meta, the talent they brought onto this project deserves credit. They took a different approach in a world full of chatbots and search bars, and the exec team’s response to my first article, which got some eyeballs, has been pretty amazing too, as has their willingness to reach out to a nobody and explain their thinking.

It’s not every day you get to look inside the filesystem of a product that could reach hundreds of millions of people.

Seeing inside a runtime cell gives us an early look at where this whole personal agent thing might be going, and it’s been really fascinating to read through all of it this week.

If you worked on Muse at all, please feel free to reach out. I’d love to learn more and perhaps contribute.

Things seem to be moving fast. Not really breaking, yet.

pete at mouse dot dev

-Pete

@heypeterjames