TheDaily Front

Kolibri is an open-weight large language model (LLM) from Aleph Alpha for German and English: a mixture of experts with 78 billion parameters that only uses about 3.5 billion of them for each token it reads or writes. It came out on 3 October 2026 under the Apache 2.0 license, the weights are on Hugging Face, and it was trained from scratch on infrastructure in Germany and Finland. (Kolibri is German for hummingbird, which is cute for a model whose whole trick is being light.)

I live in Germany. At SmashingConf New York in 2024 I told the room what I’d heard in the US when I said where I’m based: “you regulate, you don’t innovate.” It hurt to hear. What I wished for on that stage was the middle, “the right balance between innovation and regulation around data privacy, data stewardship, environmental constraints and energy requirements.” Kolibri is a pretty direct answer to that: a German team built it with the EU AI Act in mind “from the ground up”. In Aleph Alpha’s own evaluation it scores above every compared model of its size in both languages.

Huge congrats to everyone at Aleph Alpha who built it, my good friend Michael Hofmann among them!

This post is about how Kolibri works, where it’s strong, where it isn’t, how to run it and when to pick it. Everything here comes from Aleph Alpha’s 189 page technical report, the model card and their launch post, plus one experiment I ran on its tokenizer.

What is Kolibri?

Parameters	78.1 billion in total, 3.46 billion per token (4.4%)
Languages	German and English
Context	262,144 tokens natively, tested up to 1,048,576
License	Apache 2.0 for the weights and configuration files (Aleph Alpha keeps the rights to its training code and methods)
Memory	about 78 GB of weights in 8-bit floating point (FP8)
Reasoning	4 levels: none, low, medium and high
Tool calling	Yes
Knowledge cutoff	18 June 2026
Training	about 24 trillion tokens, more than a fifth of them German, on 768 NVIDIA B200 graphics processing units (GPUs)

Aleph Alpha calls Kolibri sovereign. In their launch post that means 2 things. The first is how it was built: “teams built the model in Germany, trained it on infrastructure in Germany and Finland, under European and German law, with no foreign control.” The second is what customers get: “full freedom of deployment and intellectual-property safety, so compliance comes as an inherited property.” In plain words, a ministry or a car supplier can run it on its own servers, with its data never leaving the building. Nobody can change or switch off the model under them. Aleph Alpha has also signed the European Union’s General-Purpose AI (GPAI) Code of Practice.

Sovereign doesn’t mean that nothing from outside Europe went in. The model card says so itself: English web text was rephrased with Google’s Gemma 4, German with Mistral-NeMo. Qwen3-32B labeled data for the quality filters. They then filtered the training data for the political bias such models can have, which they’ve measured in Chinese open models themselves.

How Kolibri works

Kolibri is 6 ideas stacked on top of each other. Each one is there to make German cheaper, longer or more honest.

1. 384 specialists and each token sees 6

In a normal (dense) model, every token goes through every parameter. In a mixture of experts (MoE), each layer has a crowd of small sub-networks called experts and a router that picks a few of them for each token. Kolibri has 50 layers, each with 384 experts plus 1 shared expert that every token goes through, and its router sends each token to 6 of the 384. That’s how 78.1 billion parameters turn into 3.46 billion of actual work per token.

In 2024 I gave a talk called Why Small Language Models are the future. I argued for “smaller language models with fewer parameters and fewer places things can go wrong that require lesser compute.” My analogy was a doctor who has read every medical book in the world against a specialist in hematology: go to the first one with a blood condition and “they may not get it right cuz they know too much.” A mixture of experts puts a hospital full of specialists inside one model. The router is the receptionist who sends each token to the right 6.

The analogy breaks in 2 places though. The experts aren’t neat topics like “German law”: when researchers look inside MoE models they mostly find experts for patterns of tokens, like punctuation or proper nouns, not subjects a person would pick. And the hospital has to keep all 384 specialists on staff even if you only see 6, so Kolibri computes like a 3.5 billion parameter model but needs the memory of a 78 billion parameter one. The model card says it plainly: “the full model must be held in memory even though only part of it is active at any time.”

2. A tokenizer that reads long German words

A model doesn’t read letters or words, it reads tokens: chunks of text from a fixed vocabulary, picked when the tokenizer is trained. German glues words together into long compound words. A tokenizer that learned mostly from English chops them into pieces. Here’s the German name of the Federal Constitutional Court, split by the tokenizer GPT-4o and GPT-5 use (o200k_base, through OpenAI’s tiktoken) and by Kolibri’s:

o200k_base (GPT-5):  Bund | es | ver | fass | ungs | gericht     6 tokens
Kolibri:             Bundes | verfassungsgericht                 2 tokens

Kolibri’s tokenizer has 128,000 tokens, trained with a new algorithm Aleph Alpha calls UniBPE: it keeps the bottom-up merging of byte-pair encoding (BPE) and picks each merge with a different scoring rule (the Unigram objective), which respects how German builds words. The report says it needs 11.2% fewer tokens for German text than GPT-5’s tokenizer, the best of the 9 others they measured.

I wanted to see that for myself, so I ran 6 tokenizers over all of the Basic Law for the Federal Republic of Germany, the German constitution (185 KB of very German legal text), and over its official English translation:

Tokenizer	German tokens	More than Kolibri	English tokens	More than Kolibri
Kolibri 1	35,190		39,875
o200k_base (GPT-4o, GPT-5, GPT-OSS)	41,482	17.9%	39,737	-0.3%
Qwen3.5 35B-A3B	42,907	21.9%	41,650	4.5%
Mistral Small 4	43,478	23.6%	41,301	3.6%
Gemma 4	43,850	24.6%	41,564	4.2%

On legal German, Kolibri needed 15% fewer tokens than GPT-5’s tokenizer, even more than Aleph Alpha’s own 11.2%. In English it tied with it. Wild. Fewer tokens means fewer steps to read or write the same German text and more German in the same context window. (I counted each one with its own tokenizer.json through Hugging Face’s tokenizers library, except o200k_base, which I counted with tiktoken.)

3. Most layers only look nearby

40 of Kolibri’s 50 layers use sliding-window attention: each token only looks at the 512 tokens before it. Every 5th layer looks at everything before it. It’s like reading a long contract while mostly paying attention to the sentence you’re on and stopping every few pages to think about all of it. That’s what keeps a 1 million token context affordable.

There’s a clever detail in there too. Only the sliding-window layers know where a token sits (through rotary position embeddings). The full-attention layers don’t, so the context stretches past the 262,144 tokens it was trained on without any extra position tricks. Aleph Alpha validated it up to 1,048,576.

In my talk Unlocking Value with AI Today I called finite context one of “the big three” problems of generative AI, next to hallucination and the knowledge cutoff. Kolibri goes after all 3. At 1 million tokens, on the RULER long-context test, Kolibri’s base model scores 63.2, against 57.5 for Qwen3.5 35B-A3B’s base model.

4. It thinks in German

Reasoning models think before they answer. Even on German prompts, they mostly think in English. Aleph Alpha posted about this on 24 September and wrote it up as Through the Valley of Tears: they generated about 800,000 German reasoning examples and found that a little German reasoning data is worse than none. Their model’s German math score dropped from 70.2 to 48.3: its German thoughts kept going around in circles and never finished. It only climbed back (to 67.3) with a lot more German data.

Kolibri got the lot more. It reasons in German on German prompts. Its German math scores are the best of the models with about 3 billion active parameters: 87.5 on the American Invitational Mathematics Examination (AIME) 2025 in German, against 84.4 for the next best, NVIDIA’s Nemotron 3 Nano.

5. It’s trained to say “I don’t know”

Hallucination was number 1 of my big three. The fix in that talk was retrieval augmented generation (RAG): you look up good, authoritative information and put it in the prompt. RAG only works if the model admits when the documents don’t have the answer, though. That’s what Aleph Alpha trained for, with their own method, the Merlin-Arthur protocol.

It works like a game with 3 players. Arthur is the model. He gets a question with parts of the supporting document hidden. Merlin hides parts so that the correct answer gets easier to find. Arthur is trained to answer those. Morgana hides the evidence the answer depends on. Arthur is trained to say he doesn’t know. Arthur never knows which of the 2 he’s facing, so the only way to win is to actually check whether the evidence in front of him supports an answer.

It shows. On Artificial Analysis’s Omniscience test, when Kolibri didn’t know an answer, it said so (or gave a partial answer) 44% of the time instead of making one up. Qwen3.5 35B-A3B did that 11.1% of the time and GPT-OSS 120B 23.7%. Of all the mixture-of-experts models Aleph Alpha compared, only Qwen3.6 35B-A3B did better, at 56.7%.

6. You pick how hard it thinks

Every request can set reasoning_effort to none, low, medium or high, so a quick lookup answers right away and a hard question gets a long think, from the same model on the same server.

What Kolibri is good at

These are the rows where Kolibri leads the open models of its size in Aleph Alpha’s evaluation, which runs every model through the same setup with the sampling settings its makers recommend:

	What Kolibri	Closest model with about 3B active parameters
Overall score, English	75.5	74.7 (Qwen3.5 35B-A3B)
Overall score, German	70.8	69.8 (Qwen3.5 35B-A3B)
AIME 2025, English	96.9	89.6 (Nemotron 3 Nano)
AIME 2025, German	87.5	84.4 (Nemotron 3 Nano)
Questions over unseen company documents, English	89.7	87.0 (Qwen3.5 35B-A3B)
Long context at 1 million tokens (base model)	63.2	58.5 (Nemotron 3 Nano)

On AIME 2025 and 2026 in English its math beats every MoE model in the comparison, including the ones with 12 billion active parameters, and only the dense Qwen3.8 27B scores higher. The company-documents row is 5 tests Aleph Alpha built from customer-like work in semiconductors, the German public sector, aerospace, an automotive supplier and industrial drives, run over documents and questions the model never saw in training.

What Kolibri is bad at

Aleph Alpha publishes its weakest rows right next to its best ones in the model card, so here they are:

• Kolibri knows less from memory. It’s last of the 12 models on a closed-book test (the Retrieval-Augmented Generation Benchmark’s questions with no documents, 51.0). It answers only 14.8% of the Omniscience questions correctly, against 22.2% for Qwen3.5 35B-A3B. It knows when it doesn’t know, which is great, but it also knows less: fine with documents in the prompt, bad as a trivia oracle.
• Multi-turn tool calling is weaker. On the Berkeley Function Calling Leaderboard’s multi-turn tests it scores 39.8, against 58.2 for GLM-4.7 Flash and 54.0 for Qwen3.5 35B-A3B.
• Kolibri isn’t the best coding agent. It scores 27.7 on Terminal-Bench 2.1 against 39.7 for Qwen3.5 35B-A3B, and 66.4 on SWE-bench Verified against 73.8 for Qwen3.6 35B-A3B.
• Long context dips in the middle. At 128,000 tokens on RULER, Qwen3.5’s base model scores 89.9 to Kolibri’s 67.9. Kolibri only pulls ahead at the very long end.
• The memory. 78 GB of weights means a data-center GPU or two (it won’t fit on a laptop, whatever the 3.5 billion suggests).
• The plumbing is new. It needs Aleph Alpha’s vLLM plugin, which supports one vLLM version at a time (0.29 today), and on launch day no hosted provider serves it yet.
• 2 languages only. That’s on purpose: the model card calls it “a deliberate choice of depth over breadth.”
• A bigger dense model beats it. Qwen3.8 27B scores 80.2 in English and 79.9 in German, but it uses nearly 8 times as many parameters for every token. That’s the trade Kolibri is built around.

How to run Kolibri

You need about 78 GB of GPU memory: 2 NVIDIA A100s or H100s with 80 GB each at the least, or a single H200, B200 or B300. I haven’t run the model itself yet, since it needs a data-center GPU and nobody hosts it so far, so this is straight from the model card. First the plugin, which installs the vLLM version it supports:

pip install 'aleph-alpha-inference>=1'

vllm serve Aleph-Alpha/Kolibri-1 --kv-cache-dtype fp8 \
  --reasoning-parser kolibri1 \
  --tool-call-parser kolibri1 \
  --enable-auto-tool-choice

That gives you an OpenAI-compatible server, so any OpenAI client talks to it. The reasoning effort goes through the chat template:

# from the Kolibri model card (trimmed)
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="Aleph-Alpha/Kolibri-1",
    messages=[{"role": "user", "content": "Erkläre kurz, was ein Mixture-of-Experts-Modell ist."}],
    extra_body={"chat_template_kwargs": {"reasoning_effort": "high", "enable_thinking": True}},
)
print(response.choices[0].message.content)

The model card recommends temperature=1.0, top_p=0.97 and top_k=128, and contexts of at most 262,144 tokens for anything latency-sensitive. For the full million, serve it with 2 more flags:

vllm serve Aleph-Alpha/Kolibri-1 --kv-cache-dtype fp8 \
  --max-model-len 1048576 \
  --hf-overrides '{"max_position_embeddings": 1048576}'

When to use Kolibri

Kolibri is the pick when German text and your own hardware both matter: a public authority, a bank, a manufacturer or an aerospace supplier that has to keep its documents in house, wants answers in German that reason in German, and would rather hear “I don’t know” than a confident wrong answer. RAG over long German documents (laws, contracts, manuals) plays to every strength above: the tokenizer, the 1 million token context and the abstention.

That’s the kind of project I’m working on right now, with Prof. Dr. Heinrich Audebert, who heads neurology at Campus Benjamin Franklin, one of the Charité’s hospitals in Berlin. Today, a patient with a neurological complaint goes to their general practitioner (GP), and the GP has to see them, which is very demanding for a busy practice. In what we’re building, the patient sits down at a computer in the GP’s practice, an AI avatar takes them through a battery of tests, and it grades how urgent their symptoms are: a referral to a specialist right away, or they can wait a bit. The conversations are in German, they’re about people’s health, and the model has to run where we control it, so we’re thinking of using Kolibri for it.

It’s the wrong pick for a coding agent, where Qwen3.6 35B-A3B leads, for questions the model has to answer from memory, for any language besides German and English, and for anyone who can’t spare 78 GB of GPU memory. For that last case, the small specialized model I argued for in 2024 is still the answer: for my podcast search I fine-tuned Mistral 7B on my Apple silicon laptop instead of paying for GPT-4o.

Which is better for your own documents, a hospital of specialists like Kolibri or one small model trained for a single job, only shows when you try both on your data. Swapping the model under an agent without breaking it is what my workshop on reliable AI agents covers: guardrails and retries that work whichever model sits underneath.

Questions

What is Aleph Alpha's Kolibri?

Kolibri is an open-weight large language model from Aleph Alpha for German and English, released on 3 October 2026 under the Apache 2.0 license. It's a mixture of experts with 78 billion parameters, of which about 3.5 billion work on each token, it reads up to 1 million tokens of context, and it was trained from scratch on infrastructure in Germany and Finland.

What is a sovereign AI model?

A sovereign AI model is one an organization can run entirely on infrastructure it controls, built under the laws it has to follow, so its data stays in its hands and nobody else can change or switch off the model. Aleph Alpha uses the word in 2 senses for Kolibri: how the model was built (by teams in Germany, on infrastructure in Germany and Finland, under European and German law) and how it reaches customers (open weights they can run anywhere).

Is Kolibri open source?

Kolibri's weights are open under the Apache 2.0 license on Hugging Face, so anyone can download them, run them, fine-tune them and build products on them. The license covers the weights and configuration files, and Aleph Alpha keeps the rights to its training code and methods.

What hardware does Kolibri need?

Kolibri needs about 78 GB of GPU memory for its weights, so the minimum is 2 NVIDIA A100 or H100 GPUs with 80 GB each, or 1 H200, B200 or B300. Only about 3.5 billion parameters work on each token, but all 78 billion have to sit in memory.

Is Kolibri the best German LLM?

Among open mixture-of-experts models that use a few billion parameters per token, Kolibri has the highest German score in Aleph Alpha's own evaluation: 70.8, against 69.8 for Qwen3.5 35B-A3B, which has 35 billion parameters with 3 billion active (A3B) per token. The dense Qwen3.8 27B scores higher at 79.9, but it uses nearly 8 times as many parameters for every token.

Abstract

This report introduces Kolibri, an English–German Mixture-of-Experts transformer with 78.1B total parameters and 3.46B active parameters per token, released as open weights under the Apache 2.0 licence. We built Kolibri for sovereign and specialised deployment, with a particular focus on German and on regulated domains such as public administration, industry, and aerospace. To support these settings, we control the full model-development process, including data, architecture, training infrastructure, post-training, and evaluation.

We train Kolibri on 24T tokens across pre-training, mid-training, and long-context extension, with German accounting for more than 20 % of the data mix. To reach this share, we complement German datasets with more than 2T German tokens that we curate from the web with our own pipeline and generate synthetically. Post-training combines supervised fine-tuning on curated German and English data with reinforcement learning on more than 1.2M internally curated tasks, which cover reasoning, tool use, instruction following, code, and retrieval. The model activates 4.4 % of its parameters per token, and among the open models evaluated in this report, it lies on the Pareto frontier of quality and serving cost in English and German.

Kolibri builds on Kolibri Origin, an earlier iteration that validated our full pipeline at scale. Both models come from our Model Factory, which connects data, experimentation, training, evaluation, and deployment through version-controlled workflows.

Figure 1. Average English and German benchmark scores of base (top) and post-trained (bottom) models against decoded bytes of text per second and GPU, where higher and further right is better. Every model runs in its fastest configuration on eight B200 GPUs at a sequence length of 4k (base) or 16k (post-trained). Scores taken from Tables 17, 28 and 29; eval description in Section 3.3.1; plot description in Appendix A.

Introduction

Organisations in public administration, industry, and aerospace process sensitive data under regulation. For these domains, a language model has to run on infrastructure that the organisation controls, rest on training in line with European values, and serve German as well as English. In addition, the serving cost of the model limits the scale at which an organisation can deploy it.

Kolibri, which we built for these requirements, is an English–German MoE transformer with 78.1B total parameters, of which 3.46B, or 4.4 %, are active per token. The Apache 2.0 licence of the open weights permits the deploying organisation to operate the model on its own infrastructure. Among the open models we evaluate, Kolibri lies on the Pareto frontier of quality and serving cost in English and German, both as a base and as a post-trained model (Figure 1). Figure 2 shows that Kolibri scores higher than three open models on most benchmarks across Agentic, Industry, Math, Code and General Knowledge. The model processes contexts of up to 1M tokens, and users set a reasoning effort per request to trade cost and latency against answer quality.

We designed the development pipeline of Kolibri with applicable regulation, such as the EU AI Act, its General-Purpose AI Code of Practice and the General Data Protection Regulation in mind from the ground up. Our data pipeline filters against illegal, harmful, and pirated content and redacts personal data from data sources. In addition, we align the answers of the model with the values of the democratic consensus in Europe and train the model to abstain when the provided context does not support an answer.

Teams in Germany developed Kolibri end to end and train it on infrastructure in Germany and Finland. Training spans 24T tokens across pre-training, mid-training, and long-context extension, with a German share of more than 20 % and sequences of up to 256k tokens in the final stage. To reach this share, we complement German datasets with more than 2T German tokens that we curate from the web with our own pipeline and generate synthetically. Post-training combines SFT on curated German and English data with RL on more than 1.2M internally curated tasks, which cover reasoning, tool use, instruction following, code, and retrieval. The predecessor of Kolibri, Kolibri Origin, is an internal model that we used to validate the full development pipeline at scale. From Kolibri Origin to Kolibri, we scaled the data, the model size, the context length, and post-training, and we refined the architecture and the training recipe (Table 1).

The entire pipeline, from data curation through pre- and post-training to evaluation, runs in our Model Factory, which comprises the infrastructure, tooling, and development processes that connect data, experimentation, training, evaluation, and deployment. The Model Factory expresses these workflows as version-controlled code, which makes every experiment reproducible, composable, and traceable. A new model generation, an additional language, or a further specialisation thus becomes a change to this code.

The remainder of this report follows the development of Kolibri. Part 2 describes the architecture, the optimisation, the data, and the stages of pre-training, mid-training, and long-context extension, together with the experiments behind the design choices, and evaluates the base model. Post-training follows in Part 3, which covers SFT and RL and evaluates the post-trained model. Part 4 describes the Model Factory, and Part 5 summarises the results. The appendices in Part 6 give the details of the individual stages and the quality-cost analysis.

Figure 2. Post-training benchmark scores of Kolibri, Kolibri Origin and three open models in four domains, from Tables 28 and 29. Badges mark the English (EN) and German (DE) version of a benchmark; the scale runs from 0 % at the centre to 100 % at the outer ring.

Property	Kolibri Origin	Kolibri
Size	30.6A3B	78.1A3B
Pre-training tokens	8T	24T
Supported context length	64k tokens	1M tokens

Table 1. Main differences between Kolibri Origin, an internal model that is not released, and Kolibri. Active parameters exclude the embeddings. Table 2 lists the full architecture of both models.

Pre-Training

This part covers the three stages that produce the Kolibri base model: pre-training, mid-training and long-context extension.

After pre-training for 20T tokens at a 16k context length on data from nine domains (Section 2.3), mid-training continues for 3.4T tokens at a context length of 64k tokens, on a mix drawn mostly from curated datasets rather than from web text (Section 2.3.3). The long-context extension then adds 200B training tokens at 256k token context length (Section 2.3.4). Figure 3 shows the training loss throughout pre-training.

Every stage was scaled with respect to Kolibri Origin, which pre-trained on 7.5T tokens at a context of 4k tokens, mid-trained on 122B tokens at 8k and extended the context on 96B tokens at 64k. Kolibri trains on about three times as many tokens in total, at four times the maximum context. Mid-training grows the most, by a factor of about 28.

We start by describing the model architecture and how we run the pre-training in Sections 2.1 and 2.1.6, followed by our specific optimisation settings in Section 2.2. Section 2.3 describes the pre-training data: the Common Crawl pipeline, the mixing of the data sources, PII redaction and decontamination. Sections 2.3.3 and 2.3.4 describe the data of the two later stages and the ablations that selected their mixes. Section 2.4 defines the evaluation suite that the ablations of the following sections report scores on and describes the benchmark scores of the resulting base model.

2.1 Model Architecture

Kolibri is a decoder-only causal Transformer (Vaswani et al. 2017) with hybrid attention, which interleaves SWA and full-attention blocks, and sparse MoE layers of FFNs (Shazeer et al. 2017). It has 78.1B total parameters and 3.46B active parameters per token. Therefore, each token uses 4.4 % of the parameters, keeping the computation per token small at a large total capacity (Section 2.1.4). Figure 4 shows one block and Table 2 lists its dimensions. Each block uses sandwich normalisation (M. Ding et al. 2021; V. Singh et al. 2026), applying RMS normalisation (B. Zhang and Sennrich 2019) before and after both the attention and MoE layers.

The tokeniser vocabulary of 128 000 tokens is trained with UniBPE (Section 2.1.2), which builds merges bottom-up like BPE but selects each merge by Unigram loss. Compared with the GPT-5 tokeniser, it places more token boundaries on morpheme boundaries, in German 65 % against 47 %, and needs 11.2 % fewer tokens on German web text.

Attention uses GQA (Ainslie et al. 2023) with 48 query heads, 4 KV heads and head dimension 128. The blocks combine SWA and full attention at a 4:1 ratio: every fifth block uses full attention, and the other blocks use SWA (Beltagy, Peters, and Cohan 2020) over the 512 preceding tokens. Positional information is injected only in the SWA blocks using RoPE (Jianlin Su et al. 2024) with θ = 10 000. The full-attention blocks use no positional encoding. Queries and keys are normalised per head with RMS normalisation (Henry et al. 2020) before RoPE. Section 2.1.3 compares the hybrid attention pattern with full attention, selects the window size and gives the reasons for the positional encoding and QK normalisation.

Every MoE layer routes each token to 6 of 384 routed experts and also evaluates one shared expert (Dai et al. 2024). The experts are SiLU-gated FFNs (Shazeer 2020). The total parameter count follows from the sparsity sweep of Section 2.1.4. The routing scores are sigmoids of the router logits (DeepSeek-AI 2024b), but are not renormalised after top-k selection. Expert loads are balanced with Exact Quantile Balancing (EQB) and Load-Error Injection (LEI) (Neitemeier et al. 2026) (Section 2.1.5).

Kolibri Origin has a simpler architecture (Table 2), in which every block uses full attention with RoPE, the first two blocks are dense, and each MoE layer routes a token to 8 of 128 experts. It is less sparse, with 10.7 % of its parameters active per token against 4.4 % for Kolibri. Its expert loads are balanced with the combination of Soft-Clamped Momentum Expert Bias Updates (SMEBU) (V. Singh et al. 2026) expert-bias updates and a GShard-style auxiliary loss (Lepikhin et al. 2021).

Property	Kolibri	Kolibri Origin
Transformer blocks	50	50
Leading dense layers	0	2
Full attention / SWA blocks	10 / 40	50 / 0
SWA window size	512 preceding tokens	—
Model width	2560	2048
Vocabulary	128 000	96 000
Attention heads / KV heads	48 / 4	32 / 4
Attention head dimension	128	128
Dense FFN hidden dimension	—	6144
Routed experts per MoE layer	384	128
Experts per token	6 routed and 1 shared	8 routed and 1 shared
Routed-expert hidden dimension	512	768
Total parameters	78.1B	30.6B
Active parameters per token	3.46B	3.27B
Trained maximum context	256k tokens	64k tokens

Table 2. Architecture of Kolibri and, for comparison, Kolibri Origin. Active parameters exclude the embeddings.

2.1.1 Ablation Methodology

We compare recipe choices through controlled proxy-scale ablations before validating at larger scale. Each study specifies its resource-matching criteria, training budget and evaluation stage, with checkpoints compared at a common training horizon.

Proxy Models

We run ablations on two smaller proxy models: versions of Kolibri which share the main architectural choices but fewer parameters (Table 3). Both proxy models differ in width and have fewer routed experts per layer at a lower dimension compared to the target Kolibri. The proxy at scale M has 0.6B active out of 7.8B total parameters trained on 97.6B tokens. The proxy at scale L has 2.1B active out of 30.6B total parameters trained on 327.6B tokens.

Scale	M	L	Target (Kolibri)
Transformer blocks	50	50	50
Model width	1024	2048	2560
Attention heads / KV heads	16 / 2	32 / 4	48 / 4
Routed experts per layer	256	256	384
Routed experts per token	6	6	6
Routed-expert hidden dimension	192	384	512
Total parameters	7.8B	30.6B	78.1B
Active parameters per token	0.6B	2.1B	3.46B
Pre-training tokens (steps)	97.6B (10 342)	327.6B (13 016)	20T
Long-context tokens (steps)	5B (299)	15B (448)	200B

Table 3. Scale M and L proxy models, used for ablation studies next to the target configuration of Kolibri (Table 2). Tokens are the pre-training horizon of each scale, in batches of 576 (M) and 1536 (L) sequences of 16k tokens.

Evaluation

We evaluate every candidate change along three axes: modelling quality, stability in the attention and MoE layers, and efficiency, both in training and decode during inference. To assess modelling quality, we use validation loss and a suite of benchmarks, which are described in Section 2.4.1. Because we broadened the evaluation suite as the model was developed, aggregates are not directly comparable across studies. To ablate the impact on long-context performance, some studies applied a shortened long-context phase directly after pre-training.

Learning-Rate Selection

Architectural changes can shift the optimal learning rate, so using only the scaling-law prediction can confound shape quality with a suboptimal learning rate. For model-shape sweeps, we tune learning rates at the smallest ablation scale using η ∈ {η0/√2, η0, √2η0}, where η0 is the predicted learning rate.

2.1.2 Tokeniser

We designed the Kolibri tokeniser to capture the specificity of the German language, without compromising on English efficiency and downstream model performance.

The model uses a vocabulary of 128 000 tokens, up from 96 000 tokens used by Kolibri Origin. We train both Kolibri tokenisers with UniBPE, a new hybrid tokeniser training method: it keeps the greedy bottom-up approach of BPE (Gage 1994; Sennrich, Haddow, and Birch 2016) and scores merges with the Unigram loss objective (Kudo 2018).

UniBPE Merge Score

A tokeniser state assigns every token t a count ct over the training corpus, with N = ∑t ct tokens in total. The Unigram model puts probability ct/N on t, and the negative log-likelihood of the corpus under it is

L = −∑t ct log(ct/N) = h(N) − ∑t h(ct), h(x) = x log x, h(0) = 0. (1)

This is the quantity Unigram tokenisers minimise top-down by pruning tokens. UniBPE minimises it bottom-up: at each step it applies the merge that lowers L the most, and otherwise runs exactly like BPE, producing an ordinary vocabulary and merge list.

Merging the pair (a, b) into a new token ab, where the pair occurs cab times, moves cab counts out of a and out of b, creates a token with count cab, and shortens the corpus by cab tokens. Every other count is untouched, so for a ≠ b the change of the loss

∆L = h(ca) − h(ca − cab) + h(cb) − h(cb − cab) − h(cab) + h(N − cab) − h(N). (2)

is closed-form. BPE ranks candidates by cab alone; UniBPE ranks them by ∆L, the most negative first. Both only need the pair count and the two parent counts, so no new information is required over BPE.

Training

We train the tokeniser on a shuffled 100 GB slice of the pre-training dataset (Section 2.3) without Unicode normalisation. For pre-tokenisation we adapt the GPT4 pre-tokeniser (OpenAI 2023), changing the regex to split every digit into its own token. The tokeniser has byte fallback, starting with 256 byte tokens. The next 120 000 − 256 = 119 744 merges use the UniBPE algorithm, standard BPE performs the next 7900 merges, and we then add 100 reserved and special tokens, for a total vocabulary of 128 000 tokens. The second stage of standard BPE has two purposes. First, some useful merges are difficult under UniBPE, such as .., preventing ... from being a single token. Second, Kolibri Origin used a pure UniBPE algorithm, leading to degraded compression. This hybrid approach matches the compression of standard BPE in our experiments.

Compression

On German text, the Kolibri tokeniser reaches 4.90 bytes per token, the highest compression across twelve evaluated tokenisers in Figure 5, with the best of the nine other tokenisers being 4.35 for GPT-5, and is on par with BPE trained on the same pre-training corpus. On English, Kolibri reaches 4.58 bytes per token, against 4.67 for GPT-5, the best tokeniser on English. This lowers the cost of training on and serving German text, and leads to shorter context lengths in long-context tasks while staying competitive in English.

Morphology

UniBPE consistently improves on morphological alignment compared to plain BPE. In the first 60 merges each performs on the pre-training dataset, UniBPE creates the units of three or more letters ing, the, ion, tion, for, and, ation, pro, that, con, ent, ment, with, ther and the German und, ung, ver, and the German characters ü, ä, ö, ß. BPE has no German units or characters, and only five units of three or more letters, the, ing, ion, ent and and by merge 60.

We quantify this morphological alignment in Appendix B.1.1. Kolibri creates splits at morpheme boundaries with a precision of 65 % in German and 50 % in English, compared to 49 % and 42 % respectively for the best of the external tokenisers.

Downstream Model Performance

We ran a Kolibri tokeniser vs BPE ablation at scale M (see Table 3). For both training runs, Figure 6 compares the Bits per Byte (BPB) after pre-training and the recall on two retrieval tasks after the long-context stage. Both runs reach the same BPB in English and German, so the two tokenisers are matched in language-modelling quality, while the Kolibri tokeniser improves recall on long-context Variable Tracking (VT) and KV retrieval at every length.

2.1.3 Hybrid Sliding-Window Attention

One target of Kolibri is to improve long-context inference efficiency compared to Kolibri Origin which uses full attention in every block: each token attends to all preceding tokens, so the compute grows quadratically with the sequence length in all layers. During inference, the KV cache grows linearly with the sequence length, leading to increased memory usage and computational cost. A mitigation is to use a hybrid of full attention and SWA (Beltagy, Peters, and Cohan 2020) layers, which restrict attention to a fixed-size window of preceding tokens, reducing both memory usage and computational cost. We integrate SWA into Kolibri, increasing long-context inference efficiency and benchmark performance.

Hybrid Attention

Kolibri uses full attention in every fifth block (10 of 50) and SWA in the other 40. First experiments at proxy scale M (Table 3) showed improvements in long-context tasks without a loss on pre-training benchmarks. At matched training compute, the hybrid model supports sequences four times as long as the full-attention model. Moreover, the hybrid model is generally more efficient during inference.

Positional Encoding

The SWA blocks apply RoPE (Jianlin Su et al. 2024) with base θ = 10 000 to queries and keys; the full-attention blocks use No Positional Encoding (NoPE). In early experiments, removing RoPE from the full-attention blocks improved long-context performance, in line with B. Yang et al. (2025).

QK Normalisation

Every block normalises queries and keys per head with RMS normalisation before RoPE (Henry et al. 2020; B. Zhang and Sennrich 2019) which bounds the attention logits and keeps training stable at higher learning rates. We refer to Appendix B.2 for an analysis of QK normalisation effects. During RL, assuming sufficiently small gain parameters, the QK normalisation also bounds every query and key element below the FP8 range, so the FP8 KV cache cannot overflow (Appendix I.4.1).

2.1.3.1 Window Size

We select the window size w∗ ∈ {128, 256, . . . , 4096} by running ablations at scale M and validating the best two variants at scale L (Table 3).

After the pre-training stage, we add a long-context stage of 5B tokens on sequences of 256k tokens. We evaluate the pre-training benchmarks at the last pre-training checkpoint and the long-context benchmarks after the long-context stage. Across windows, we approximately match total FLOPs, KV-cache size and parameter count (Table 4).

Ablations at Scale M

We select the window for long-context score and inference efficiency, and also compare the scores on the pre-training benchmarks.

Table 4 also reports the average accuracy and BPB of our pre-training benchmarks at the last pre-training checkpoint. While the scores are generally similar across window sizes, the configuration with window size 256 achieves the highest scores.

Figure 8 shows the pooled long-context score for each window size configuration depending on the context length. At scale M, w = 128 yields the highest pooled score and leads from 8k to 64k tokens, but degrades at context length 128k. w = 512 follows closely and degrades the least with context length.

Validation at Scale L

Based on the ablations at scale M, we validate w = 128 and w = 512 at scale L (Table 3). Figure 8 also reports the pooled long-context score for these models. Their average accuracies on the pre-training benchmarks are 48.0 % for w = 128 and 47.8 % for w = 512, while their pooled long-context scores are 76.2 % and 77.1 %. Through 128k tokens, the two windows are generally close, with the largest absolute difference being 3.1 percentage points. At context length 256k, however, w = 512 scores 6.7 percentage points higher than w = 128.

We therefore use a sliding window of w = 512 in Kolibri, at the cost of 0.2 pp of accuracy on the pre-training benchmarks.

2.1.4 Scaling Sparsity

Sparse MoE architectures are able to increase model capacity at matched compute by routing each token through a subset of experts. We measure their sparsity as the share of the parameters that is active for a token. At fixed expert width and routing top-k, adding experts increases the total parameter count while leaving the number of FLOPs per token approximately unchanged. A larger total parameter count increases parameter storage, communication volume and optimiser states, and makes expert balance harder to maintain. Downstream performance, however, improves substantially with total capacity (A. Clark et al. 2022; Frantar et al. 2024; Ludziejewski et al. 2024; Abnar et al. 2025; Tian, K. Chen, et al. 2025; Kimi Team 2025), a trend we confirm in our own ablations. We compare configurations of different sparsity by their quality, inference efficiency and deployment cost.

Quality

We vary sparsity by increasing the number of routed experts from 128 to 256 and 384 at 3.4B active parameters, which gives 42.1B, 82.3B and 122.6B total parameters.

Moving from 42.1B to 82.3B improves loss and aggregate eval accuracy. The 122.6B configuration improves loss and aggregate eval accuracy only slightly further. The load balancing of Section 2.1.5 keeps the expert loads well balanced in all three configurations, including the 122.6B one. A preliminary sweep at 0.9B active parameters shows the same ordering. Appendix B.4 shows the loss and accuracy of both sweeps and the deployment footprint of each configuration.

Inference Efficiency

We focus on the inference regime that is most interesting for the RL phase, where most of the runtime is spent decoding rollouts. We therefore compare the configurations by their decode throughput: the tokens generated per second, summed over all concurrent requests. Since the decode phase is mainly memory-bound, increasing total parameters can increase inference cost through greater parameter storage and weight movement, as well as additional communication overhead. To capture these effects, we measure this throughput for the three configurations with 3.4B active parameters on 8 H100 SXM5 GPUs in BF16, using for each model and context length the best tensor-, data- and expert-parallel configuration identified through a combinatorial search.

The main sources of memory traffic are the active model weights and the KV cache. Weight reads increase with the total parameter count, although they can be amortised across tokens routed to the same experts within a large batch. KV-cache reads, by contrast, remain specific to each request and grow with its live context. At 4K context length, where weight reads account for a larger share of memory traffic, the 82.3B and 122.6B configurations are 13 % and 32 % slower than the 42.1B configuration. At 256k, the gaps narrow to 4 % and 12 % as KV-cache reads account for a larger share of total memory traffic. Most of this long-context traffic comes from the 10 full-attention layers, while the 40 sliding-window layers read a fixed window and contribute negligibly.

Deployment Footprint

In resource-constrained deployments, total parameter count directly reduces the memory available for activations and KV cache, limiting request concurrency at long context lengths (Table 31). For example, on two H100s, the 122.6B configuration fits only 3 concurrent 256k-token requests, compared with 18 for 82.3B.

The 82.3B configuration improves all reported quality metrics over 42.1B, while increasing capacity to 122.6B yields only a small further improvement, reduces decode throughput and fits far fewer long requests. We therefore target 80B total parameters at 3.4B active parameters. With the total size fixed, further ablations set the number and width of the experts and the other shape parameters. Appendix B.5.1 compares three expert configurations at this size. Informed by a preliminary sweep at 0.9B active parameters, we choose to widen the residual stream (Appendix B.5.2).

2.1.5 MoE Load Balancing

An MoE layer combines a shared expert, evaluated for every token, with E routed experts, of which only K are selected per token. For T tokens and E routed experts, each MoE layer computes router logits Z ∈ Rᵀˣᴱ. Expert biases β ∈ Rᴱ are added to the router logits Z to produce the biased logits B used for balanced expert selection:

B = Z + 1Tβ⊤,
It = Top-K(Bt,:, K),
St,e = sigmoid(Zt,e). (3)

Here Bt,: denotes the row of the biased logits for token t, and S denotes the routing scores that weight the outputs of the selected experts; 1T is the all-ones column vector, and Top-K(x, K) returns the indices of the K largest entries.

For an input representation xt, the layer output is

yt = fshared(xt) + ∑e∈It St,e fe(xt). (4)

The selected scores are not renormalised, and the biases affect selection only; the mixture weights use the unbiased sigmoid scores.

Load balancing ensures sufficient learning signal for every expert and efficient expert-parallel execution. The former requires balance across the global optimiser-step batch; the latter requires balance within each local microbatch. For either batch B, we measure the maximum relative excess load by MaxVio:

MaxVio(B) = maxe [ ce(B) / (K|B|/E) − 1 ]. (5)

where ce(B) counts the tokens assigned to expert e.

Balancing methods include auxiliary losses (Lepikhin et al. 2021) and expert-bias updates, either with a fixed step (DeepSeek-AI 2024b) or based on quantiles (Kimi Team 2026b).

Kolibri combines Exact Quantile Balancing (EQB) for global balance with Load-Error Injection (LEI) for local balance (Neitemeier et al. 2026). EQB maintains stable global load balance without a tunable balancing hyperparameter. The LEI coefficient therefore primarily controls local balance and training efficiency, giving the recipe fewer and less sensitive hyperparameter choices than the recipe of Kolibri Origin, which combines SMEBU bias updates (V. Singh et al. 2026) with a GShard-style auxiliary loss (Lepikhin et al. 2021).

Over the Kolibri pre-training run, the global MaxVio averaged over layers stays at 0.31. Only the first two layers, whose routing the bias overrides (Section 2.1.5.3), reach 2.39 on average.

2.1.5.1 Exact Quantile Balancing (EQB)

Quantile Balancing (QB) (Kimi Team 2026b) computes token biases α ∈ Rᵀ and expert biases β through alternating quantile updates:

αt ← −QK(Zt,: + β⊤),
βe ← −Qr(Z:,e + α),
r = TK/E, (6)

where Qm denotes the (m + 1)th largest entry. Subtracting this cutoff leaves m positive entries, so the token bias update targets K assignments per token and the expert bias update r assignments per expert.

Training performs one token-bias update followed by one expert-bias update per optimiser step. The resulting expert biases β are centred by subtracting their mean and we use the top-K of the biased logits for routing. Using top-K per token approximates the token biases computed by the first step of the QB iteration during training and inference.

The global batch is partitioned across data-parallel ranks and gradient-accumulation microbatches. Each rank holds the router scores Zt,: for all experts for its local tokens, together with the shared expert biases β. The token bias αt can therefore be computed locally from Zt,: + β⊤. Updating an expert bias βe, however, requires a quantile of Z:,e + α across all tokens in the global batch. These values are distributed across ranks and microbatches, so their statistics must be combined.

Marin (Dial 2026) averages quantiles computed separately on each rank, which generally differs from the quantile of the combined batch. Kimi (Kimi Team 2026b) accumulates a 1000-bin histogram per expert over local microbatches and sums the counts across ranks with an all-reduce. The resulting histogram represents the global batch, but its quantile estimate is limited by the bin width.

EQB (Neitemeier et al. 2026) computes this global quantile exactly for bfloat16 adjusted logits using two-pass radix selection (Yifei Li et al. 2024). Each rank encodes its local values as order-preserving 16 bit keys. In the first pass, we build 256-bin histograms over the high byte and sum the counts across microbatches to locate the bin containing the global quantile. In the second pass, we repeat this for the histogram of the low byte of values in that bin to recover the exact quantile. Each layer requires two all-reduces of 256E int32 counts per optimiser step, independent of T.

2.1.5.2 Load-Error Injection (LEI)

A shared global bias cannot balance all microbatches individually. A GShard-style auxiliary loss is a common choice for local balancing, originally formulated for normalised softmax probabilities (Lepikhin et al. 2021). With sigmoid scores, models such as DeepSeek-V3 and Arcee Trinity add a separate normalisation for this loss (DeepSeek-AI 2024b; V. Singh et al. 2026). We introduce LEI to balance each microbatch, which directly injects each expert’s deviation from its target into the score gradients. Neitemeier et al. (2026, App. D) derive the LEI update as the straight-through estimate for unnormalised sigmoid scores, while the GShard loss is the corresponding estimate for normalised softmax probabilities.

For a local microbatch Bl, with ce(Bl) assignments to expert e, the relative load and relative load error are

fe = ce(Bl) / (K|Bl|/E),
ρe = fe − 1. (7)

We inject a rescaled version of the load-error vector into the sigmoid-score gradient of every token in the microbatch:

Gt,e ← Gt,e + λ tanh(ξ)/ξ ρe,
ξ = ‖ρ‖∞/τ,
t ∈ Bl. (8)

Here Gt,e is the incoming gradient with respect to St,e, λ > 0 controls the correction strength, and τ > 0 bounds the rescaled load error, which we found to help training stability. At ξ = 0, the scaling factor is 1 by continuity. Kolibri uses λ = 1 × 10−5 and τ = 1.

2.1.5.3 Leading Dense Layers

Unlike Kolibri Origin, which has dense FFNs in its first two blocks, Kolibri uses MoE layers in all 50 blocks. This is motivated by an ablation of the Kolibri architecture over the 327.6B-token budget of scale L that found slightly lower loss and BPB and higher accuracy when replacing leading dense blocks with MoE layers.

Diagnostics of the full Kolibri run, however, show that while the expert loads are balanced, its first two MoE layers make almost no use of their routed experts. Appendix B.6 gives the routing diagnostics of the full run and an analysis of the first two MoE layers.

2.1.6 Infrastructure

Pre-training ran on 768 NVIDIA B200 GPUs, across 96 nodes with 8 GPUs per node. The cluster has an NVIDIA SuperPOD™ topology. Within each node, the GPUs share an NVLink™ Generation 5 interconnect with 1.8 TB s−1 bidirectional bitrate. The nodes are connected via InfiniBand by 8 NVIDIA ConnectX-7® network adapters per node, so each GPU can sustain a cross-node bidirectional bitrate of 100 GB s−1.

Our parallelism setup for pre-training is EP 8, FSDP 16, DP 6. With this, we make use of the hierarchical interconnect in pre-training: Expert parallel across the 8 GPUs within each node and (fully-sharded) data parallel across nodes. Thus, dense parameters are sharded across EP × FSDP = 128 GPUs (one full replica), while sparse expert parameters are sharded across FSDP = 16 GPUs—the same local GPU on each node in one full replica. Mid-training and long-context extension use FSDP 128, DP 6.

We leverage several existing libraries, namely PyTorch 2.14 (Ansel et al. 2024), DeepEP v2 (C. Zhao et al. 2025) for expert parallelism, and FlashAttention 4 (Zadouri et al. 2026) for both SWA and full attention computations. Ablations without EP use SonicMoE (W. Guo et al. 2025) as an efficient MoE implementation.

For further details, see Appendix B.7, including Appendix B.7.1 on straggler detection.

2.2 Optimisation

Kolibri is trained with a Warmup-Stable-Merge (WSM) recipe (Tian, J. Wang, et al. 2025). The learning rate warms up linearly over the first 100B tokens and then stays constant through pre-training, mid-training and long-context training. In our experiments, a constant learning rate with checkpoint merging improves post-trained performance over a learning-rate cooldown (Maskey et al. 2026). Each stage (except pre-training) continues with the optimiser state after finishing the previous one and therefore needs no further warmup. We then determine the base model weights as the average of the last 20 checkpoints of the long-context stage (Yunshui Li et al. 2025), which are 10B tokens apart. Over the three stages the context length grows from 16k to 256k tokens and the token batch from 75.5M to 201M tokens per step.

The architecture improvements, particularly the hybrid of sliding-window and full attention, enable longer training sequences than in Kolibri Origin, which pre-trains at 4k tokens and reaches 64k tokens only in its long-context stage (Sections 2.1 and 2.1.3.1). Sequences are assembled online from a Random Sequential Document Buffer (RSDB) (V. Singh et al. 2026), which spreads the chunks of a long document over many optimiser steps, by bin packing these chunks without padding tokens or unnecessary document cuts (Appendices C.1, C.1.1 and C.1.2).

We optimise attention and expert weights with Muon (Jordan et al. 2024), embedding weights and norm scales with Adam (Kingma and Ba 2015), and LM head and router weights with AdamW (Loshchilov and Hutter 2019). In contrast to the commonly used convention of matching Muon’s update RMS to that of AdamW (Jingyuan Liu et al. 2025; Kimi Team 2025), we adopt the spectral convention (Bernstein 2025), which keeps the Muon learning rate independent of model width. We use decoupled weight decay for all optimiser groups except the embedding and norm scales. Adam and AdamW use ϵ = 1 × 10−15, small enough that the updates of parameters with small gradients, such as the LM head and the routers, stay independent of the gradient scale, as Muon updates are by construction.

Optimiser	Peak learning rate	Weight decay	Parameters	Initialisation
Muon, spectral	1 × 10−3	2−12	Q, K, V; expert up and gate	std 1/√din
			Attention output; expert down	zero
AdamW	2.5 × 10−5	2−12	Router gates	std 1/din
			LM head	zero
Adam	2 × 10−3	none	Token embedding	std 1
			Norm scales	1
			Post-sublayer (sandwich) norm scales	1/√L

Table 6. Optimiser and initialisation of Kolibri by parameter group. Muon uses momentum μ = 0.95 with Nesterov and spectral scaling, which multiplies the orthogonalised update by √(dout/din); Adam and AdamW use β1 = 0.9, β2 = 0.95 and ϵ = 1 × 10−15. Decoupled weight decay multiplies the decayed weights by 1 − 2−12 in every step, independent of the learning rate. Normal initialisations are truncated at three standard deviations and rescaled to the stated standard deviation; din is the fan-in of a layer, L = 50 the number of layers, and expert biases start at zero.

2.2.1 Hyperparameter Transfer

The optimal hyperparameters depend on model size and token horizon (G. Yang, E. J. Hu, et al. 2021; McCandlish et al. 2018; Bjorck et al. 2024; Porian et al. 2024; Bergsma et al. 2025), but a sweep at the scale of the target run is infeasible. Proxy runs on smaller models and shorter horizons therefore guide the choice of hyperparameters for the full-scale run. A common approach fits scaling laws that extrapolate along both dimensions jointly (DeepSeek-AI 2024a; H. Li et al. 2025; Porian et al. 2024; Poolside 2026), which requires sweeps at many points of a 2D grid over model size and horizon. Instead, we factorise the transfer across model size and token horizon, and vary model size only through the width, at the 50 layers of the target model.

For the width transfer we use Maximal Update Parametrisation (μP) (G. Yang, E. J. Hu, et al. 2021; G. Yang, Simon, and Bernstein 2024), which makes the optimal base learning rate of every parameter group approximately width-independent, so that hyperparameters found on narrower models carry over to the full-scale model (Dey et al. 2025; Meituan LongCat Team 2025). The dependence on the token horizon can then be studied at a single narrower width. Instead of scaling laws for the optimal learning rate and batch (DeepSeek-AI 2024a; H. Li et al. 2025; Bjorck et al. 2024; Bergsma et al. 2025), we use loss extrapolation (Domhan, Springenberg, and Hutter 2015) to predict the learning-rate landscape at the target horizon and pick the best cell of a grid of Muon learning rates and batch sizes on a narrower proxy.

The remainder of this section fixes the learning-rate ratios between the parameter groups (Section 2.2.1.1), introduces the width transfer with μP (Section 2.2.1.2) and the horizon transfer by loss extrapolation (Section 2.2.1.3) for learning rate, batch size and weight decay.

2.2.1.1 Learning-Rate Ratios

We find that different parameter groups require different learning-rate scales to achieve optimal performance. To avoid an explosion of the grid in later sweeps we optimise the ratios between the learning rates of the different embedding, norm and head parameter groups as a factor of the Muon learning rate in a preliminary sweep. In internal sweeps, we find these ratios to be relatively stable across widths and token horizons, and therefore assume that they can be held fixed when transferring hyperparameters. The router learning rate is set equal to the head learning rate, based on internal ablations. The resulting production learning rates are listed in Table 6.

2.2.1.2 Transfer Across Width

While there is work on joint width and depth transfer with μP (Bordelon et al. 2023; G. Yang, D. Yu, et al. 2023; Dey et al. 2025), it is less established and introduces additional complexity, and we therefore restrict ourselves to width transfer only. We keep the number of experts and top-k fixed and scale the size of every expert, so that the routing and load-balancing dynamics of the proxy match the production model. Attention heads keep their dimension of 128 and we only scale the number of heads, as in μP (G. Yang, E. J. Hu, et al. 2021). We assume the optimal batch size to be only dependent on token horizon (H. Li et al. 2025).

As described in Section 2.2, Muon runs in the spectral convention: after the Newton–Schulz orthogonalisation the update is scaled by √(dout/din), so that every layer receives an update of unit RMS operator norm times the learning rate regardless of its shape (Bernstein 2025). This ensures that the optimisation dynamics satisfy the spectral condition (G. Yang, Simon, and Bernstein 2024), and makes the Muon learning rate width-independent. The learning rates of embedding and norm scales are width-independent as well, whereas those of the LM head and router scale with 1/d. μP also shrinks the initialisation of the output layers with width.

We initialise the LM head at the infinite-width limit of this scaling, zero, which removes the width dependence of the initial outputs and brings the optimal hyperparameters of narrow and wide models closer together (G. Yang, E. J. Hu, et al. 2021, App. D.2). The attention output and expert down projections also start at zero, so that every block starts as the identity map (Hongyi Zhang, Dauphin, and T. Ma 2019); in our ablations this converged faster and still transfers across width. The router is initialised with standard deviation 1/din (Vankadara et al. 2026).

To validate the transfer we swept the base learning rate on the proxies M and L (Table 3), where L is M scaled to twice the width, both trained for 97.6B tokens, the horizon of M, with a batch of 576 sequences of 16k tokens and with learning rates scaled according to the rules described above. They are not scaled down from the production model, since the width sweep is an earlier ablation. Results are visualised in Figure 10a, with the loss averaged over the trailing 3B tokens of each run. We see a shift of the optimal learning rate by √2, which we deem acceptable. The loss differences within a √2 learning rate factor of the optimum are less than 0.1 %.

2.2.1.3 Transfer Across Token Horizon

Transferring the optimal learning rate and batch size to a target token horizon is often done via scaling laws (DeepSeek-AI 2024a; H. Li et al. 2025; Bjorck et al. 2024; Porian et al. 2024). These necessitate a large grid of values to have enough points for a reliable power-law fit, making them expensive. Additionally, they complicate transferring other hyperparameters such as weight decay from short to long horizons, as this would combinatorially explode the grid.

For Kolibri Origin, we fit power laws to the optimal learning rate and batch size at horizons from 100B to 430B tokens and extrapolate them to its pre-training budget of 7.5T tokens. Extrapolating those laws to the Kolibri setting gives unrealistic hyperparameter predictions, motivating a method that uses the observed loss trajectories of the candidate configurations.

We employ loss extrapolation (Domhan, Springenberg, and Hutter 2015) to predict the hyperparameters with the lowest loss at longer token horizons based on the observed training dynamics at shorter horizons. This allows us to use smaller grids for the search and employ coordinate descent to tune further hyperparameters iteratively. Since the sweep uses the batch size, warmup, learning-rate schedule and data of the production run on the width-scaled proxy, it doubles as a test of the production configuration, and its runs are reused for the weight-decay sweep described below.

We use a power law with floor (Hoffmann et al. 2022) in the cumulative-learning-rate coordinate (Tissue, V. Wang, and L. Wang 2025),

L(D) = E + A(D − W/2)−α, (9)

where D is the number of tokens, W the length of the linear warmup in tokens, E the irreducible loss, and A and α the scale and exponent of the reducible part. After a linear warmup of W tokens at a constant learning rate, the cumulative learning rate is proportional to D − W/2. We validated the method on internal loss curves, not reported here, and find an average prediction error at the target token horizon below 1 %.

All tested methods are fairly sensitive to the fit start point. For the method used, we discard the first 20 % of the post-warmup prefix before fitting. We find that the optimal grid point is robust to the specific form of the power law used for extrapolation: we cross-checked with a shifted power law with floor (Domhan, Springenberg, and Hutter 2015) and a one-break broken power law (Caballero et al. 2023) and obtained the same ranking.

Learning Rate and Batch Size

We sweep a grid of Muon learning rates from 5 × 10−4 to 8 × 10−3 and two batch sizes on the width-scaled proxy model, which halves the width of the production model with the same rules as in Section 2.2.1.2, for 500B tokens. The observed loss is averaged over the trailing 2B tokens of each run and, at 500B tokens, is still lowest at the largest learning rate for both batches. The prediction at 20T tokens is visualised in Figure 10b, with an optimal Muon learning rate of 1 × 10−3 and a batch size of 4608 sequences of 16k tokens each. The selected batch is the smaller of the two, so while a further reduction in batch size might be beneficial, we prioritise maintaining training efficiency.

	Width	Heads / KV heads	Expert hidden	Experts, top-k	Layers
Kolibri	2560	48 / 4	512	384, 6	50
Proxy	1280	24 / 2	256	384, 6	50

Table 7. Production model and the width-scaled proxy used for the loss-extrapolation sweeps over learning rate, batch and weight decay. Layers, experts, head dimension and the attention pattern are shared.

Weight Decay

We use the same loss extrapolation to set the weight decay in a one-dimensional sweep. Starting from the selected learning rate and batch size, we vary the weight decay and compare the predicted loss at 20T tokens. The baseline cell was trained with a weight decay of 2−13. Doubling the weight decay to 2−12 lowers both the observed loss and the forecasts at every horizon, whereas halving it raises them. All estimators agree on this order, and we use 2−12 for Kolibri. Larger values were not tested.

Limitations

This optimisation is done purely with respect to the training loss under a warmup-stable learning-rate schedule. While evidence shows that a WSM schedule can lead to better post-trained performance (Maskey et al. 2026), merging and post-trained performance are not considered here. Additionally, uncertainties are not modelled and some of the differences between grid points may be within the noise level of the measurements, although all three forms of the extrapolation law select the same cell (Section 2.2.1.3). We leave modelling merged-checkpoint and post-trained performance, as well as uncertainty of the forecasts, to future iterations of the recipe.

2.3 Data and Training Stages

2.3.1 Overview

Kolibri Base is trained in three stages: pre-training, mid-training, and long-context extension. Pre-training consumes 20T tokens from a broad English-German data pool and establishes the model’s general capabilities. Mid-training shifts the data distribution towards higher-quality reasoning, mathematics, code, and instruction-following data, training on 3.4T tokens. Finally, long-context extension trains on 200B tokens and extends the context length to 256k tokens.

Across these stages, data selection progressively becomes more selective and task-oriented. Pre-training emphasises breadth and coverage, mid-training concentrates on capabilities that benefit from higher-quality and more structured data, and long-context extension adapts these capabilities to substantially longer sequences.

In addition to filtering illegal, harmful, toxic and pirated content, we redact PII from all training sources. Each category is detected by a regular expression, checked by a validator that rejects false positives, and replaced by a special token. A document whose redacted bytes exceed 15 % of its length is dropped. In one illustrative run on a mid-training data sample, 5.3 % of scanned documents received at least one redaction, with PDF data accounting for 92 % of all redactions. We report these numbers only as an example of the redaction behaviour; the sample is not representative of the overall training corpus.

The following sections describe the data construction, mixing, and evaluation choices for each stage.

2.3.2 Stage 1: Pre-training Data

The pre-training dataset pool consists of 78 English- and German-language datasets with approximately 29.3T materialised tokens, from which the final training mix consumes 20T. The pool spans web text, code, instruction data, scientific and technical text, and legal and public-sector data.

We group datasets into nine domains according to language, subject, and degree of structure. Datasets spanning several domains are assigned to the domain containing the majority of their tokens. Table 8 summarises the resulting pool.

Web data makes up the largest part of the pool. Its largest individual source comes from our internally developed CC pipeline, which processes and filters Common Crawl snapshots to produce more than 308B quality-scored documents for downstream data mixing.

German accounts for 21.3 % of the final pre-training tokens and is treated separately in the data construction process because its availability and filtering requirements differ from English.

Approximately 24 % (4.8T tokens) of our pre-training data is synthetic in origin (including rephrases and translations). We produced synthetic data in English, using our deduplicated English CC data (see Section 2.3.2.1) as seeds for rephrasing with Gemma-4-26B-A4B. We followed a Nemotron-CC-style recipe (D. Su et al. 2025), using five prompt templates applied to each seed document: diverse QA pairs, distillation, knowledge list, knowledge extraction, and encyclopedia-style rewriting. The outputs were post-processed and filtered to remove LLM-generation artefacts and erroneous generations. In Section 2.3.2.2 we describe our work on German synthetic data.

The following subsections describe key decisions around the Common Crawl (Common Crawl Foundation 2026) pipeline, the construction of the German data pool, quality annotation, and the final data-source mixing procedure in more detail.

Domain	Short	Datasets	Tokens (B)
English web and documents	EN web	23	12,248
German web and documents	DE web	16	6,712
English instruction and reasoning	EN instr.	7	3,548
German instruction and reasoning	DE instr.	3	139
Code	Code	3	4,323
Agentic code and tool use	Agentic	1	1
STEM (documents)	STEM docs	7	1,462
STEM (QA-style)	STEM QA	9	789
Specialized domains	Spec.	9	80
Total		78	29,301

Table 8. The pre-training pool by domain, in billions of materialised tokens.

2.3.2.1 Common Crawl Pipeline

The CC pipeline is designed to process Common Crawl snapshots to curate high quality pre-training data at scale. Text is extracted from the raw archives with Resiliparse (Bevendorff et al. 2018), yielding more than 308B documents.

Language-agnostic heuristic filters and language identification route each document to an English or a German branch, which apply separate thresholds. Exact deduplication removes byte-identical documents across all snapshots, which yields 21.16B distinct documents. Fuzzy deduplication, which compares MinHash signatures of documents bucketed by locality-sensitive hashing, removes a further 25.9 % as near-duplicates (Appendix D.1.3).

Substring deduplication cuts repeated blocks of at least 500 bytes, such as navigation menus and page templates, out of the remaining documents, removing 21.3 % of text bytes (Appendix D.1.4). Appendix D.1 describes the stages in more detail.

2.3.2.2 German Pre-Training Data

At our 20 T token budget, our ablations show strong German benchmark performance with around 20 % German data in the training mix. This leaves us with a lower bound of 4 T German tokens needed for training. Note that this refers to training tokens, not unique tokens. Since datasets are commonly upsampled during training, the underlying data pool can be smaller than 4 T, though not arbitrarily so, because repeating data can have diminishing returns after roughly four epochs (Muennighoff, Rush, et al. 2023).

To put the scale of this requirement into context, we first look at the amount of German pre-training data available in existing open datasets. Table 9 summarises several major open German datasets. These datasets are shown only as a reference point for the scale of publicly available German data and does not describe our training set. For further details on training data, refer to the Public Summary of Training Content.

The open datasets in the upper part of Table 9 add up to around 1.94T tokens. This token count does not account for filtering, deduplication, or overlap between datasets, so the amount of usable unique data would in practice be smaller. This comparison illustrates the challenge of sourcing German data at the scale implied by our 4T-token training target. In the following, we describe the three approaches we considered to construct our German data pool.

First, we process raw Common Crawl dumps with a pipeline that is parameterised by language, since German and English web data cannot be filtered in exactly the same way. In particular, we retune several filtering parameters for German. For example, web-data pipelines commonly remove documents with unusually long words. However, German administrative prose often contains substantially longer words than English, so applying English thresholds can disproportionately remove this register. We therefore use FineWeb2’s German word-length settings, which allow a mean word length between 0 and 15 characters, instead of the 3 to 10 character range used by Gopher for English. With these language-specific adjustments, our pipeline produces 0.8T unique tokens of organic German web data.

Our second approach to augmenting our German corpus is to rephrase German documents. We use Mistral-Nemo-Instruct-2407 to rewrite organic German documents as an encyclopedia entry, a Q&A dialogue or an educational-style passage, preserving its semantic information. This teaches the model the same facts in several surface forms and multiplies scarce organic data, and it is a different operation from translation: the source is German, so the subject matter, the institutions and the register stay German. Rephrasing gave us about 1.1T unique tokens, making it the single largest source of German in the model. This data had already been produced for the Aleph-Alpha-GermanWeb project (T. F. Burns et al. 2026).

The third approach is translation. Translating English into German can expand the available data when the model, prompts, and chunking are chosen carefully, but it does not provide the same kind of German data as organically sourced text. Translated output can retain traces of the source language, such as literal renderings of idioms; for example, “Drive safe!” may become “Fahre sicher!” rather than the more natural “Komm gut an!”. More importantly, translation preserves much of the source corpus’s cultural and institutional distribution. A corpus translated from English therefore remains, in many respects, an English-language view of the world expressed in German. We experimented with translated data at scale for Kolibri Origin, but for Kolibri we relied on organic and rephrased German data alongside a carefully curated set of translated German data.

One side-effect of us building our own CC pipeline is that we can incorporate new crawls as they arrive so that the resulting data pool remains up-to-date. Table 9 also lists the year from which the latest source data of each listed dataset comes from. Using our own pipeline, we can ensure that the most recent data in the pool dates from 2026.

German benchmarks are also limited in coverage and sensitivity. We therefore built a dedicated German evaluation suite, combining hand-crafted and model-generated tasks, to better measure the impact of our German data. See Section 2.4.

Dataset	Size (B tokens)	Latest source data
Klettermix	500	2024
HPLT 3.0 (German slice)	609	2025
FinePDFs (German slice)	177	2025
German Commons	154	2025
MT-Nemotron-CC	117	2024
FineWeb2 (German slice)	378	2024
Our organic web data	1300	2026
Rephrased German data	1100	2025

Table 9. Sizes of selected German pre-training data sources in billions of tokens to illustrate German dataset scale. Note that this table does not describe our training set.

2.3.2.3 Quality Annotation

Method

Prior works (J. Li, Fang, Smyrnis, et al. 2024; Ettinger et al. 2025) have shown that simple, single fastText classifiers (Joulin et al. 2017) trained to distinguish, e.g., SFT-style data vs. random CC samples, can be a successful method for curating CC data for quality, leading to improved model performance on evaluation benchmarks. Other works have reported methods of ensembling classifier scores in ways which appear to outperform using individual classifiers alone (D. Su et al. 2025; T. F. Burns et al. 2026). We therefore experimented with ensembling approaches and compared this with a strong single-classifier baseline.

Simultaneously, we wanted to keep in mind scalability, e.g., it could be wasteful to apply an LLM-as-a-judge to tens of billions of documents. For German, we built upon our public work introduced in GermanWeb (T. F. Burns et al. 2026) by ensembling quality and grammar fastText classifiers using an average of their scores. To score the English documents, we developed the following method.

We first scored a fixed 1M-document English sample of our curated CC data with a judge, Qwen3-32B (Qwen Team 2025a), using more than 20 prompts. Prompts include educational value, chat-likeness, encyclopedia-likeness, content versus format properties, binary ‘quality’ judgements, and benchmark usefulness (Penedo, Kydlíček, Ben Allal, et al. 2024; J. Li, Fang, Smyrnis, et al. 2024; Sachdeva et al. 2026; Wettig et al. 2024). For each prompt, the judge returns structured JSON, enforced via constrained decoding against a schema.

We noted that disagreement between judged axes was asymmetric. Low-scoring documents did so on multiple (sometimes all) prompts, whereas high-scoring documents scored well on some prompts but not others. This implies that the properties by which documents were judged as ‘good’ were more distinctive than the absence of those properties. Document-level Spearman correlations among the judge scores was also structured into groups of highly-correlated scores, e.g., content versus format, conversational, maths, code, and evaluation- or task-usefulness.

To score the rest of the crawl, we distil each scalar judgement axis into a binary task (top quartile versus bottom quartile, discarding the middle two) and train three families of classifiers: fastText (Joulin et al. 2017), a frozen Luxical-one embedding (Datolog

Then–Meta chief AI scientist Yann LeCun in February 2025.STEPHANE DE SAKUTIN—AFP/Getty Images

Yann LeCun won the 2018 Turing Award—computer science’s equivalent of the Nobel Prize—along with fellow AI researchers Geoffrey Hinton and Yoshua Bengio for their work on deep learning, the neural network technology that underpins today’s AI boom. But today, nearly a decade later, LeCun is the only one of the three AI “godfathers” who is not deeply concerned about the risks of artificial intelligence. In fact, he thinks many of those risks are overblown, and that the constant warning about them from executives at some of the leading AI companies is misguided and counterproductive.

LeCun isn’t worried “at all” about AI wiping out humanity, and he has “zero concerns” about the recent string of rogue AI incidents, including OpenAI’s agents autonomously hacking Hugging Face in July. He attributes the incidents to poor human oversight and system design, and says they’re “totally preventable,” a belief also held by U.S. Treasury Secretary Scott Bessent, who this month called the Hugging Face incident the “responsibility of OpenAI management.”

“Those agents are doing exactly what they’ve been asked to do,” LeCun said. “They were supposed to be in sandboxes, but the sandboxes were leaky and horribly designed.” Many AI labs lack a fundamental understanding of cybersecurity, he said, something an OpenAI safety researcher also called out this week as one of the main reasons AI may cause “great harm to the world.”

Many people working in AI safety “usually have an agenda to push,” LeCun says, and then clarifies that he’s talking about effective altruism, or EA, the philosophical movement that has been obsessed with the risks AI poses to humanity. EA was little known among the general public until it made mainstream news headlines in recent weeks in the wake of a viral social media post from a former Anthropic and OpenAI researcher named Jacob Coxon, who claimed the employees at these companies “earnestly believe [AI] could kill us all by the end of the decade.”

LeCun thinks EA is “super toxic” and a “complete disaster.” Its adherents who are working in AI labs suffer from “paranoia” that causes them to make poor decisions, he said. “Apparently people are having mental issues.” This month, the Financial Times also reported that some staffers at the U.K.’s AI Security Institute, as well as at OpenAI, Anthropic, and Google DeepMind, have sought counseling, taken time off work, and spoken publicly about experiencing distress because of fears their work could cause serious harm.

Amodei is ‘deluded’ and ‘crazy,’ LeCun says

Anthropic CEO Dario Amodei and many of the company’s founding staff members are known to be sympathetic to EA ideas and to have attended EA events in the past, although Amodei has denied being an EA adherent and Anthropic says its employees represent a diverse range of views. LeCun noted that Amodei’s sister, Daniela, who is also a cofounder of Anthropic and the company’s president, is married to Holden Karnofsky, who cofounded two EA-aligned philanthropies, including Open Philanthropy (now called Coefficient Giving). Karnofsky was also a member of OpenAI’s board from 2017 to 2021.

“Dario tries to distance himself from Open Philanthropy, but he’s totally into it,” LeCun said. “I think he’s completely deluded.” Later in the interview, he calls Amodei “crazy.”

Amodei and OpenAI CEO Sam Altman “saying, ‘AI can kill us all’ is incredibly destructive for everybody, including for the public who’s scared shitless,” LeCun said. It’s also bad for the AI industry, and “the worst marketing campaign you can possibly imagine.”

He’s right: The public is scared shitless. LeCun and I are chatting at a circular table for two in a posh café in New York City’s SoHo neighborhood, just around the corner from the offices of his new company, AMI Labs. We’re drinking tea, although, as the Frenchman pointed out, our evening time slot was more suited to apéro, or drinks and light appetizers. But he had an anniversary dinner with his wife afterward, so tea was just fine, he said.

The shop is about half full of other patrons, nearly all are working on their laptops. When LeCun and I chat with one of them, I mention to her that LeCun is a luminary figure in AI. (She didn’t recognize him.) Then she immediately asks, “So, is AI going to kill us?”

LeCun is no fan of Trump. But on AI policy, the two agree on much

The public’s fears have become so widespread that President Trump has started rebranding artificial intelligence in a way he feels will instill more confidence. He signed an executive order this week renaming it “super intelligence,” and for much of this month his administration’s social media accounts posted anti–AI doomerism messages.

“Americanism, not effective altruism,” the Department of War tweeted in mid-September.

Meanwhile, most of LeCun’s social media presence is dedicated to condemning the Trump administration, particularly the way it has slashed research budgets that are necessary to promote innovation and waged a “war on science.” But when it comes to AI fears, the two are unusually aligned.

LeCun is more concerned about AI regulatory capture than human extinction. Regulatory capture refers to the idea that dominant companies can shape government rules in ways that lock in their market position, making it difficult or impossible for smaller competitors to arise. EA-backed groups are active lobbyists for AI safety bills on Capitol Hill, as are many of the major tech and AI companies. LeCun, in contrast, doesn’t think AI requires new regulation.

Amodei is “honest,” LeCun concedes. “He’s trying to do the best thing for the company, which is close to its IPO. But claiming AI is too dangerous to put in our hands, and saying it should be regulated, and saying open-source [models] are too dangerous—this is regulatory capture that would have a terrible effect if it’s followed by acts of Congress.”

Building AMI Labs

In his day-to-day, LeCun’s main focus is building his new company, AMI Labs, which he founded after leaving Meta after 12 years, seven of which he served as chief AI scientist. He remains on good terms with the company, and has started using its new Muse AI agents. He named his Hal, after HAL 9000, an AI character in 2001: A Space Odyssey.

The café where we met was around the corner from AMI Labs’ New York office, although the company is headquartered in Paris with staffers in Montreal and Singapore as well. The company’s name is pronounced as one word, “ami,” or “friend” in French—and since going public in March 2026 its now 60-some employees have been building a technology called world models, which LeCun has long argued will eventually supersede large language models as the primary system underpinning AI.

In particular, AMI is building world models that leverage JEPA (Joint Embedding Predictive Architecture), a neural network architecture that LeCun pioneered and that teaches models to predict data in a representational space within a neural network’s middle layers, rather than generating raw pixels, as many competing world models do, or words, as LLMs do.

The company’s primary focus, for now, is industrial applications. “It’s AI for the physical world, so it’s not language-related,” LeCun said. “It’s systems that understand the real world, like a manufacturing plant or turbojet engine.” Some of the main applications are anomaly detection or robotics: “If you have a machine and all of a sudden it makes a strange noise and starts breaking, you would’ve wanted to detect that as early as possible.” He also gave the example of a system that might understand the world like a cat does, for example, which knows that if it pushes a vase off the counter it will fall.

AMI Labs is LeCun’s big chance to prove he’s been right all along. He stayed mum about the company’s first product, only mentioning that it’s debuting “soon.” Some open models may be involved, he said, given that he is a big proponent of them.

“The thing I want people to know is that AI is not over,” LeCun said, then hailed a cab to get to his dinner.

A memory plugin analyzes your conversations. It generates 1,000 isolated snippets and inserts them into a vector database. With your every prompt, it attaches the five most similar snippets; if the agent is confused (which it is), it manually searches for more. That’s the product they call “memory.”

It’s a strange thing to use when you think about what problem you’re trying to solve. You want your agent to understand your project. To know where a feature is, why it was built, what you agreed on and what you care about. Instead, you get a lottery over RAG snippets, injected on every prompt, hoping the right ones float up.

Even when it does work, the agent still doesn’t understand your project. The entire memory plugin ecosystem is solving the wrong problem.

Because agents don’t need memory. They need documentation.

It’s All Just RAG

Every memory plugin on the market works the same way:

1. Go through session transcripts
2. Generate snippets of “memories”
3. Insert into a RAG database
4. On every prompt, retrieve the top 5 and inject
5. Need more? Give the agents a tool to search through the RAG database

That’s the whole architecture. Some tools are extra fancy; they let the agent search through past transcripts word for word. Or they implement some kind of multi-tier memory system that classifies short or long-term memory. Or they add a bunch of background daemons to review, merge, or deduplicate memories. “Dreamers” that rewrite memories overnight. Continuous context compression. Rerankers. Et cetera.

Each plugin tries to add new token-burning “features” to fix the same flawed architecture underneath. And that’s why none of them reliably work.

The Problem with Recall

All of these memory plugins suffer from the same broad set of problems.

• Memories are surfaced by similarity. Similarity search ranks how close two snippets are in embedding space. That’s it. You don’t know which is correct, current, or what’s missing.
• Memories are stored without context. A RAG snippet can only contain so much. You lose everything else: context, motivations, lessons, environment, and more.
• The past is treated as truth. All of these plugins rely on recall; whether it’s search through transcripts or a vector database. But the codebase changes everyday; so how accurate are each of the 500 snippets about authentication?
• Agents can’t search for what they don’t know. Even if you do expose a search tool to the agent, how would the agent know when to use it? The agent doesn’t know what it doesn’t know.
• The store is unauditable. There are 10,000 embeddings in SQLite. Which memories exist? Which are stale? Which have never been retrieved? Which are incorrect and secretly affecting the way your agent works?

These are only five of the many problems that memory plugins face, attempt, and fail to fix, because they all make the same assumption:

Agents forget: that’s the problem. So the fix is to remember. To remember better, we should capture more, index better, retrieve smarter.

Their entire thesis revolves around capturing and recalling the past. But that’s not how anyone else handles knowledge. No one rewatches a team meeting from 3 years ago to remember constraints around a feature. People write things down and use those records instead.

Similarly, the solution is NOT to give an agent a search tool that it uses to search over the past 10 million tokens worth of conversations, reconstructing fragments of what happens.

The solution, is document-based memory.

Today, people use AI to pump out products, features, and slop at the speed of light, all without reading or understanding a single line of code. Under these circumstances, it’s easy to see how documentation ends up becoming an afterthought when it should be more important than ever.

Documentation Over Recall

People already knew agents needed context. That’s why they invented AGENTS.md files: so agents don’t jump into a codebase blind. It works. But oftentimes, that single file is the ONLY documentation that the project has.

A single file is not enough. The agent needs an entire brain, a structured workspace where it can record instructions, specs, decisions, research, indexes, anything without being asked. Instructions for how the code review process works, specs detailing what was discussed with the user, reusable research into a foreign library or API.

When the agent works, the agent can read files from the brain to gain relevant and complete context. After the agent works, it updates what’s outdated, adding new documents where necessary, while the full picture is still in context. In this way, the agentic loop changes from prompt → build → forget to prompt → consult → build → update. Memory turns from a RAG database you bolt onto the agent into a workspace you can read, update, and even share.

Putting It to the Test

I recognized this problem over a year ago when I first started programming with AI. I wanted a way for an agent to remember work across sessions, so I started by creating an internal/ folder where I asked the agent to write everything down: specs, plans, indexes. I tasked the agent with always reading the appropriate documents and index before doing work and updating afterwards.

This set of rudimentary instructions slowly transformed into a formal system, and then into a plugin called Operator Memory that I’ve been using regularly within all of my projects.

Operator Memory provides document-based memory using the model described above. Operator provides your agent with a Markdown brain where it can persist important knowledge: instructions, specs, research, indexes. Before working, the agent always consults the brain for relevant documents. After working, the agent updates the brain; revisiting stale documents, adding documents where missing.

No vector databases. No embeddings. No summarizers, curators, updaters, dreamers, or any other token-burning background daemon. No black-box retrieval.

With Operator, everything is a plain Markdown document that you can read, update, commit, and share with your team. I’ve been using this system for over a year. If you want to try it out, it’s free and open source: https://github.com/aerovato/operator-memory

GitHub was built for a world where humans write code, organize it into repositories, and collaborate through branches, commits, issues, and pull requests.

But the next generation of software is going to be built differently because it is going to be built by a different kind of developer: agents.

Agents are already writing more code than ever before — they’re fixing bugs, building features, writing tests, reviewing changes, updating dependencies, and doing the routine maintenance required to keep an application running.

So in this new world where you have hundreds, or even thousands, of agents working on the same codebase at the same time, what does the foundation look like?

How do agents know what other agents are working on? What happens when they make conflicting changes? How do you review everything they produce? How do you keep track of not just what changed, but why a change was made?

And so the burning question is: What does the next GitHub look like?

We want you to help us answer it, by building it out.

Earlier this year, we launched Artifacts, a versioned filesystem that speaks Git and can scale to millions of repositories. From the start, we designed Artifacts as a set of programmable primitives that developers could use to build their own products, workflows, and abstractions.

Artifacts provides the foundation: repositories that can be created and forked programmatically, versioned storage for code and agent context, and the Git operations agents already know how to use.

With that foundation in place, you can focus on the layer above it: how agents coordinate their work, how changes are reviewed and merged, and what the developer experience should look like when hundreds or thousands of agents are working on the same codebase.

That is the layer we want you to build.

Now that Artifacts is in open beta, we’re holding a competition to see who can build the next Git platform on Cloudflare using Workers and Artifacts.

Artifacts is in open beta. Here’s why you should build on it

When we launched Artifacts, our goal was to make it possible to create a repository for every agent, session, task, or user — and to do that at the scale agents require.

Since then, we’ve seen developers use Artifacts in a range of ways: Vibe-coding platforms are using it to store the projects their users create. Developers are using it to persist the code and context from agent sessions. Others are creating isolated repositories, so multiple agents can safely work from the same starting point and compare or merge the results later.

Here are some new capabilities we’ve added since the initial launch.

Deploy Artifacts repos to Workers

You can now connect an Artifacts repository to a Worker through Workers Builds. When you or an agent pushes code to the Artifacts repository, Cloudflare will build the project and, for the production branch, deploy the updated Worker. Pushes to other branches automatically create or update Workers Previews, giving you an isolated, shareable version of your Worker where you can test changes before they go live.

You can connect an existing Worker to an Artifacts repository or start a new project and automatically store it in Artifacts.

Manage Artifacts directly from Workers

You can interact with Artifacts repositories directly from a Worker using an Artifacts binding to create or fork repos, inspect files and commits, and issue repo-scoped Git tokens. This makes your Git workflow programmable. When a new task arrives, a Worker can fork the project for an agent, read the files it needs for context, and give it a repository to work in. When the agent pushes a change, your automation can inspect the result and start a review. You define those steps in code to fit how your agents work.

For example, here’s how to fork a project for a new agent task and read its AGENTS.md for instructions:

using project = await env.ARTIFACTS.get("my-project");
const { defaultBranch } = await project.info();
const workspace = await project.fork(`task-${crypto.randomUUID()}`);

using repo = await env.ARTIFACTS.get(workspace.name);
const instructions = await repo.readFile({
  ref: defaultBranch,
  path: "AGENTS.md",
});

const agentTask = {
  remote: workspace.remote,
  token: workspace.token,
  instructions: instructions ? await instructions.text() : null,
};

React to every change with event subscriptions

Artifacts publishes events whenever a repository is created, imported, forked, deleted, pushed to, cloned, or fetched. You can subscribe to these events to decide what happens next: run CI, kick off a code review agent, or deploy a change.

For example, you can subscribe to Artifacts push events and have a Worker start a code review workflow for each push. The Worker passes the repository, branch, and new commit to the Workflow, giving a review agent the context it needs to inspect the change:

export default {
  async queue(batch, env) {
    for (const message of batch.messages) {
      const event = message.body;
      if (event.type !== "cf.artifacts.repo.pushed") continue;

      await env.REVIEW_WORKFLOW.create({
        params: {
          namespace: event.source.namespace,
          repo: event.source.repoName,
          ref: event.payload.ref,
          commit: event.payload.after,
        },
      });
    }
  },
};

Data jurisdiction for Artifacts repos

You can now choose where Artifacts stores and processes your repository data. Set a U.S. or EU jurisdiction when you create a namespace, and every repository created in that namespace will automatically follow the same restriction.

curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/artifacts/namespaces" \
  -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  --json '{"namespace":"my-eu-namespace","jurisdiction":"eu"}'

View Artifacts metrics

You can now see metrics for your Artifacts repositories in the Cloudflare dashboard. For each repository, you can now see total operations, pulls, pushes, errors, and error rate, helping you understand how the repository is being used and spot failures. You can also query Artifacts metrics directly to build your own dashboards or monitoring.

Pricing

Artifacts pricing is based on repository operations and the amount of data stored. We will begin billing for Artifacts usage on October 15, 2026.

Competition: Build the next Git platform on Cloudflare

We want you to build your vision for the Git platform of the agentic era using Cloudflare Workers and Artifacts.

You could rethink repositories, branches, pull requests, worktrees, code review, and merge conflicts — or build new ways to preserve agent context, compare multiple changes at the same time, and decide which one should ship.

We aren’t looking for GitHub as it exists today with agents added on top. At a minimum, we want to see multiple agents working on changes concurrently. Beyond that, we want you to get creative — what you think comes next.

How to enter

Submit:

• A 5-10 minute video demonstrating what you built, what it enables agents and developers to do, and how it works
• A link to the source code, which must be provided under a permissive open source license (MIT, Apache, BSD)
• Instructions for running or trying the project

Deadline

Submissions are open until October 14, 2026.

Why should you participate?

We’ll select the top three projects and fly up to two members from each team to San Francisco to attend Cloudflare Connect and show what they built.

The first-place team will also receive $25,000 in Cloudflare credits, along with invitations to the VIP speaker dinner on Monday night at Connect.

Get started

Artifacts is available in open beta to customers on the Workers Paid plan.

Get started with your coding agent: copy the prompt below to set up your first Artifacts repository and start pushing code to it.

You can view or create the Artifacts repositories in the dashboard or if you’re looking to learn more, check out the documentation.

Today, end users carry too much of the burden of online privacy. To avoid third-party trackers or targeted ads, users are instructed to use a VPN, disable cookies, or install adblockers. Meanwhile, some app developers end up knowing more about their users than they’d care to: a typical client-server exchange creates a trail of user data, like the client’s IP address or TLS fingerprint. This level of visibility can be a burden.

That’s why Cloudflare builds infrastructure that helps developers bake privacy into their apps. Oblivious HTTP (OHTTP) is an IETF standard designed to enable app backends to receive HTTP requests without seeing user IP addresses.

This fall, we’re launching the Cloudflare OHTTP Gateway. Customers will be able to enable our new OHTTP Gateway as a paid add-on to their zone and start receiving OHTTP traffic with just a few clicks. Register through our form to join our waitlist. Read on to learn more.

Expanding our OHTTP product suite

With OHTTP, requests travel through two independently-operated hops: a relay and a gateway. An OHTTP relay blindly forwards encrypted requests in order to hide client identifiers from app servers. An OHTTP gateway performs the cryptographic work of decapsulating encrypted requests and encapsulating responses such that app servers can handle OHTTP requests as if they were plain HTTP. The separation of trust between relay and gateway is critical: it ensures that no single party sees both client identifiers and request contents.

In 2022, we launched an OHTTP relay product, Privacy Gateway. Privacy Gateway enables our customers to offer more privacy-preserving experiences to their users. For example, Flo Health uses OHTTP for their app’s Anonymous Mode, and Apple’s Private Cloud Compute uses OHTTP to disassociate AI inference requests from user identities. But customers who are already protecting their servers behind Cloudflare can’t also use a Cloudflare-operated relay — they need an OHTTP gateway instead.

With the existing Cloudflare OHTTP Relay, customers must bring their own Gateway to preserve a separation of trust.

In our experience running OHTTP relays, we’ve seen how difficult it can be to build and operate a secure, performant OHTTP gateway at scale. Today, we’re launching the closed beta for our self-serve Cloudflare OHTTP Gateway. We’re also renaming our “Privacy Gateway” to “Cloudflare OHTTP Relay” to better distinguish the two products.

Now, customers who want an OHTTP architecture with the necessary separation of trust have two options:

1. Use Cloudflare’s OHTTP Relay (formerly Cloudflare Privacy Gateway) and run your gateway yourself. This is best if your application servers are hosted off Cloudflare, and you’re able to run your own OHTTP gateway.
2. Use Cloudflare’s new OHTTP Gateway with a third-party relay. This is best if your app servers are already behind Cloudflare (on our CDN or Workers, for example), if you’re accepting OHTTP requests from a third party (like Apple’s LiveCallerID), or if you want a managed gateway to minimize latency and operational overhead.

We’re working to raise the bar for privacy across the Internet, and we believe that protocols like OHTTP can help — if we make them easy enough to adopt. It’s always been our goal to expand our OHTTP product suite and make our trusted privacy infrastructure accessible to a broader swath of the Internet.

Why we built the Cloudflare OHTTP Gateway

Since we launched our OHTTP Relay product, we’ve observed a few things.

First, we’ve seen that there's a growing appetite among developers for accessible, usable privacy infrastructure. Developers of privacy-oriented apps want to bake network privacy into their applications by default, but doing so remains harder than it should be.

Second, we’ve learned that building and operating an OHTTP gateway can be tough for customers. Any proxying architecture introduces some latency because requests must travel an extra hop or two around the Internet. Combine that with the cost to decrypt requests and encrypt responses, and the latency hit of a homegrown OHTTP setup can be significant. We’re well-positioned to solve this problem: the same building blocks that enable us to operate fast, reliable privacy infrastructure for products like 1.1.1.1 and iCloud Private Relay make us a good home for an OHTTP gateway. Because of Cloudflare’s anycast approach, our OHTTP Gateway will run on every server on Cloudflare’s global edge network, minimizing latency in relay-to-gateway hops. If you use our CDN, user requests can be decrypted by our Gateway and resolved by your app servers on the same Cloudflare metals, saving gateway-to-origin latency.

Finally, recall that OHTTP’s privacy model requires that the relay and app server be operated by separate, non-colluding parties. We want to provide our customers with the best possible range of options for their privacy infrastructure. Before, developers who protected their app servers behind Cloudflare weren’t able to use our OHTTP Relay, because Cloudflare would see both client metadata and the decrypted contents of requests, breaking OHTTP’s privacy model. Now, developers can choose whether a Cloudflare OHTTP Relay or Gateway is a better fit for their architecture.

A primer on OHTTP

A typical interaction between a client and application server reveals information about the client. When a client and app server talk to one another, the app server learns the client’s IP address because each packet in which data is sent is labeled with a source IP — similar to the “from” label on an envelope. App servers can also “fingerprint” a client based on attributes like supported TLS versions or cipher suites. These signals make it possible for app servers to link multiple requests back to the same user.

But what if I wanted to build an app that really doesn’t know much about my users? For example: Flo Health wanted to build an Anonymous Mode to enable users to access personal health data without it being linkable to possible user identifiers.

OHTTP introduces a proxy, called a “relay,” that forwards requests and responses between client and app server to obfuscate the client’s identity from the app server. The relay sees client identifiers like IP address and TLS fingerprint, but strips them before forwarding on requests. This prevents app servers from linking multiple requests back to the same user, and means that request contents can’t be associated with the user’s IP address.

For example, a regular client-server exchange might reveal the following information about a client:

- ipAddress: 192.0.2.33 # the client’s IP address 
- ASN: 7922
- tlsCipher: AEAD-CHACHA20-POLY1305-SHA256 # potentially unique
- tlsVersion: TLSv1.3
- Country: US
- Region: California # the client's location
- City: Campbell

A request first sent through an OHTTP relay would reveal only the relay’s information to the app server receiving the request:

- ipAddress: 128.62.37.13 # the relay's IP address & fingerprint 
- ASN: 18 
- tlsCipher: AEAD-AES-128-GCM-SHA256 
- tlsVersion: TLSv1.3 
- Country: US
- Region: Texas  # the relay's location
- City: Austin

This means that for each request, the app server doesn’t learn the location and TLS fingerprint of the end user. Plus, if many different users are sending requests through the relay, the app server won’t be able to distinguish which requests are coming from whom, limiting their ability to trace app activity back to a single end user. This creates a strong privacy boundary.

What really differentiates OHTTP from a basic forwarding proxy, however, is the encryption of data between client and app server. Requests and responses are encapsulated using Hybrid Public Key Encryption (HPKE) such that only the client and app server can see plaintext, and the relay sees only a jumble of ciphertext. A “gateway” sits between the relay and app server to handle all of this cryptography — decapsulating requests, encapsulating responses — and the app server handles only plain HTTP.

This creates a “double-blind” privacy model: the relay sees only client identifiers; the gateway and app server see only request contents; no party sees both.

How we built the OHTTP Gateway

In building our OHTTP gateway-as-a-service, our goal is to bring our secure, performant privacy infrastructure to a broader swath of the Internet. Performance and easy onboarding are critical. So, we built our Gateway as a flexible service deployed across our global network. With just a couple of clicks, you can enable the Gateway on your zone and start sending OHTTP to https://your-zone.com/.well-known/ohttp-gateway. We’ll scale the service up and down automatically, so you don’t need to worry about capacity.

We had a few other user needs in mind, informed by the pain points we’d seen OHTTP Relay customers run into when operating their own OHTTP gateways.

First: We wanted to abstract away as much of the complexity of OHTTP as possible for your app servers. We wanted developers to be able to start receiving OHTTP while continuing to accept regular HTTP traffic if they chose. So, we designed the Gateway as a feature of your zone, where clients send well-formatted OHTTP requests to a /.well-known/ohttp-gateway endpoint on your zone. We support both standard and chunked OHTTP — and we recommend using chunked OHTTP for better performance, because it enables us to process requests incrementally (in “chunks”).

Our Gateway service will intercept each request, decrypt it, issue a subrequest to your app server, and return an encrypted response to the client. All non-OHTTP requests will travel to your server without invoking the Gateway.

Binding your Gateway to your zone also enables us to protect your Gateway from abuse. A client sending requests to your zone example.com may send to foo.example.com or bar.example.com, but not wikipedia.com. Without you needing to worry about it, this prevents unauthorized clients from using your zone as a way to target other domains.

Second: Seamless key management is critical. Gateways need to maintain a public HPKE key configuration to enable clients to encrypt requests, but managing keys securely is a challenge. So, we designed the Gateway to fully manage all keys for customers, and to serve public keys as responses to GET requests to /.well-known/ohttp-gateway. For stronger privacy, clients can download keys over a different IP than they request the gateway.

Third: Gateways need to be able to authenticate relays. Because the Gateway (by design) knows very little about the client sending a given request, it places trust in the relay to authenticate clients and forward traffic responsibly. But how do you ensure that only trusted relays can send traffic to your gateway?

We designed the Gateway such that Cloudflare Access, Cloudflare’s zero trust network access product, runs before requests are decrypted, enabling you to use any standard Access policies to authenticate incoming traffic and protect your Gateway from abuse. Options include mutual TLS, static service credentials, and custom external logic.

Finally: Mistakes happen, and we anticipated that customers might accidentally break OHTTP’s privacy model by running both their relay and gateway on Cloudflare. So, to preserve OHTTP’s separation of trust and ensure that Cloudflare never sees both client identities and decrypted inner requests, our Gateway will refuse to decrypt requests sent from Cloudflare Workers or from proxied hosts on Cloudflare.

When is the OHTTP Gateway a better fit than the OHTTP Relay?

If you want to use Cloudflare’s OHTTP product suite, but you’re wondering why you’d pick Cloudflare’s OHTTP Gateway instead of the OHTTP Relay, here are a couple of considerations.

First, do you want your app servers on Cloudflare – behind our CDN or built on Workers, for example? If so, the OHTTP Gateway is a better fit to ensure adherence to OHTTP’s privacy model.

Second, what’s your use case? If you want to receive OHTTP requests from a third-party client and relay — to use Apple’s LiveCallerID SDK, for example — then the OHTTP Gateway is likely the better solution for you.

Getting started

If you have a feature request or would like to register for our waitlist, so we can notify you when the product launches, sign up here.

Then, you’ll need to implement an OHTTP client. See ohttp.info or our sample client library for some examples to help you get started. One flag as you build the client: OHTTP provides privacy at the network level, and doesn’t touch the inner request body. So, to preserve user privacy, it’s up to you not to send identifying information (e.g. a user’s email address or username) in the request body.

Next, you’ll need to bring your own relay. Relays can run on any infrastructure provider, and they’re simple: here’s some sample code. The challenge and the reason you might want a dedicated OHTTP relay provider, is to verifiably promise to your users that you won’t inspect logs with client identifiers. Otherwise, you’d be able to correlate clients at the relay with decrypted requests at your app servers.

Finally, once your OHTTP deployment is live, check out our pvcli client to help with testing and debugging.

We’re excited to bring accessible privacy infrastructure to developers everywhere. Reach out to us if you’d like to try out the new OHTTP Gateway and raise the bar for privacy online.

What does this methodology measure anyway?

It would shock you to know that Costco, according to a new Harvard School of Public Health methodology, caused 120,687 cases of cancer mortality nationally every year, simply due to living in close proximity to their warehouses. Private colleges accounted for 83,782 more. Harvard itself is responsible for 5,842 cancer deaths. Living near a Major League Baseball field causes almost 19 times as many cancer deaths as living near a Major League Soccer pitch. These conclusions are obviously absurd, but they come from the exact same methodology that researchers at Harvard use to “prove” that nuclear power plants cause cancer in nearby communities.

Earlier this year, Adam Stein and I called out a series of Harvard studies showing that nuclear power plants were associated with higher rates of cancer incidence and cancer mortality as nonsense. Now that we’ve spent the last few months replicating the analysis and expanding it, we are able to show that no matter what landmark you use, their methodology will show an increased cancer risk and mortality. The studies simply do not prove anything about cancer incidence.

Let’s recap: In December 2025, researchers led by Yazan Alwadi at Harvard’s T.H. Chan School of Public Health published a paper in Environmental Health that claimed to find that cancer incidence increased for people living closer to nuclear power plants in Massachusetts. In March, the same researchers published an expanded nationwide study claiming a similar result—this time looking at cancer mortality rates, rather than incidence—in Nature Communications. This was followed by a paper in the Journal of Exposure Science & Environmental Epidemiology that looked at associations of lung, breast, and colon cancers. Most recently, a study of total mortality, not just cancer, was published in the European journal Environmental Epidemiology.

The papers construct a “proximity score” based on distance from nuclear plants, up to 120 km in Massachusetts and 200 km in the national studies (about 75 and 125 miles). Every ZIP code or county inside those radii is treated as exposed, with closer locations receiving higher weights. If a ZIP code or county is within range of multiple sites, then the effect is cumulative. That means that a county that is in close proximity to one nuclear power plant could have a smaller score than another county that is further away from any one power plant but within range of many.

From this proximity score, the authors run regressions testing the outcome (cancer incidence, cancer mortality, or total mortality) on the proximity of the county and a collection of covariates. From this, they use the statistical coefficients, construct the Relative Risk for each county, age, and sex group, and compute the Attributable Fraction, from which they derive the number of attributed deaths from the nuclear power plant from 2000 to 2018. The authors state that their methodology and results provide the scientific basis and public health justification for an expanded research program.

But proximity is not exposure. We have methods of measuring exposure for nuclear power plant workers. While the radiation exposure of nuclear workers will always be greater than or equal to that received by the surrounding public, most of the closely monitored US nuclear workforce receive no measurable annual dose. When workers are exposed to radiation, the average dose received is only 2 percent of the occupational limit. If operators and workers who are on-site at nuclear power plants receive an annual dose between zero and one-fiftieth of the occupational limit, how is it possible that residents 5, 10, 25, 50, 120, or 200 kilometers away would receive any measurable dose from the same plant?

In talks, the authors hedge that their papers are merely ecological studies that show association, but never prove causality. Ecological studies are used to understand the relationship between outcome and exposure at a population level. This leads us to ask: what would the mechanism of exposure be? Well, according to the authors, we can just ignore the broader literature and physics, and instead make up exposure pathways and mechanisms. The use of an “ecological study” allows a lot of leeway in terms of explaining the broader world.

Over the last several months, we have replicated the results of these papers. The authors supplied us with eight lines of code and answered a couple of questions about the covariates, which did not replicate the results. Most of our replication was done through first principles combined with trial and error. Once we were reasonably close to the results of the first national study on cancer mortality, we took the methodology and applied it to numerous other landmarks.

Ridiculous things you can “prove” caused cancer mortality

Everything causes cancer. Sounds cliché; maybe those California warning tags were right all along. But thanks to the methodology created by Alwadi et al., we can now prove that anything and everything causes cancer. Oh, sorry, that is too strong of an assertion. To use the authors’ words, since they have claimed their papers don’t prove causality, we can create an association between any physical landmark and cancer and, from that association, figure out how much cancer is attributable to that thing. Which is totally not the same as saying “that thing causes cancer.”

For instance, living near a private four-year university is associated with a 15-fold increase in cancer mortality when compared to living near a nuclear power plant.

Costco has the largest effect of all the locations we have tested. Over 2.2 million cancer deaths can be attributed to Costco; that’s more than 20% of all cancer deaths between 2000 and 2018. Hot dogs, bulk spices, and reasonably priced clothes come with a cost. But it goes to show that the authors’ choice of nuclear power plants wasn’t serious. Private 4-year colleges, including Harvard, are associated with the deaths of 1.59 million people, public 4-year colleges 626,212, and Superfund sites 895,258. Once you include the attributable deaths from state capitals at 132,389, we can account for 5.54 million cancer deaths, over one-half of all cancer deaths, between 2000 and 2018.

If we look at professional sports venues, the spread looks even more bizarre. Living near an NFL stadium was associated with 418,520 deaths, NBA arenas 471,794, MLB fields 944,769, NHL rinks 126,751, and MLS pitches 50,889. None of these sports have any kind of emissions that leave the playing field, except for the occasional home run or foul ball. The difference across the sports might be the kinds of concessions being sold. Baseball is right up there with Costco, selling copious amounts of hot dogs. Perhaps most surprising of all in the realm of professional sports is NASCAR, which accounted for 79,278 cancer deaths. One would expect the emissions, noise, smoking, and copious amounts of light beer to be more impactful than the emission-free ball and puck sports, but moving closer to a speedway might just save your life. With sports added to the mix, over 7.85 million cancer deaths can be attributed across a variety of landmarks; that’s 3/4s of all cancer deaths between 2000 and 2018.

The supposed 115,586 attributable deaths from nuclear power plants look like chump change to the real perpetrators. Harvard, when singled out, accounts for 111,003 deaths. That’s just one institution threatening every community within 200 km. Once again, we are left asking what the mechanism could be. Plus, our study only goes from 2000 to 2018; the school has been around since 1636, so the actual death toll could be much higher.

According to a webinar presentation given by Dr. Koutrakis, one of the coauthors, the supposed release of radioactive effluence from nuclear power plants comes from the occasional refueling of the reactors, when operators open the reactor core, and radioactive dust escapes from the fuel assemblies, past the containment vessel, through the outer shielding, and out into the world without ever being detected by the myriad number of dosimeters and radiological sensors throughout the facility. If we really are so terrible at measuring radionuclides (we are actually very good at it), then it is entirely plausible that Harvard could have unlawful access to special nuclear materials and the brains of Cambridge could have built a nuclear reactor under the squash courts without the proper licensure.

What does the methodology actually show?

Cancer is a scourge across the human race. You have a ~40% chance of getting cancer sometime in your life. Which is why we should take cancer studies very seriously. If one is showing a surprising result, we want to get to the bottom of it. All kinds of claims can be made with statistics, but if we want to make good policies based on evidence, it is worth spending time making sure the evidence upholds its claims.

In our original rebuttal, we remarked that the studies can’t prove their assertions because they lacked proper control. It seems they also neglected to do any placebo testing. They chose nuclear power plants because a story could be built around that framework. When the researchers got positive results across our nation’s nuclear power plants, they didn’t check what their shiny new methodology would do using other landmarks. This is their pitfall: by taking the easy way out—getting results and making up a story around those results without double-checking their method—the authors could have no idea that what they were actually capturing was the methodology itself. In our replication and expansion, we have shown that choosing a number of sites and landmarks from capitals to warehouse stores can all yield a positive result without any reasonable pathway for American citizens to become exposed to radiation, develop cancer, and then die. The variations are random noise, all sloping in a positive direction. Even when using randomized outcomes, the methodology outputs positive results.

The papers by Alwadi et al. don’t show a novel mechanism by which nuclear power plants meaningfully contribute to cancer mortality; instead, they show an association of data points across 400 km-wide circles. 48,519 square miles, about the size of Mississippi, is a massive area to claim any kind of exposure. Most studies measuring distance-based exposures look at much smaller distances, such as under 10 km (an area of 113 square miles). The methodology is blind, which can be a feature in research areas needing to avoid bias, but in this case it is a bug; it doesn’t understand radiation exposure or dose. All it understands are its inputs: coordinates, proximity, covariates, and cancer deaths. From these, it can give you a number and even a positive result, but it cannot explain why that result exists. It is still an ecological study, one with a sophisticated statistical technique, but not a very useful one.

Sophisticated statistics can produce extremely precise nonsense when the design fails to identify the causal mechanism. Bad science like this is especially dangerous when applied to something culturally plausible that gets a positive result—it is easy to twist a result into a compelling narrative, especially if it confirms one’s biases. Nuclear power plants create energy through radiation, and radiation causes cancer; therefore, nuclear power plants cause cancer.

It is especially telling that no matter what landmark we applied to the methodology, we have yet to get a negative result. There is, in fact, a real chance that you simply cannot get a negative result from this method.

The attributable number of deaths from this methodology is probably zero, but the attributable number of bad papers is at least four.

The coming proxy wars will be devastating for civilians and do little to improve governance and human welfare in Ethiopia and the wider Horn

Thank you for being a regular reader of An Africanist Perspective. If you haven’t done so yet, please hit subscribe to receive timely updates on new posts along with over 36,000 other subscribers. New regular content is free. Book reviews and the archives are gated.

I: On Ethiopia’s wars of choice, motivated by historical grievances and elite hubris

Source: Wikipedia

This is from the ICG:

Northern Ethiopia is slipping into renewed war. After months of building tensions, on 23 September large-scale hostilities broke out between the Tigray People’s Liberation Front (TPLF) and Amhara Fano militias, on one side, and the federal government and allied Tigrayan forces on the other. Responsibility for the escalation lies primarily with the TPLF, which launched ground assaults in coordination with other rebels with whom it has joined hands in a new alliance that vows to overthrow Ethiopia’s government. The upsurge in fighting comes almost five months after the TPLF retook power in the Tigray region, dealing a blow to a 2022 peace deal that had ended the last Tigray war and leading Addis Ababa to mount an intensifying drone campaign against TPLF forces.

I maintain that war in Ethiopia (and the wider Horn) is not inevitable. The current outbreak of war reflects the fact that Ethiopian elites keep choosing war over peace. Unfortunately, despite repeatedly choosing war, Ethiopian leaders continue to enjoy support from a cast of warmongering academics, journalists, public intellectuals, civil society actors, and a whole host of foreign actors — all of whom have long taken sides and become cheerleaders for their respective camps.

Let’s be blunt. There are a lot of things not to like about the Abiy Ahmed administration. The strong elements of autocratic intolerance, often expressed in the form of indiscriminate force. The unwillingness to (publicly) countenance alternative worldviews and policy positions. The abiding reliance on the old vindictive governance styles anchored around identity politics and collective punishment. And the hubristic promotion of a personality cult, which makes it difficult to build the broad and durable coalitions needed to govern a diverse country like Ethiopia.

Perhaps the political situation would be marginally better if Prime Minister Abiy were a better politician.

However, I doubt it. To understand why, it’s worth stepping back to appreciate the structural drivers of elite instability and conflict in Ethiopia:

The last three decades in Ethiopia have been a search for a new myth. The ethno-federalist system had legitimate logic: bringing about the dignity of (cultural and linguistic) difference between nations and nationalities. However, its rhetoric was drawn from the difficult past instead of the hope of better future. To make matters worse, it became a breeding ground for social and economic injustice.

In the absence of farsighted political elites who may have been able to craft a new inclusive myth out of the stories of nations and nationalities, ethnic groups had to walk back to find their stories in their own small compartments. This exacerbated narrow ethnic histories and ideals.

It’s easy to demonize Abiy. But the hard truth is that Ethiopia’s structural problems preceded him and, if left unresolved, will outlast him.

Those beating the war drums the loudest in Ethiopia — and who’ve trashed the Pretoria Agreement — are convinced that Ethiopia can only be well governed if they are in charge. Nearly all are motivated by maximalist demands and are quick to weaponize legitimate historical grievances for parochial ends. The leading characters involved are (mostly) men out to settle decades-old scores and who prefer to go out fighting rather than accept political defeat by their perceived inferiors. These same characters have been able to mobilize fighters under the banner of uncompromising primordislist ethno-nationalism and commitments to their respective ethno-states.

In light of all this, Teklehaymanot G. Weldemichel is spot in making the case for following through on the Pretoria Agreement. Yes, it is imperfect and the various belligerents haven’t always followed through on the deal’s requirements. However, the deal “brought large-scale fighting to an end and contains commitments that address some key unresolved issues.” It’s a start. It ought to be given a chance.

Unfortunately, the plight of civilians hardly ever features in the calculus of the warmongers. Estimate vary, but the death toll in the Tigray war may have been as high as a staggering 600,000. This is a monumentally catastrophic figure. Even the lower bound estimates out there would make it one of the deadliest wars of the 21st century. Given what we know about how belligerents go about waging war in Ethiopia (including by exacerbating the problem of food insecurity), why would anyone support a return to all out war?

Why are Ethiopian elites and their cheerleaders willing to impose such trauma on their peoples?

II: The wages of ethnic federalism

There is bound to be a lot of discourse on ethnic polarization as a driver of conflict in Ethiopia. However, it is worth also considering how Ethiopia’s brand of federalism contributes to ethnic polarization. Existing research shows that the creation of ethno-states after the end of the civil war increased the salience of ethnic difference:

[I]ndividuals politically socialized under the conditions of ethnic federalism are indeed significantly more likely to embrace an ethnic rather than a national identity. However, generational affects do not appear to be related to perceptions of continuing ethnic federalism as a political arrangement. Differences of opinion on ethnic federalism are more a function of one’s group identity, rather than the intensity of ethnic identification.

This is in line with the historical experience from other jurisdictions (see Lebanon, the UK, etc). Institutionalizing ethnic difference through administrative means or identity-based reservations creates incentives to invest in the targeted identities as loci of political mobilization. From that point it’s a short walk to mobilizing those same differences for conflict under permissive conditions. Notice the implication of the last line in the above quote: once ethnonationalism becomes the dominant game, everyone gets forced to pick a side, regardless of their individual-level intensity of ethnic identification.

Most people kind of admit that a leading structural driver of political instability and conflict in Ethiopia is its brand of federalism:

Ethiopia’s federal system was flawed from the beginning because it didn’t foresee potential sources of conflict or that regional states would make claims against one another. Trust among regional states was never high, and has deteriorated over the last three decades. On top of this, the federal government’s ability and readiness to mitigate or solve domestic conflict has been open to question. Currently, there are regions and regional leadership that are having difficulty working together. The federal government at the centre is too weak to impose its will on the regional administrations. The result is that there aren’t common political and economic national standards across the country.

To be clear, Ethiopian civilians do not want war; and also don’t trust the state’s ability to avoid or end wars. And while reported inter-ethnic hostility remains low (despite the historical wars), the rate of encounters with state-directed ethnic discrimination ranks among the highest in Africa.

In Round 9 of the Afrobarometer Survey, 55% of respondents noted that the state is doing badly at preventing or resolving violent. In the same survey, less than a quarter (23.7%) of respondents reported feeling that their ethnic identity trumps their Ethiopian identity. A roughly similar proportion (23.6%) also reported not at all trusting people from other ethnic groups. Meanwhile, 39.3% reported feeling that their ethnic group is never treated unfairly by the government.

There are two takeaways from this. First, the problem appears to be the state/institutions rather than society. The vast majority of Ethiopians seem open to the idea of multicultural co-existence. However, the state (and supporting institutions — e.g., ethnic federalism) seem intent on balkanization by reinforcing difference in everyday interactions. Second, while Ethiopia is in the bottom fifth of the distribution across the Continent, it’s not particularly unique (unlike what many Horn scholars would like to believe). There are other countries with close or worse results on the metric of state-directed ethnic discrimination.

Overall, the most important driver of conflict in Ethiopian is the state’s centuries old inability to deter or crush rebellions in their infancy. All polities have pockets of grievances against the state that could potentially give rise to armed rebellion. While some grievances invite legitimate armed resistance, others are the manifestations of taste-based warmongers, war profiteers and their clients, and useful idiots carrying water for geopolitical adversaries. Regardless of their founding motivations, not all would be rebellions materialize — because capable states don’t let them.

Instead of monopolizing the use of violence, the Ethiopian state has adapted to using indiscriminate violence and chaos as governance styles (to manage intra-elite political instability, and to fight back against mass level identity-driven centrifugal forces). Collective punishment of whole communities is common, and has, in turn, radicalized ever more people against the state. Consequently, there won’t be any shortage of young people ready to take up arms against the state in this round of wars.

III: The regional/geopolitcal dimension

The Ethiopian state’s inability to deter or eradicate violence invites all manner of foreign meddling. This is terrible for conflict duration, as it prevents both state and non-state belligerents from fully internalizing the cost of war.

Lots of countries — including Egypt (the Nile/wider Red Sea supremacy), Eritrea (Ethiopia’s desire for access to the sea), Sudan (Ethiopian support for the RSF), Somalia (historical enmity/Ethiopian support for Somaliland), and Saudi Arabia (as benefactor of Egypt/Sudan/Eritrea axis) — have good reasons to keep Addis Ababa weak and preoccupied with unending domestic wars. These dynamics, coupled with the geography of war in Ethiopia mean that Addis Ababa will struggle to keep arms and fighters from easily moving across its borders with Sudan and Eritrea.

On its part, the Ethiopian state will likely continue to enjoy support from the United Arab Emirates; and maintain the capacity to produce the weapons needed to keep rebels at bay. Ominously, divisions among elites in Tigray and other regions mean that the rebellion may not clear the threshold needed to incentivize a quick resolution either militarily or at the negotiating table. Therefore, slow-burning, but still catastrophically deadly conflicts may be on the cards.

A key risk facing Ethiopians is that, like in Sudan, international/geopolitical interests and considerations may quickly overshadow the domestic cost of the war, especially on civilians. The inability to extricate the domestic bargaining “game” from regional/geopolitical considerations will unnecessarily complicate peace negotiations and very likely prolong the conflict.

Notably, a prolonged conflict will likely also be significantly civilianized. Careful to avoid being overstretched, the Ethiopian state may decide to deputize regional militias and other violence entrepreneurs in its fights against subnational armed groups. To this end, it will lean heavily on divisions across armed groups and within the regional states that have witnessed rebellions.

IV: Tough questions to ask now and after the wars get concluded

As noted above, it would be a mistake to characterize political instability and conflict in Ethiopia as being caused by any single leader. Removing Abiy from power — the stated goal of the latest armed coalition against the Ethiopian state — will not solve the many structural problems facing Ethiopia. Which is why it is important for all involved to have honest conversations about the fundamental causes of conflict, and how to get out of the cycles of violence. Four issues immediately come to mind:

1. The thorny question of what it means to be Ethiopian, in a context of diverse peoples, nations, and nationalities: What does it mean to be Ethiopian? Should Ethiopians deal with their rich cultural tapestry and history by genuflecting to ideas motivated by narrow ethno-nationalisms or embracing cosmopolitan multiculturalism as a national goal? These questions do not have easy answers, and I cannot pretend to offer any here. However, it’s high time Ethiopians openly confronted these questions rather than ceding ground to those out to weaponize the country’s history to stoke ethnic tension and wage perpetual wars of self aggrandizement.
2. The meaning of self-determination in a cosmopolitan 21st century state: What is the correct structure of the Ethiopian state? And what should be the balance of power between national and subnational units? If the resolution to the questions above converge on keeping Ethiopia whole (as I hope is the case), the next question would be to come up with institutional and administrative structures that at once give ordinary Ethiopians a sense of self-determination at the local level and empowers the center to prevent centrifugal tendencies from collapsing the state.
3. How to stabilize Ethiopia’s regional relations: While Ethiopia’s problems are principally domestic, it’s also true that its neighborhood (comprised of states with histories of internal conflict) and antagonistic foreign posture (especially vis-a-vis Somalia and Eritrea) exacerbate this problem. Of course there are things that Addis Ababa cannot change about its foreign relations (e.g., need to check Egypt’s supremacy ambitions). However, there are others it can. It can choose to pursue a path that ends the proxy wars with Sudan, Eritrea, and Somalia. This will reduce overall conflict risk in the wider Horn, while also starving would-be Ethiopian rebels from getting ready support from regional rivals and generally reducing the size of the Horn’s war economies (arms, fighters, ideologies).
4. What’s the positive case for Ethiopian unity? Is it to reclaim the country’s imperial glory? Is it to aggrandize the person of the Ethiopian leader? Is it to become a geopolitical powerhouse in the Horn and the Red Sea regions? Or is it to deliver economic growth, development, and prosperity for Ethiopians? Some commentators like to compare Ethiopian with the former Yugoslavia, thereby raising the specter of partition. This should be viewed as a challenge for those that want to see a united Ethiopia. If people are to continue buying into the Ethiopia Project, there has to be a positive case for doing so. Merely relying on history or inertia will not be enough. And importantly, the pitch has to be sufficiently inclusive and must accommodate changing public attitudes. In particular, Ethiopian elites must update and understand that the old political culture of violence, domination, and humiliation as styles of governance will not go down well with current and future generations of Ethiopians.

In the final analysis, and understanding that the sources of instability and conflict in Ethiopia are principally domestic, there is really no better way to put it that this:

Ethiopia’s federal government must be pressed to exercise restraint and return to meaningful political negotiation. Military force and coercion cannot become the first choice for resolving disputes. Tigrayan political and security actors must likewise recognise that political disagreements cannot be sustainably resolved through armed mobilisation. External powers should not be permitted to turn Ethiopia’s internal political disputes into a wider contest for influence. The involvement of the United Arab Emirates, Eritrea and others supporting the warring parties should be addressed through transparent diplomacy. Their competing interests risk fuelling a wider confrontation.

I’m an EMT (Emergency Medical Technician).

My main job is software engineering, but I’ve always been interested in medicine. A friend turned me onto the idea in college, and it rattled around my head for years. I delayed for a long time — until finally, in 2024, I took a month off and got my EMT.

Here are actual reasons I delayed getting my license, sorted loosely by ridiculousness:

Not ridiculous

1. It will take a lot of my spare time.

   Yes; my accelerated class (NOLS Wilderness EMT) took 28 consecutive days. I was only able to take the class because Microsoft switched to unlimited vacation, my boss was great, and I had the money to fly to Wyoming for the course (the usual alternative is a semester of night school!). And to be a good EMT, you have to actually take calls on an ambulance, hopefully more than the typical “1 weekend a month” minimum.

2. I won’t like the culture.

   Yes. This was a huge barrier: I had an awful experience with some first responders a few years after college — bad enough that I gave up on getting my EMT for a while.

   There are a lot of burned-out people in EMS. Many services — including the one I saw — are incredibly fratty, to be charitable. Lord help you if you are a woman. You will hear many excuses for these issues, but, yes, culture is a huge concern.

NOLS WEMT: sometimes you can have a good culture

Kinda ridiculous

3. I won’t fit in with the people.

   This is similar to culture, but a bit more silly. Yes, first responders tend to be a tight-knit, hereditary community. NYC firefighters, for example, often come from big firefighter families. But first responders (especially volunteers!) also come from all backgrounds — the person who onboarded me into my current position is a PM at another tech company.

4. I won’t fit in with the students.

   So maybe EMTs were ok — but what about the EMT students I would learn with? Wouldn’t they all be 18-year-old adventure junkies or jocks?

   Yes, most were, and they were awesome. My course was full of enthusiastic 18-year-old pre-meds who were excited to learn, and several people my age, who wanted to learn medicine for the same reasons I did.

   If you’re nervous, I think it helped that I chose a Wilderness EMT program in July. It filters to that crowd.

5. I want to focus on my day job instead.

   I would often consider studying EMS (Emergency Medical Services) when my day job wasn’t going well. And so it would always be a bit of a sign that I should refocus at work — onto more meaningful things.

   While that’s a good signal, it’s alright to put energy into things beyond work. And, once I got my certificate, balancing work & EMS has been straightforward.

6. I don’t want to work on an ambulance.

   Yup — I never dreamed of riding on an ambulance. Some people really do. It’s their vocation. It’s not mine.

   But no one forces you to step onto an ambulance once you have your EMT. You can just… know how to do CPR really well. That’s a really cool skill to have.

7. I don’t understand the scope of practice.

   If you’re not an EMT, it’s not super clear what an EMT can do. On the one hand, it’s not just “driving an ambulance,” but on the other hand, you need a paramedic for most serious calls. In some sense, it kinda is “knowing how to do CPR really well.” I trained with the same AEDs any layperson uses (EMTs aren’t allowed to use manual defibrillators), but I trained many, many more times than a layperson (plus I get to use oxygen).

   But, as before, the training is still pretty cool. I loved every bit of first aid training I got; EMT school was just as fun.

   

   I know how to use all this stuff!

8. It’s hard to find a volunteer job.

   Kinda. Big cities tend to have all-paid staff; volunteer positions are a bit farther out. But not that much farther out: even ignoring religious services like Hatzalah in NYC, Hoboken across the water has volunteer EMS. It’s, like, one stop on the PATH. (Volunteering in Seattle or near San Francisco is more difficult).

   I didn’t want to work a part-time position — then it would just be another job, and I like my current job. But volunteer jobs are straightforward enough to find, especially if you have a car.

9. The certificate is just gonna expire if I don’t use it.

   The culture barrier was strong enough that, for a time, I was convinced never to join an ambulance company. If you’re not on an ambulance, it’s very hard to fulfill the NREMT’s continuing education requirements, so your certificate will quickly expire.

   But, again, it’s OK to learn stuff without planning to use it anywhere. And isn’t continuous learning part of the point?

10. I wanna do search and rescue instead.

    Again, trying to avoid ambulances, I found search and rescue. Search and rescue (SAR) is badass. A snippet from one SAR orientation packet:

    If you have registered and do not show up at training that weekend, we will make every effort to locate you, and we are incredible at finding people.

    But in my heart I always wanted to treat people; and when I met SAR folks, it was clear that I wouldn’t be an effective EMT without experience on a truck.

11. I should do easier, cheaper prep first.

    I did 😃. I read books and Reddit posts. I took CPR and Stop the Bleed classes. I took a weekend Wilderness First Aid class with NOLS (strongly recommend!).

    But I still craved more. At a certain point — as I mention in How this site works—just do the real thing.

12. It’s expensive, and getting new credentials is an expensive hobby.

    Indeed! My course cost about $5,000, not including airfare, renting a car, & taking vacation for a month. You can do this for much cheaper (a typical course is around $2,000 and does not require 28 days vacation), but $2k is still quite expensive for some fun.

    But if you’ve done the cheaper stuff, if you’re still interested after so many years, if you’re excited about using your money for an experience like this… how could it possibly be a waste?

13. I have no real use for it.

    So I signed up with no real use for it! After years of delay, I started an EMT course with no plans to use it after. I was simply excited to learn.

    Working on an ambulance after was basically a happy accident, where I met a crew I happened to like. But it’s always fine to learn something for its own sake.

NOLS WEMT: I don’t expect to improvise a litter again (photo: NOLS)

Ridiculous

14. I don’t really wanna learn this anyway.

    For someone who “didn’t want to do this,” I sure spent a lot of time on Reddit reading about what the job would entail. I sure signed up for a lot of First Aid classes. I sure told a lot of people I was interested.

    I will always be a software engineer: it’s just who I am. And I learned during training that full-time EMS was not for me, after watching a woman negotiate with insurance to see if they’d cover the helicopter that would save her husband’s life.

    But not interested in learning this at all? Not being able to help people; to be a source of calm when things are actually on fire? Nonsense.

15. I won’t like the work.

    EMTs do menial work. They load & unload stretchers. They fold blankets, restock, and clean ambulances. EMTs are basically the least-trained medical personnel in any emergency (ignoring the odd EMR). They cannot diagnose; they follow fixed protocols, and they defer to better-trained personnel for anything more severe than a broken arm.

    Descriptions of EMT work inadvertently make the job seem directly heroic or glamorous: waking up at 3AM to rush a heart attack patient to the ER. It is not. The bulk of the job is waiting around, doing paperwork, cleaning, and backing up narrow driveways. It is very frequently boring & aimless.

    But that tedium is because you are waiting & ready to help. So it is indirectly glamorous. As an EMT, I get to put on a uniform, and, while we’re at Dunkin Donuts for coffee, wave at the kids staring at the shiny ambulance.

    Who wouldn’t like that work?

16. The class won’t make me a good EMT.

    I worried that an EMT certificate would be basically useless: that it would not teach me everything I expected, or that I’d still need more training on the job.

    Both things are true. I learned a lot, but upon stepping onto an ambulance for the first time, it was clear how much more there was to know. Every other EMT knows so much more than I do.

    But that’s the most magic part of being an EMT. It opens up an entire universe of new things to learn. It is the minimum qualification you need to enter this world: to work with firefighters and paramedics and doctors and social workers and search & rescue and disaster response personnel — and to see what you want to learn next.

17. EMTs only know ambulance stuff.

    A similar complaint I see on Reddit is that an EMT course only really teaches how to run an ambulance. That it is not useful for anyone interested in self-reliance or general survival skills because it requires supplies & logistics only available in a full EMS system.

    Not only is that demonstrably false (my Wilderness EMT certificate is specifically for scenarios where supplies are limited and evacuation is difficult), it’s also unhelpful. What alternatives are there? Become a doctor? Join the National Guard?

    Again, EMT is the fastest, easiest way to become a first responder. A good EMT course teaches you to be calm & useful in terrifying situations. A good EMT job drills that in. When that’s not enough, you’ll be ready for deeper qualifications.

    That was some of the most unhelpful advice I found before getting my EMT. Ignore it.

    

    NSFW Fake blood

    NOLS WEMT: none of my training was on an ambulance (photo: NOLS)

18. I can learn a lot about survival on my own.

    That’s the thing — you cannot learn this alone. EMS services have spent decades learning how to help people effectively, and most of that knowledge is not yet available on YouTube.

    This makes the worst services fratty, full of insider knowledge that they refuse to share. But the best services become places to learn, to become the next part of a chain that goes back to the first public Freedom House ambulances in the 60’s.

    There is quite a lot of medicine you can do on your own in a bunker. But there is quite a lot more that you can’t. You do that medicine the way we always have — as part of a team, working with hundreds of other people.

19. I will want to become a paramedic and quit my job.

    This is my favorite excuse: that I would like the work so much it would lead me to quit my tech job, pursue 2 more years of school, and become a full-time paramedic. In other words…

20. I might enjoy it.

    It got to the point where I avoided EMT school because I might enjoy the work. That it would hoover up my free time and energy, pull me from my other hobbies, or even my career. I worried I would toss everything for the thrill of riding in a truck with sirens on.

    That happens to some folks. It has not happened to me. I like the EMS work I do, but I also adore my tech job. This will not surprise anyone who knows me, and it shouldn’t have surprised me. And, actually, how awful would it be to find something so exciting that it drags me into a new career? It should have been obvious to me that this, of all things, was an excuse.

    An excuse not to do something scary and new.

A group of NOLS WEMT classmates posing after a moulage exercise, covered in fake blood and gore

NSFW Fake blood & gore

NOLS WEMT: scary, new, and fun… plus (fake!) blood (photo: NOLS)

Because it is scary

This is the silliest, truest, biggest reason I delayed so long:

21. It is scary.

    It is scary to practice medicine. It is scary to learn new things. It is scary to be responsible for being calm on the worst day of someone’s life; and for me it was, oddly, scarier to join a class full of strangers — strangers who would be much younger & on very different lifepaths.

    I come from the world of software engineers and office buildings. Of nine-to-fives and salad bowls. EMS is a very different world. Stepping into that world, without being sure it was where I wanted to be, was terrifying.

    It took a lot of time to reckon with that.

Because, ultimately, I am good at inventing reasons not to do things.

I love researching stuff before I do it. It’s fun to compare options, to see what other people think, to try to buy the best digital piano or visit the coolest museums in Rome. But I can get stuck. I perseverate — I revisit simple decisions over and over.

There are many legitimate reasons not to become an EMT, just like there are many legitimate reasons not to go on that vacation, or watch that scary movie, or go to that board game meetup. You do not have to do any of those things — you do not have to want to do any of them. I have no desire to see horror films (to my wife’s chagrin). But I kept wanting to be an EMT, despite all my reasons not to.

Perhaps you feel the same.

If you are delaying doing something scary and new, too, think: are you inventing reasons not to do it? Maybe you should do it anyway.

You might enjoy it.

Porchlights illuminate a few cabins against the fading sunset

NOLS WEMT: returning to the dorms at twilight

---

Thanks to NOLS and the NOLS WEMT students for making the course so awesome. Thanks to NHVA for welcoming me. Thanks to Javier Valencia, Jaime Romo, & Dan Walker for the NOLS photos. Used with permission. Thanks to Atherai Maran for editing.

---

Fired at 17 for leaking secrets. Served a cybercrime ring. Named by the Social Security whistleblower. Now the government’s model employee.

On Monday, September 14, Scott Kupor, the director of the Office of Personnel Management, went on Benny Johnson’s show to talk about hiring. Kupor sets personnel policy for the entire federal workforce.

Asked what kind of people he wants, he said: “I wanna see people like ‘Big Balls,’ or Ed, as we call him colloquially. We need people like that.”

He went on: “Ed is a great example. He dropped out of college. He never had substantive work experience before coming to DOGE.”

So the head of federal hiring named, as his model employee, a 20 year old who was fired from his only real job at 17 for leaking company secrets, ran a web service for a cybercrime group, and is named in a whistleblower disclosure over a live copy of every Social Security record in the country.

Edward Coristine was born in December 2005 and grew up in New Canaan, Connecticut. He graduated from high school in Rye, New York, in 2024 and was enrolled at Northeastern University as a mechanical engineering and physics major when he dropped out to join Elon Musk’s Department of Government Efficiency.

His father is Charles Coristine, who bought the snack company LesserEvil in 2011 and runs it as chief executive. His maternal grandfather was Valery Martynov, a KGB lieutenant colonel who spied for the FBI and CIA in the 1980s and was executed in Moscow in 1987 after Aldrich Ames and Robert Hanssen betrayed him. Coristine has said on Fox News that his grandfather “died so that I could come here and live in this free country that I love so much.”

Before government, Coristine built a trail of ventures and controversies that Bloomberg, Wired, Reuters, CNN, and Brian Krebs each documented.

He owns Tesla.Sexy LLC, launched in 2021 when he was about 16, which has managed image-hosting domains, at least two of them registered in Russia.

In 2022, as a 17 year old intern at the cybersecurity firm Path Network, he was fired. A Path executive wrote in an internal message reviewed by Bloomberg: “Edward has been terminated for leaking internal information to the competitors. This is unacceptable and there is zero tolerance for this.” He later boasted on Discord that he’d kept access to Path’s systems for months after termination.

His content delivery network, DiamondCDN, provided services to a cybercrime group called EGodly, which claimed on Telegram to have stolen cryptocurrency, hijacked phone numbers, broken into law enforcement email accounts, and harassed an FBI agent. On February 15, 2023, EGodly posted its thanks: “We extend our gratitude to our valued partners DiamondCDN for generously providing us with their amazing DDoS protection and caching systems, which allow us to securely host and safeguard our website.” Reuters traced the EGodly site to internet addresses registered to DiamondCDN and other Coristine companies from October 2022 to June 2023.

Krebs reported that Coristine was active in “The Com,” a network of Discord and Telegram channels where cybercriminals recruit and collaborate, under the handles “Rivage” and “JoeyCrafter.” Caleb Ecarma at Muskwatch reported, from web traffic records, that an image-sharing site Coristine ran was reachable through redirect addresses referencing child sexual abuse material and the Ku Klux Klan.

Coristine has not answered any outlet’s request for comment on any of it.

In 2025 he became one of the most recognizable people in DOGE. He worked out of the Office of Personnel Management and sat in on calls where federal technology workers were made to justify their jobs. Fox 5 DC reported he was one of two DOGE operatives given wide-ranging access to Small Business Administration systems in February 2025, which reached the National Finance Center, the system that processes payroll for agencies including the Department of Justice and the Department of Homeland Security.

Reuters found him listed as a senior adviser at the State Department and at the Cybersecurity and Infrastructure Security Agency, where Nextgov confirmed he had a Homeland Security email address and physical access to the building. On May 31, 2025, the General Services Administration hired him full time at GS-15, the highest grade in the civil service.

He was 19.

He resigned from the General Services Administration in June 2025, and within days the Social Security Administration confirmed it had hired him as a special government employee to work on the functionality of the Social Security website.

The Social Security Administration keeps a master file called NUMIDENT, the record of everyone who has ever held a Social Security number, living or dead, with the name, birth date, parents’ names, and citizenship of each holder.

On August 26, 2025, the agency’s chief data officer, Charles Borges, filed a whistleblower disclosure with the Office of Special Counsel and Congress through the Government Accountability Project. Borges alleged that DOGE personnel had copied NUMIDENT into a cloud environment they controlled, outside the agency’s security monitoring, covering more than 300 million Americans.

The disclosure names four DOGE personnel. Coristine is listed first, and a footnote describes his “lengthy history of facilitating, soliciting, or possibly participating in cybercrime.” Borges also names Coristine among the officials he asked for information and who did not respond.

The copy itself went like this. John Solly, another of the four, called a career agency employee on June 10, 2025 to open discussions about copying NUMIDENT, four days after the Supreme Court restored DOGE’s access to Social Security data, and Michael Russo approved it. The agency’s own security review rated the project high risk with potential catastrophic impact. Aram Moghaddassi signed the provisional authorization to operate on July 15, 2025, writing: “I have determined the business need is higher than the security risk associated with this implementation and I accept all risks.”

Coristine, according to the records in the disclosure, was involved in the project and was to be given access to the server.

On January 16, 2026, the Department of Justice filed a Notice of Corrections in the Social Security case in federal court in Maryland. The filing admitted that sworn statements the government had given the court about DOGE’s access to Social Security systems were materially inaccurate, that DOGE staff had routed data through an unapproved third-party server, and that a DOGE member had signed a data agreement with a political group.

NPR reported from the filing that on the morning of March 24, 2025, while Judge Ellen Hollander’s restraining order was in effect, a DOGE staffer was searching the master database at the same time agency officials were telling the court that all access had ended. The filing does not name the DOGE members.

On March 9, 2026, House Oversight Democrats sent Coristine a letter demanding a transcribed interview. The March 2026 thumb drive story, in which a former DOGE engineer allegedly claimed “God-level access” and walked out with Social Security records, is about Solly; Wired identified him, and the Social Security inspector general confirmed an open review.

Bloomberg reported the Path firing on February 7, 2025, and Musk posted that week: “Personally, I voted for Big Balls.”

On August 3, 2025, Coristine and a companion were attacked in an attempted carjacking near Logan Circle in Washington, D.C. He suffered a broken nose and a concussion, and two 15 year olds were arrested. Marjorie Taylor Greene posted that he “is also a HERO for sacrificing his own safety to defend a helpless woman!”

Eight days later the President signed the memorandum mobilizing the District’s National Guard, and it cites “the Administration staffer who was mercilessly beaten by a violent mob days ago.” A separate executive order the same day placed the District’s police under federal control. The two teenagers were sentenced in October 2025 to 12 months and 9 months of probation.

As of this month, Coristine works for an administration initiative to improve the functionality of agency websites, through the White House National Design Studio, the office Airbnb co-founder Joe Gebbia leads.

Five crimes fit the facts above.

Intentionally getting into a government computer beyond your authorization and taking information out of it is a federal crime under 18 U.S.C. § 1030. Oklahoma’s Computer Crimes Act says the same thing in state law: it is a felony to willfully and without authorization gain access to data and copy, disclose, make use of, or take possession of it, and a separate felony to exceed the limits of your authorization and do the same. Every state that allows citizen grand jury petitions has a statute like it, and each county kit cites its own.

The Privacy Act has its own criminal provision. An officer or employee who willfully discloses protected records to anyone not entitled to receive them commits a crime under 5 U.S.C. § 552a, and the master file is a Privacy Act system of records.

The March 24 search is contempt of court. A database a federal judge has ordered you out of is a database you don’t search, and 18 U.S.C. § 401 lets the court punish disobedience of its orders by fine or imprisonment. The order was in effect that morning, and Borges’s disclosure separately reports that DOGE violated the March 20, 2025 court order barring continued access to the data.

Then there are the false sworn statements. Whoever supplied false facts about DOGE’s access to the officials who swore to them in front of the judge committed obstruction of justice under 18 U.S.C. § 1503, which reaches anyone who corruptly endeavors to obstruct the due administration of justice. The government has admitted the statements were false in material respects and hasn’t said who supplied the information.

Conspiracy covers everyone who took part in planning the copy, in federal law and in every state’s, once one of them acted on the plan. Solly’s call on June 10 is that act, and the disclosure’s own records place Coristine in the project.

Conspiracy is the charge the record attaches to his name.

If it was carelessness rather than intent, the Privacy Act’s civil damages standard, “intentional or willful,” has been read by the D.C. Circuit since Albright v. United States in 1984 to cover conduct done without grounds for believing it to be lawful, or in flagrant disregard of others’ rights under the Act, and since Laningham v. United States Navy in 1987 to cover conduct so patently egregious that anyone undertaking it should have known it was unlawful. Carelessness at that level is enough.

And the record makes the argument unnecessary: the copy was requested on June 10, 2025 and approved, then given a provisional authorization to operate on July 15, 2025, after the agency’s own security review called the harm catastrophic.

That review rated the project “catastrophic impact.” Borges called it “a substantial and specific threat to public health and safety.” The master file is the identity record of every American who has ever held a Social Security number.

That is a national security harm, done by choice, in a series of dated steps, by named people.

A state can prosecute this.

The crime’s result, the copying of the record of every person in Tulsa County who has a Social Security number, occurred in Tulsa County, and state criminal law reaches a crime committed partly inside the state, which includes a crime whose result occurs there. Kansas, Nebraska, and New Mexico expressly place identity theft in the county where the victim lives, regardless of where the perpetrator sat. The Supreme Court held in Gamble v. United States in 2019 that a state and the federal government are separate sovereigns, so a state may prosecute the same conduct under its own law whether or not the federal government ever charges anyone.

Federal employees prosecuted in state court claim immunity under the Supremacy Clause. The Supreme Court set the rule in In re Neagle in 1890: a federal officer is immune from state prosecution for an act that federal law authorized him to perform, and only if he did no more than was necessary and proper to perform it. The defendant removes the state case to federal court under 28 U.S.C. § 1442, the federal officer removal statute, and a federal judge decides whether the act was authorized and necessary, with the Justice Department defending him. If the answer is no, the state prosecution continues in federal court.

The government’s best fact is the Supreme Court’s stay of June 6, 2025, which restored DOGE’s access to Social Security data four days before the copy request. Access to the data was never authorization to copy the master file into an environment the agency itself had rated catastrophic, and no court order authorized a search of that file on the morning of March 24, the morning the government told the judge the searching had stopped.

Six states allow citizens to petition a district court directly for a criminal grand jury: Kansas, Nebraska, Nevada, New Mexico, North Dakota, and Oklahoma. The petition goes to the district court. The judge’s only two questions are whether there are enough valid signatures and whether the conduct, if true, is a crime under state law. No district attorney or attorney general has any part in it.

Petitioners in Tulsa County, Oklahoma, Nelson County, North Dakota, Douglas and Lancaster Counties, Nebraska, and Shawnee County, Kansas, are now set up with template petitioner kits made by The Existentialist Republic to facilitate activists pursuing justice for their fellow Social Security having neighbors. This particular campaign is for signatures on petitions built on the government’s own January 2026 admissions about the Social Security data.

The templates, signature sheets, and reference files for each county are free to download here, and anyone in the six states who wants to petition in their own county can sign up for a chat with our petition campaign experts and Existentialist Republic activist guides to share the publicly available information on the subject in digestible and helpful ways through the project’s sign-up form.

I wrote this because the government’s head of hiring described the person above as the model, on the record, this month, and the model is the subject of a whistleblower disclosure, a congressional demand letter, and grand jury petitions in five counties and counting.

We see you, Big Balls. And we want justice.