When you talk about Papua New Guinea, people may say, oh, is that the place where:

• People are 4.8% Denisovan
• There are nearly distinct 1000 languages spoken, about 12% of all the languages on earth, and by far the most language diversity in the world
• Nobody is sure if its population is 10 million or 17 million
• They were having bow and arrow battles as late as 1963:

Historic Vids@historyinmemes

Pre-bronze age war between two tribes in West Papua, 1963

12:03 PM · Jan 19, 2024 · 18.5M Views

1.68K Replies · 7.26K Reposts · 66K Likes

• People still shoot arrows at each other in urban centres
• The capital is the most dangerous city in the world
• They practice cannibalism, and Michael Rockefeller was likely eaten there, and maybe Joe Biden’s uncle too - and one group ate their dead relatives’ brains at funerals so consistently that they gave themselves a mad cow type disease called kuru, which killed a large percentage of their population and then, within a few generations, evolved a genetic mutation that protects against it, one of the fastest cases of natural selection ever documented in humans
• Jared Diamond thinks the people there are genetically smarter than Europeans

But beyond these interesting tidbits, most people don’t actually have a conceptual understanding of what New Guinea is. Up until a few weeks ago, I had no idea either, until I randomly stumbled upon one of the most mind blowing books I’ve read in my life: First Contact: New Guinea’s Highlanders Encounter the Outside World, which describes the 1930 discovery of a million people living in the highlands of New Guinea—a group the colonial governments and local coastal populations had no idea existed, and who themselves had no knowledge of the outside world.

In 1930, the island of New Guinea was nominally colonized: the western half by the Dutch (now, Indonesia), and the eastern half (what is today Papua New Guinea) by Australia. Everything I say about the highlands applies to both halves. Despite this occupation, there was very little foreign governance or control in New Guinea.

In terms of geography, New Guinea is divided into two regions, the coastal region and the highlands. The coastal region mostly consists of malaria infested rivers, swamps, rainforest and jungle. The highlands, despite being just by the equator, have glaciers and snow, with peaks reaching just below 5,000 metres. At the time, the highlands of New Guinea had never once been explored, because from the outside, after you trek through the malaria infested jungles to get there, they look like massive impenetrable mountains permanently covered in cloud, where nothing could possibly live.

So in 1930, New Guinea was just thought of as the coastal region. It was believed no people lived in the mountains of what is now known as the New Guinea highlands.

This all changed shortly after Australians found gold in New Guinea in 1926 and started mining the surrounding territory. Looking for more gold, an intrepid Australian named Mick Leahy decided to venture as far inland as he could — making progress up the never before penetrated mountains.

To cut the story very short, in 1930 Leahy accidentally stumbled into the eastern edge of the highlands, only to find that the highlands were not empty mountains, but rather lush green valleys with no malaria and with roughly a million people living in them.

The book is not really a story, but a documentation of this exploration and first contact. Leahy brought with him high quality cameras and a movie camera, and documented with thousands of photos and hours of film the first encounter of hundreds of thousands of people. The book is filled with the most incredible photos of all the scenes surrounding this first encounter. Beyond the photos, there are details of Leahy’s next ten years exploring the highlands and accompanied by 1970s interviews with New Guineans who were living in the highlands at the point of first contact and offering their perspective.

This includes descriptions of how the highlanders thought the white men were ghosts of their dead relatives at first - and then, to investigate this, spied on them defecating and smelled the poop, only to realize that they must also be human like them. How the Australians would do demonstrations of “strength” and gather whole communities to watch them shoot pigs, or make New Guineans listen to records, play with child dolls, look at themselves in mirrors, watch airplanes land etc. mostly thinking everything was a holy spirit. How highlanders would take pieces of the Australians’ garbage like a Kellogg’s cereal wrapper, and wear them around their heads as prized jewellery. And to really show the highlanders the power of their world, they would take kids on their airplanes to the coast, make them see the sea and civilization and come back, so they could report to their relatives how powerful the white men were.

The most powerful story described in the book is the one of hyperinflation. The highlanders were obsessed with shells, then an incredibly rare commodity, the jewel that the leaders and most powerful men would wear to demonstrate their power, and the closest thing they had to a currency. But knowing this, the Australians brought shells in by the planeload, to use in exchange first for food and then also for labour. At first this felt like a great deal for the highlanders, who were willing to give up nearly everything for more shells. But the planes kept coming, and the price of everything in shells kept rising, until it went from one shell making someone a leader and “rich”, to almost all people, including children, being draped in hundreds of shells, and the shell eventually being worthless.

There’s also a documentary, First Contact built from Leahy’s original footage and those interviews, so you can watch the scenes I’ve just described.

There is a lot more captured here in the book, but just imagine, 1,000,000 people separated from the rest of civilization for nearly all of human history, suddenly meeting a group of Australians who captured the entire exchange on photo and film.

To clarify how crazy this first encounter story is.

In 1930 (pretty frickin recently), it was thought that there was nothing in the middle of New Guinea except rock. Australian patrol officers had even walked from south to north across the whole island yet somehow entirely missed this. And then all of a sudden, some random Aussie stumbles upon the highlands and discovers a million people living there. There is no comparable first encounter story — and all of this documented in hundreds of photos and video!

To take a step back and understand how we got here: people first arrived in New Guinea around 50,000 years ago. They got there by walking and canoeing from Southeast Asia, island hopping through what is today Indonesia. These are the same people who would go on to walk south and become the Australian Aboriginal people, at the time when New Guinea and Australia were connected by land.

Various people settled on the New Guinea coast but eventually, people found their way to the highlands. Around 10,000 years ago, as agriculture took hold there and the valleys filled with people, the highlanders became functionally cut off from the coast, and stayed that way until 1930.

What fascinates me is the level of insularity of the group. Nearly all people in history existed with some integration with their neighbours and knowledge of the broader world. It’s shocking to learn of a group that lived on their own for 50,000 years, and sealed off from everyone else for the last 10,000, nearly completely divergent from the rest of humanity.

This insularity was possible because New Guinea is one of the very few places on Earth that independently invented agriculture. Around 10,000 years ago, at the same time people in the Middle East were figuring out wheat, New Guineans were draining swamps and learning to cultivate taro and bananas. Bananas and sugarcane were both first domesticated in New Guinea. To understand the impact New Guinea has on your life - the main banana variety we eat partially derives from there, as does nearly all the sugar consumed in the world.

The next thing that’s fascinating about the highlands is that different tribes never centralized and agglomerated into a larger society. It seems like this is due to a few structural reasons. In the highlands, despite the invention of agriculture, crops would go bad quickly - root crops like taro rot within weeks of being pulled out of the ground, unlike grain, which you can pile in a shed for years - so there was no produce to store, trade en masse or tax, or to take on broader projects. There were no animals to work the fields or carry anything. There was no writing to document anything. And while there was a constant supply of food, there was never enough of it, especially protein. So while each of these things seems small, together they meant that the highlands never evolved to the next stage of development and formed larger centralized communities, where cultural innovation and less warfare could have led to greater progress.

This left groups typically of a few hundred people living side by side, in constant warfare. This made travel and trade very difficult, as most highlanders were unable to travel through adjacent neighbouring zones without fear of death. It also meant that every group was focused on the same two things, feeding and defending itself, and not much else. No group ever had the motivation or capabilities to easily expand or explore past the next valley. Note, there was a broader highlands trade for salt and stone axe blades, which did make its way across multiple different adjacent groups.

Highlanders were farming thousands of years before the Egyptians. And four and a half thousand years after the pyramids went up, highlanders were still farming with the same limited technology. Without meaningful integration with the rest of the world, and with the structural factors above stopping them from ever centralizing, they had no way to evolve past this stage of development.

Despite the small communities and constant warfare, there was a surprising degree of homogeneity within the highlands. It seems that this is because things that worked travelled even when people didn’t - a better crop, a better way of building a house, a new ritual, would get copied by the neighbours they fought with and often married into, and then copied again by their neighbours. So nearly all groups eventually internalized a similar set of norms, a similar food culture and similar political institutions, without ever being part of the same political unit.

Most groups were governed by what’s referred to as a big man leader. This was fuelled by what is known as moka, a system of competitive gift giving, where you give someone in your tribe goods like a pile of pigs and shells, and they are then obligated to give you back more later, and you them more again after that. Whoever gives the most, publicly, becomes the big man. The noted problem with the big man leader is that this had to be earned in each leader’s lifetime and was not hereditary, meaning that leadership could not grow, could not think long term and was never secure - so not only was there constant external fighting, there was also constant internal fighting. Notably, in the parts of the coast settled by the Austronesians (a seafaring people who arrived from Southeast Asia about 3,000 years ago), where chiefs were hereditary, you do get different political structures.

And this leads to the most interesting part of this story: how the highlanders were so isolated for so long.

I want to acknowledge that the highlanders were not completely, hermetically sealed off from the outside world - it’s just that knowledge of outside world never actually made it to them or vice versa.

Over time, a few goods did make their way up to the highlands: pigs, about 3,000 years ago; the sweet potato, which is South American and arrived about 300 to 400 years ago; and tobacco, also American, which came the same way.

The journey the sweet potato and tobacco took to get there is a good example of how this all worked without transmitting further information. The Spanish and Portuguese brought both plants to the spice islands (in what is now Indonesia), just off the western tip of New Guinea, in the 1500s. The sultanates had been trading with the western tip of the coastal region of New Guinea for centuries. So the plants crossed over and started moving east along the coast and up into the interior. Beyond that point, there were no merchants. But when a woman married into the next clan over, she took cuttings from her family’s garden with her. Each new family then planted the same, saw that it worked, and passed it on. At that pace the sweet potato crossed the highlands in a century or two, with nobody knowing where it came from beyond the tribe beside them.

And what fascinates me is that even with the highlands as sealed off as they were, such significant goods still got in. Sweet potatoes allowed a population explosion in the highlands: they give far more calories per acre, and pigs can live off them. So in a 10,000 year sealed off universe, after the invention of agriculture, the biggest change to highland life happened in the last 500 years, which I guess isn’t a coincidence, as the sweet potato got there through the same structural forces that later brought the Australians.

Aside from these one time crop exchanges, there was a permanent flow of shells from the coastal region to the highlands, although in limited quantities. Marine shells show up in highland caves from around 10,000 years ago, so this trickle had been running for as long as there had been agriculture up there. While the highlanders did not know where the shells came from (some groups thought they came from the sky), they viewed them as the prize possession in their communities, their main fashion accessory and the basis of their currencies.

The ignorance that came with all this trade ran in both directions. For 200 years, Europeans believed that birds of paradise (birds from the New Guinea coastal region) had no feet, because when the dead bird skins left New Guinea the traders cut the feet off. Their scientific name is still Paradisaea apoda, which means footless. The shells went up the chain and lost their origin. The birds went to Europe and lost their feet.

The reason goods travelled but knowledge didn’t, I think, is that plants and animals had an easier time reproducing from one group to the next, making it easier to spread. Ideas require explanations and communication, and every handoff along that chain was between two groups who didn’t share a language and expected to attack each other.

So my understanding of how this came to be is that highlanders couldn’t really mosey around. If they approached the edge of the mountains, which was only really possible for the few small groups living in that area, and tried to go further beyond the edge of the valleys, they would face no food (their food rotted quickly) and no people, or worse, people who would treat any stranger as an enemy. The country in between isn’t a line where the highlands stop and the coast starts. It’s steep, wet, too rough to farm and low enough to have malaria, thinly populated by people living off sago and hunting who were neither quite highlanders nor coastal people. Getting from the last highland village to the first coastal one meant passing through several of these groups, each a few days’ walk from the next, none of them with any reason to go further than their own neighbours. And even if the highlanders could travel farther, they had never been exposed to malaria and wouldn’t have survived easily (when the Australians later sent highlanders to work on coastal plantations in the 1950s, they died at alarming rates).

The only place on the whole island where the two worlds nearly collide is the Markham valley near Lae, where a 1,000 metre climb over about 30 km takes you straight up onto the edge of the eastern highlands. This is where Leahy walked and first discovered the highlanders.

The reason why I believe that, despite the minimal trade that existed, the highlands were functionally cut off, is that when the Europeans occupied New Guinea, they themselves did not ever speak of or show knowledge of the highlanders. Similarly, when the highlanders first encountered Australians, they had no concept of the coastal people, or where the shells came from, or any further concept of the broader world.

The second source of evidence for this is genetic. The coastal New Guinea people have one genetic set of traits, which has been admixed with the Austronesians, a seafaring people who came from Taiwan via the Philippines to New Guinea about 3,300 years ago (and who went on to become the Polynesians). All along the coast and the islands, there is 10 to 30% Austronesian ancestry, and Austronesian languages. But when sampled, highlanders have no Austronesian admixture at all. Given how common it was for both highlanders and coastal people to offer brides to neighbouring groups for diplomatic and trade reasons, there was a lot of intergroup offspring, yet in 3,000 years, no highlander offspring mixing with the coastal people. Similarly, highlanders have no genetic adaptation to malaria, unlike the coastal people, who carry some of the densest concentrations in the world of the blood mutations that protect against it. This indicates that the highlanders never had the forcing function to develop them, and never interbred with those who did.

Lastly, there are no signs of Austronesian language, or of any of the unrelated coastal language families, making its way into the highlands.

In terms of languages, the diversity is actually almost all with the coastal people. In the highlands, there is great diversity, but everyone speaks a language that is at least from the same language family (called Trans-New Guinea — note: this is not just dialects of each other like Spanish and Portuguese are “different languages”, but with the range and diversity of English to Russian to Hindi - at the time, all without writing or an alphabet). The dozens of language families that are completely unrelated to each other, which is where the real diversity is, are almost all crammed into the north coast of New Guinea. The reason for the complete domination of this one language family, unlike the coastal region of New Guinea, is that when agriculture took off in the highlands 6,000 to 10,000 years ago, the farmers spread along the highland valleys and their language spread with them, wiping out whatever was spoken there before, the same way Indo-European wiped out almost every older language in Europe (except Basque, Finnish/Hungarian). The coastal regions never had one group or language spread like that, so 50,000 years of languages drifting apart just piled up, with the Austronesian languages layered on top 3,000 years ago.

In terms of Papua New Guinea today, it, more than any other state I’ve read about, seems to be a state living its pre-contact life. While there is an obvious level of modernization and nearly everyone is using cell phones, the state, and by extension integration with the broader world, has only minimally impacted Papua New Guinea. The reason for this is that it’s just too difficult to connect people living in such terrain and conditions at their level of wealth.

When you look at a map of Papua New Guinea, you see that the capital city, Port Moresby, doesn’t connect with anywhere. There aren’t any roads that go from Port Moresby to any other major population centre or really, anywhere outside of the city. The country has only one proper medium distance road, the Highlands Highway, which goes from Lae (the one bridge between the coastal region and the highlands - where Mick Leahy had walked all those years before) and into the mountains, but even this is only around 700 km, connects a very limited number of places, and is in such bad condition that landslides and bandits regularly close it and trucks travel in convoys to avoid being robbed. Most of the country moves by propeller planes.

In terms of language, the lingua franca of Papua New Guinea has become a language called Tok Pisin. Tok Pisin is a creole of mostly English with bits of German, formed by Melanesian indentured labourers working on plantations in Queensland, Samoa and German New Guinea in the late 1800s, who all spoke different languages and needed to communicate. Since then it has slowly spread through the coastal regions and eventually the highlands of Papua New Guinea. But even with this, language barriers are still significant.

Which brings me back to the list at the top. Nobody knows Papua New Guinea’s population because counting people requires a government/state that has a presence in each region of the country, and Papua New Guinea lacks this.

Today, the feeling of tribalism permeates the whole country. This is why Port Moresby is so dangerous - rival clans have taken their conflict with them into the urban capital. Papua New Guinea has the concept of wantok, meaning “one talk” — the people who speak your language are the people you “care” about. While Tok Pisin gave everyone in Papua New Guinea a shared language, nothing ever gave it a shared identity. The historic clan is still the main social group and there is no new central identity above it. People are still John Smith of the such and such tribe, not John Smith of Papua New Guinea.

What I find so fascinating about all of this is just how separated this group was from the rest of history, and that there will never be a finding like this ever again. The world has effectively been explored, but as late as 1930 that wasn’t the case. There are people alive today, from a very large society, who were born into a world that had never seen metal. I also wouldn’t have expected the trade to work the way it did. Plants and animals from the outside world found a way in, but technology and ideas didn’t. The sweet potato made it 1,000 km inland but the concept of the ocean didn’t. The timing of the discovery was also quite fortuitous for the highlanders. They were exposed to the outside world at a point where the world had germ theory and vaccines, so while dysentery and flu did spread through to the highlands after first contact, there was nothing like the smallpox and measles that killed most of the native population of the Americas.

So this is why I can’t stop thinking about Papua New Guinea. A people that had a completely divergent path of history - a nation living more of its pre-contact life than any other - and a first contact story entirely captured on film. I highly recommend you read First Contact, by Bob Connolly and Robin Anderson, if only just to see the pictures.

Models have been superhuman at chat for years, so where is all the automation?

This has been my driving question for the last four years. At OpenAI, I helped build the methods that made language models useful at following instructions and talking with people. That work ended up as the research behind ChatGPT. At the time, I thought maybe chat models would lead to AGI, but despite the hype it became obvious to me that there was something really big missing.

After two years in stealth, countless technical challenges, and research breakthroughs… I am beyond excited to announce that today, TypeSafe AI is releasing our first System One Model: a new class of frontier models built to make fast, structured decisions that software can use directly.

We built a new stack entirely focused on automation: with a new model architecture, parallel sampler for maximum efficiency, and training method we call Reinforcement Learning for Calibrated Decisions (RLCD).

Our first public model is Jev, available today in early access. Jev achieves similar levels of intelligence on System One tasks compared to existing LLMs, while being two orders of magnitude faster and more efficient. While Jev gives up string generation, it’s optimized for structured outputs and can’t hallucinate.

Think of Jev as a frontier-intelligence function call: unstructured state in, typed probabilistic decisions out.

Extraordinary claims require extraordinary evidence so see below for the receipts. 💅

Frontiers, Old and New

	Existing LLMs	System One + Jev
Optimized with	Reinforcement Learning with Human Feedback (RLHF) / Reinforcement Learning with Verifiable Rewards (RLVR)	Reinforcement Learning for Calibrated Decisions (RLCD)
Optimizes for	Human preference: writeups and chat responses that human raters prefer.	Verifiable rewards: outputs that can be programmatically verified. Calibrated decisions: answers with epistemically honest probabilities on System One tasks.
Inputs	Unstructured data (e.g. text) with an emphasis on sequential messages.	Unstructured data (e.g. text) with an emphasis on structured program state.
Outputs	Strings / generated text. Strings are flexible and can be anything: chat responses, code, hallucinations, refusals, or even type-safe structured values. To be used by software, responses need to be parsed + validated. There is also always some risk that the AI goes off the rails.	Type-safe structured values. Possible outputs and structure are defined in advance. The model never makes type errors. All answers are accompanied with calibrated probabilities and confidence scores.
Sampling	Sequential. Generates one token at a time, each conditioned on the last.	Parallel. Generates all outputs in a single query. Incredibly efficient and hardware-aware.
Cost	Input tokens: from $0.20 to $10 / MTok. Output tokens: ~5x more expensive than input tokens.	Input tokens: $0.042 / MTok ($42 per billion tokens). Output tokens: FREE (too cheap to meter).
Speed	End-to-end response time is 3 to 329 seconds for frontier models. Fast enough for interfacing with humans, but a big bottleneck when integrated in code.	End-to-end response time is 70ms-500ms for TypeSafe. This can range from 40x-200x faster for the same levels of frontier intelligence for System One shaped queries.
Confidence	Even if prompted for a confidence estimate, models tend to be overconfident and inconsistent. If a model can do a task 95% of the time but doesn’t say when it’s in the 5%, it can’t automate that task.	Always communicates confidence and uncertainty with every output. Calibrated: higher confidence means higher accuracy. More consistent: returns similar answers for similar inputs.
Use cases	Human-in-the-loop tasks (chatbots, copilots, coding agents). General and powerful, but requires human oversight because their freedom also means they might go off the rails. Verifiable problems (math proofs, kernel optimization). When correctness can be checked cheaply and automatically, LLMs can generate, test, and iterate until they find something that works. Demos. The flexibility of strings allows it to be incredible for quickly making prototypes that only work sometimes.	AI-Powered Workflows / smart if-statements. Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle. The surrounding code constrains their freedom, making them easier to compose into reliable systems. Map-reducing over big data. Turn petabytes of data into features and insights. Real-time applications. 100ms speeds means you can use AI in your applications where UX is critical. Verify everything. Score, judge, verify, guardrail, and detect jailbreaks of LLM prompts, reasoning traces, and/or outputs.

Evidence / Technical Results

We love skeptics, and are skeptics ourselves.

There are some claims you can easily verify:

• Speed per call: We truly are that fast, though our published evals are generally run from our laptops on the West Coast (this is where our service is currently based).
• Cost per call: We make our pricing transparent. We can’t prove it isn’t subsidized; we’ll need the long-term to prove the sustainability of our pricing (which we expect to go down, not up).
• No type errors: This would be an easy thing to falsify with just a single counter-example, but it is mathematically impossible.

For our bolder claims, we want to provide as much nuance as we can.

Side-by-side demonstration

Our side-by-side demo shows a key difference between our models and LLMs: Jev outputs all probabilities in parallel instead of autoregressively generating by token. Strings are extremely powerful and general, but costly. “Giving up” strings actually gives us a lot of superpowers!

Nuance

• For people with early access to TypeSafe, here is the actual query.
  • The query is highly simplified and questions were chosen to have descriptive, human-readable keys so that the output on the screen is understandable.
  • The state is also a short, dense, and detailed paragraph, to emphasize the difference in sampling methodology. The relatively shorter input paints our model in an advantageous light.
• For the keen eyed, for the recorded run, the only disagreement with GPT-5.6 Terra is on “Churn likelihood level”. The actual answer seems genuinely ambiguous to us.
• We used GPT-5.6 Terra with default reasoning for this example, because we’ve found it to be the most comparable at intelligence to Jev on average.
• Fun fact: a similar demo was what convinced us to go all-in in the direction of System One Models!

Workflow evals

We made a new type of evaluation to measure how well AI works within code. We don’t optimize for a ground truth classification or allow the harness and model to change (potentially allowing for overfitting via harness engineering). Instead, we assume there is a correct compute graph (a “workflow” represented in code) and use the predictions of the largest, smartest, and most expensive external models as reference probabilities.

Rephrased: every model gets the same workflow. We test how they compare to the average of the smartest models (in this case, Astra and Fable).

Jev is off the charts – owning the Pareto frontier for almost 2 orders of magnitude. We also compare to models with a generated prompt doing all the logic in their chain-of-thought, but this tends to do significantly worse than using the workflow itself.

Note that the calls here are significantly more complex than the side-by-side demonstration above. That’s because they’re more representative of the types of production workloads needed for true business automation. Below is the simplest of the 4 workflows we’re publishing:

The most reliable real-world workflows tend to have many independent, decomposed questions, with fine-grained behavior that’s dependent on probabilities instead of discrete decisions. The end result is discrete branching, but how we get to a final answer involves a lot of domain-specific engineering that needs to be done highly consistently.

See our workflow evals site for all the details: examples, disagreements, full queries, and each workflow.

Nuance

• This is where the claims of 193.6x faster, 444.6x cheaper on our home page comes from, and we expect that these are on the higher end of real world gains.
• These content of these workflows were not deliberately chosen nor constructed to make our model look good, and are not in our training distribution. However, they were made by individuals on our model capabilities team, so some bias could exist.
• We use the average of GPT-6 Astra and Fable 5.1 as the reference answer, which biases answers towards OpenAI and Anthropic’s models. We likely underestimate the relative performance of our model and DeepSeek’s models.
• The LLMs use our System One LLM wrapper, which constrains LLMs to output structured decisions compatible with our API. We have found this to be the most accurate way to get decisions from LLMs, but this tends to be slower and more expensive than giving decisions without probabilities.

Hallucination and Type-safety

Hallucination and type-safety are intrinsically related, and we think the latter is table stakes for automation. Having a hallucinated tool call is inconvenient in an agent, but is an absolute deal-breaker if it’s part of a system with latency guarantees or it’s buried several layers deep in a dependency chain. Existing models, no matter how smart, still hallucinate and have type errors.

Nuance

• The numbers for LLMs are from OpenRouter i.e., there almost certainly is bias here: more complex queries might be routed to better models.
• Our number is not empirical. Schema matching is guaranteed, thus we can confidently add 0% into the plots.

Fun Demos

Perhaps the most exciting part of our work is enabling new use cases. We have a lot more to show you, but here are a couple of the team’s favorites:

Doom

We love how this doomo doomonstrates real-time intelligence and what can be doone with code + AI. The engineer behind it was worried about making 10 queries a second (which ends up costing ~$7/hour), but the rest of us agreed that was lower than expected! This is so fun we intend to not only release an in-depth walkthrough, but also host some events to hack on this.

Nuance

• The demo is on structured state as a data structure with text, not on images (yet…)
• A non-AI doom bot could play better, but we wanted a bot that was reactive to different representations of game state, and most importantly… following instructions was cool as heck!

Wikiracing

The objective of the game is to start on one Wikipedia page and reach a specific other Wikipedia page using only links you come across while traversing. Each step can mean choosing between hundreds to thousands of links! It’s a great playground for demonstrating not just intelligence-per-second, but also the compounding benefits of not hallucinating with high-cardinality choices.

Nuance

• As far as we know, it was completely random that both the 2nd and 3rd challenges started with “Rubber Duck.” The author only noticed when the team pointed it out.
• Our speedups here tend to be a lot less than in previous demos. That’s because this is against the non-reasoning modes of the models (except Astra which was set to the lowest reasoning setting). This is also why Jev tended to finish in fewer steps (a sign of greater intelligence). This was to make the demo more bearable to watch. The LLMs look much worse at this task than with reasoning enabled.
• Jev supports a cardinality up to 255. For the higher cardinality choices, we do a 2 stage-system of scoring independently then making an explicit choice, hence the occassional slowdown.

What’s next

We’re still in Jev’s early days. We have a lot more in the pipeline and are so excited to keep on shipping 🔥.

Today, we are opening early access and bringing developers off the waitlist as quickly as we can. We want to hear which decisions you need to automate, where Jev works, and where it falls short. Tell us what sci-fi you want to build!!

We started TypeSafe because we believe that AI needs an interface software could depend on. We can't wait to see new use cases continuously diffuse through the community and economy.

We Give A FAQ

Where do the names “System One Models” and “Jev” come from?

We were inspired by Daniel Kahneman, Thinking, Fast and Slow. The model class name draws on the distinction between fast, intuitive System 1 thinking and slow, deliberate System 2 reasoning.

“System 1 thinking” has also implied error-prone. For reasons we will get into in the future, we believe System One Models can be made more reliable than its alternatives.

We named Jev after William Stanley Jevons. We expect machine intelligence to follow a similar path to coal, after steam-engine efficiency led to an increase in demand. Every order of magnitude drop in the cost of intelligence unlocks orders of magnitude more use cases.

Why was a new training algorithm needed?

What use cases is Jev good for?

Is Jev just a smaller LLM?

How does Jev perform against public benchmarks?

Where does our training data come from?

These are results are kinda crazy - how is it possible?

25 Years of Mass Surveillance Is Enough

One of the many legacies of the terrorist attacks of Sept. 11 is the government-wide shift from targeted surveillance—such as individual wiretaps or pen register/trap and trace orders—to mass surveillance techniques—such as tapping into the internet backbone or mass collection of telephone or internet metadata. The legal and technical architecture of modern mass surveillance, initially framed as a necessary defense against terrorist threats, has grown far beyond that justification and national security in general. Mass surveillance is now a routine tool used by law enforcement. ICE uses it in immigration actions and against people exercising their First Amendment rights to protest. It’s also increasingly part of private security systems, such as facial recognition at venues such as Madison Square Garden and networked Flock license plate capture systems on roads and in parking lots.

The interrelation between private and governmental mass surveillance is worth examining. Surveillance is the business model of the internet; companies like Google and Facebook constantly spy on their users’ behavior. From the National Security Agency relying on data collected by telecommunication and internet companies, to local sheriffs and ICE agents relying on cellphone location data and privately managed automatic license plate readers, governments primarily obtain the mass surveillance information through private companies. Increasingly, access doesn’t just come through legal processes, either. FBI Director Kash Patel recently confirmed in congressional testimony that the agency is purchasing information on Americans from data brokers and intends to continue to do so.

This pipeline from private collection to governmental collection means that as companies collect more information for surveillance capitalism purposes, more is available to law enforcement as well. And as the technology for mass surveillance and analysis improves, especially with the increased use of AI technologies, the problems attendant to mass surveillance grow as well.

After 9/11, the idea that the government could surveil the population to safety took hold. In 2001, the fear of terrorism reached a frequency and intensity never before seen. Along with that came the fear that the enemy could be anyone, anywhere. As a result, the government’s response was to watch everyone, everywhere. This line of reasoning underpinned the shift from targeted to mass surveillance. Or, in the words of an internal National Security Agency (NSA) presentation that was made public as part of Edward Snowden’s 2013 disclosures, a government that can “Collect it All,” “Process it All,” “Exploit it All,” “Partner it All,” and “Sniff it All,” will ultimately, “Know it All.” Similar rationales support the rise of domestic mass surveillance: if law enforcement could see and hear everything, it could more effectively interdict and solve serious crimes.

The national security community has never provided a full analysis of the costs and benefits of these mass surveillance programs, either in terms of taxpayer dollars or diversion of resources from other efforts—or any demonstration that those techniques stopped attacks that otherwise they would not have been able to prevent. While the NSA occasionally presents examples of the successes due to its mass surveillance programs, especially when those techniques are under public pressure, the examples also regularly fall apart upon serious scrutiny. And even if some utility exists, it must be seriously weighed against the costs.

Similarly, there has never been any comprehensive analysis about whether domestic immigration or law enforcement’s use of these techniques actually makes people safer, or whether other techniques could produce the same results. Instead, both the police and the companies selling these tools float anecdotes and dubious data. For example, Flock’s data equates the number of law enforcement hits in their database with actually solving crimes.

Twenty-five years after 9/11, it seems reasonable to step back and evaluate the costs of this shift to mass surveillance, especially in terms of Americans’ rights and freedoms.

The Shift

The easiest place to see a shift to mass surveillance was in the government’s decision immediately after 9/11 to collect Americans’ telephone records. The program started under an argument of pure executive power as the “President’s Surveillance Program.” But in 2006, that argument secretly shifted to a novel interpretation of Section 215 of the Patriot. Act which had only previously authorized more targeted access to record. While some media and public interest organizations struggled to force the government to reveal the program as early as late 2005, the government only officially confirmed it after the 2013 Snowden disclosures. In 2015, the Second Circuit Court of Appeals rejected the government’s interpretation of Section 215 as allowing mass collection of telephone records. Later the same year, Congress passed the USA Freedom Act. While this new law still allows collection of a tremendous amount of domestic telephone records, it ended the indiscriminate mass collection that had occurred for nearly fourteen years.

Other shifts to mass surveillance continue through today. The NSA launched its Upstream program, which involved intercepting both metadata and content from key telecommunications junctures inside the U.S., soon after 9/11. It was also initially conducted under a claim of purely presidential authority. This program was brought under marginal congressional and programmatic (not targeted) Foreign Intelligence Surveillance Act (FISA) court review via Section 702 of the 2008 FISA Amendments Act. In 2017, more than15 years after its inception, the NSA ended content searches due to FISA court pressure, but the mass collection continues.

Despite the stated goal of conducting mass spying only on people outside the U.S.—which itself is problematic given international law’s requirement that surveillance be both necessary and proportionate—mass surveillance collects a tremendous amount of U.S. persons’ communications. This can happen because people communicate with people abroad, or because of overcollection—when government agencies gather far more personal data on non-targeted US persons than authorized by law. The concerns about collecting Americans’ data on U.S. soil led Congress to allow the program to officially expire in 2026, although the previously-approved mass surveillance itself continues until at least Spring of 2027.

The shift to mass surveillance would be notable enough even if it remained only a strategy of the intelligence community. It has not. Americans are awash in mass surveillance. Networks of automated license plate readers such as those offered by Flock and Vigilant Solutions blanket both public and private roadways and parking lots. These networks often allow searches by law enforcement, including across jurisdictions. They are, for example, being used to track people seeking abortions across state lines. Facial recognition tools, once the province of only the more elite parts of federal law enforcement, are increasingly used by Immigration and Customs Enforcement agents on immigrants and protesters, in airports by the Transportation Security Administration, as well as by private entities. And, of course, modern phones track users’ locations constantly—and that information is readily available to law enforcement, often with only minimal process protections.

Constitutional Costs

Regardless of the murkiness of its actual usefulness, the shift from targeted to mass surveillance has profound implications for Americans’rights. It has created risks that have become increasingly evident, especially under the Trump administration.

At a basic level, the Fourth Amendment guarantees that citizens can be secure in their “persons, houses, papers and effects” from unreasonable searches. Warrants breaching that security should be supported by probable cause and particular descriptions of the place to be searched and items to be seized. Mass surveillance turns that promise on its head, allowing access to our “papers and effects” by the government without individualized suspicion or a particularized description of what data is being seized, much less probable cause. This protection was in response to colonial British misuse of writs of assistance, which authorized indiscriminate searches rather than targeted ones.

The justifications for exempting mass surveillance from constitutional protection vary. For Section 702, the government has taken the position that U.S. persons’ communications caught up in the dragnet, either due to overcollection or because they were communicating with someone outside the United States, do not require a warrant prior to initial collection or secondary access by the FBI and several other agencies. The argument is that if the initial collection was not aimed at Americans, the information is free from constitutional protection for any later uses, even for reasons far afield from the initial rationale for collection.

Other arguments rest on the claim that metadata is outside the Fourth Amendment, despite its demonstrated ability to reveal intimate details of all of our lives. Still others rest on the Supreme Court-created Third Party Doctrine, which holds that the Fourth Amendment does not apply to data shared with companies that provide us with services. Some turn on whether analysis by machine counts, claiming that only “human eyes” matter—a particularly troubling argument with the rise of artificial intelligence. What’s more, the government has used doctrines like standing to limit the ability of those subjected to mass surveillance to seek constitutional protection. No matter the argument, the goal is the same: to place the mechanisms and fruits of mass surveillance outside the protections of the Fourth Amendment.

The overarching truth is that, due to the concerted efforts by the government since 9/11, and the rise of technologies in recent years, the slice of Americans’ lives and data that are actually protected by the Fourth Amendment has shrunk significantly in the past 25 years. Together, with the technical capabilities of mass surveillance and the increased ability for that data to be analyzed using AI tools, the “security in our papers and effects” that the constitution promises seems increasingly illusory.

In addition to the Fourth Amendment, mass surveillance creates tensions with the First Amendment. The Constitution has long recognized that the right to freedom of speech requires a zone of privacy against governmental surveillance. The right to anonymous speech as well as the right of association both recognize the chilling effect that surveillance creates for people saying unpopular things or attempting to organize for political or other societal change. Mass surveillance grants the authorities the ability to track those people, both in real time and historically, that is inconsistent with actual techniques of freedom of speech and assembly.

That is why the recently released 2026 U.S. Counterterrorism Strategy is so troubling. On page seven, the White House expressly states that it intends to target domestic activists with its heretofore foreign-targeted powers. It says that the government “will prioritize the rapid identification and neutralization of violent secular political groups whose ideology is anti-American, radically pro-transgender and anarchist” and “will use all the tools constitutionally available to us to map them at home, identify their membership, map their ties to international organizations like Antifa.” While framed as targeting “violent” groups, it’s clear that the government intends to use its national security tools, presumably including the tools of mass surveillance, against Americans in ways that will create profound tensions with the First Amendment rights of people to organize and communicate privately.

Costs Due to Mistakes and Abuse

Even assuming some utility from mass surveillance—a fact we do not dispute, even if the public record is shaky and conclusory—the history of both the national security and domestic uses of mass surveillance confirms that these tools are inevitably misused, and that mistakes have impacted huge numbers of Americans. The past twenty-five years have demonstrated that it is not possible to surveil the entire US population while staying within the bounds of even a very generous legal framework like Section 702.

As Rep. Zoe Lofgren (D-Calif.) recently stated in discussion of Section 702 in an interview with Tech Policy Press: “backdoor searches have been used improperly for protestors, 19,000 campaign donors, members of Congress, journalists, government officials, a state court judge who had complained to the FBI about police misconduct. It has been abused substantially in the past.” The NSA experienced so much abuse of its mass surveillance tools by actual or aspiring romantic partners and ex-spouses that an internal name emerged for it: “LOVEINT,” or Love Intelligence.

That same pattern of abuse is now emerging at the domestic law enforcement level. A Texas police officer misused, and then lied about, using license plate readers to track a woman suspected of seeking an abortion. Multiple law enforcement officials have been accused of tracking people they either wished to have a relationship with or who were their exes. And mass surveillance technologies have been used to track both immigration targets and citizens engaging in their First Amendment-protected right to track and record the police.

Mistakes are inevitable with collections of data of this size and scope. The history of the FISA court’s reviews of Section 702 is littered with examples of the NSA not being able to follow its own rules limiting the scope of what it collects and analyzes, even after having been given multiple chances by the court. On the local level, the technical protections that Flock, for example, put in place have repeatedly been insufficient to stop “accidental” sharing its data with out-of-state law enforcement. These mistakes have fueled growing efforts by local communities across the country to remove license plate readers. Those efforts should be the first step in a broader reconsideration of mass surveillance.

More generally, ubiquitous surveillance carries a real societal cost. The chilling effects are real and pervasive, and they tend to fall hardest on the most marginalized members of society. Moreover, social progress requires the ability to experiment in secret. It’s hard to imagine a society progressing morally to the point of accepting and legalizing things like marijuana use or gay marriage if the earliest signs of that shift are snuffed out because of overzealous surveillance.

Reversing Course

While a cost-benefit analysis is not the best frame for deciding constitutional rights, it is a place to start to evaluate government policies. If the costs are too high and the benefits too small, what should the public do? While the policy and legal frameworks can be individually complex, mass surveillance is a problem in all of its applications. So too should solutions be comprehensive rather than piecemeal.

One comprehensive strategy is to reset the promise of the Fourth Amendment and recognize that a warrant is required prior to collection, access or use of information gathered through mass surveillance. This would apply to collections that include U.S. persons, whether done for national security or domestic purposes. This protection would apply regardless of whether the information is in the form of metadata. It would apply regardless of whether the information is held in homes or by services people rely on, such as telephones, internet or social network providers, or by private entities utilizing mass surveillance for their own purposes. By passing this legislation, Congress could ensure this rejection of mass surveillance, and include real enforcement such as a private right of action and an automatic exclusionary remedy in criminal prosecutions. The courts could also recognize this protection of “papers and effects” directly as a plain language interpretation of the Fourth Amendment.

There are already a number of efforts that take on pieces of mass surveillance. Section 702 has expired and should remain so. This was due largely to efforts to block the “back door” access to Section 702-collected data without warrants. The bipartisan “Fourth Amendment is Not for Sale Act” would prevent the government from purchasing data that it would otherwise need a warrant to obtain. The Supreme Court itself has already been chipping away at the Third Party Doctrine, with a recent step in the rejection of mass geofence warrants—warrants seeking the identities of individuals based upon their proximity to a crime—in Chatrie v. United States. Now, such warrants fall, at least initially, under the Fourth Amendment.

A more comprehensive approach would also address mass surveillance carried out by private companies, and to ensure that Americans have the right to encrypt and secure their data. There are many reasons the United States would benefit from a comprehensive privacy law—and curbing mass surveillance is one of them. Addressing mass surveillance is certainly one of them. Ideas such as the banning of secondary uses of data—with roots in the Fair Information Practice Principles from the 1970s—are worth pushing forward. So are moves such as creating fiduciary duties for mass data collectors. There are many more ways to curtail private companies’ mass surveillance while staying within constitutional boundaries. But addressing the costs of mass surveillance by both companies and governments is even more important in a world where AI agents are making decisions both about the public and on their behalf based on their data and observed behavior.

Twenty-five years after the U.S. government embraced mass surveillance, it’s time to evaluate it as a whole, and consider responses that address the problem as a whole. Americans must ask: Is it consistent with a self-governing democracy to have systems that watch everyone everywhere? Is the public comfortable with governments—federal, state, local—that seek to “know it all” about its citizens? Is the public comfortable with private mass surveillance in its own right and as it’s being increasingly used to fuel government surveillance? These questions have long needed serious consideration. But as it becomes increasingly evident that the Trump administration is using mass surveillance to keep itself in power, stifle dissent, and undermine political opponents, these questions are now more urgent than ever.

The US has confirmed it deployed a space weapon in Earth's orbit, the first time it has acknowledged such offensive capabilities.

US Secretary of the Air Force Troy Meink said the "on orbit" weapon was necessary to safeguard US forces against hostile enemy action.

Speaking at the Air, Space and Cyber Conference on Monday, Meink said the US was ready for "evolving threats", but did not elaborate on the weapon's capabilities or when it was put into orbit.

Senior military leaders have previously said they planned to boost military capabilities in space, citing Russian and Chinese programmes developing ways to attack and disrupt US satellites in a future conflict.

China responded on Tuesday by warning against an "arms race" in space.

"We urge the US side to stop expanding its military capabilities and preparing for war in outer space," AFP quoted a foreign ministry spokesman as saying.

Details of how the weapon operates have not been disclosed by US officials.

A US Space Force spokesperson said: "Space control encapsulates the mission areas required to contest and control the space domain – employing kinetic and non-kinetic means to affect adversary capabilities through disruption, degradation and even destruction, if necessary.

"These capabilities can be employed for offensive and defensive purposes at the direction of combatant commands."

An expert on military uses of outer space noted that "precious little" was publicly known about the technology, and told the BBC there could be a "whole range of different weapons systems".

Speaking to Radio 4's Today programme, Dr Bleddyn Bowen, from the Royal United Services Institute (RUSI), said he was "totally guessing" but that the US could have been referring to "some kind electronic warfare or radio jamming platform".

That would be the sort of weapon that already exists on Earth and could disrupt radio communications, Bowen said.

An alternative would be "a kinetic kill vehicle of some sort - so a vehicle that would release a projectile of some sort that would ram into a satellite and destroy it".

But this was less likely because such a physically destructive tool would be capable of creating a lot of debris, Bowen added.

The Trump administration has said that space-based capabilities - and interceptor missiles - are a crucial part of its planned "Golden Dome" defence system to protect the US from missiles and other airborne threats.

Last year President Donald Trump issued an executive order underlining the US Space Force's role in not only defending assets, but also as an attacking force.

The US Space Force - the first new branch of the US military for more than 70 years - was launched in 2019 as a way to protect American assets in space, including hundreds of satellites used for communication and surveillance.

A 1967 treaty banned weapons of mass destruction in orbit. Since then, key powers including the US, Russia, and China have been exploring other capabilities outside of the treaty, such as targeting rivals' satellites - which are key for military communications, surveillance and navigation.

In early February, it was revealed that two Russian satellites had likely intercepted the communications of at least a dozen European ones since Russia's invasion of Ukraine in 2022.

Such interceptions may have allowed Russian intelligence services to read the sensitive data they transmit, or hypothetically, even to take control of the satellites.

In 2024, the US said Russia had launched a satellite which it believed may be capable of attacking other such probes. Russia did not publicly comment on the issue.

The US Space Force says China "develops and operates space and counterspace capabilities as part of a military modernisation strategy," while Russia "views space as a warfighting domain and believes space supremacy will be a decisive factor in future conflicts".

The latest edition of the Seriously Risky Business cybersecurity newsletter, now on Lawfare.

HACKERS (PT. 1) (Ifrah Yousuf, https://cybervisuals.org/visual/hackers-pt-1/; CC BY 4.0, https://creativecommons.org/licenses/by/4.0/)

Last week, Krebs on Security broke the story of a newly launched dark web service calling itself Nexus that was selling access to identity documents, including 3 million travel documents and 153 million driver's licenses from U.S. and Canadian citizens. This is a huge breach that not only will be used for run-of-the-mill cybercrime but also will feed the intelligence machines of America's adversaries.

Nexus claimed that it had gained unauthorized access to a major identity verification company and had spent more than a year "continuously" exfiltrating new data into a private database. Krebs on Security noted that in a single day the number of licenses in the database increased by nearly 400,000, suggesting regular ingestion of new data.

Krebs on Security was able to verify that the driver's licenses held by the service were genuine. In addition to Krebs’s own, it contained licenses from nine of his friends and family members. Secretary of War Pete Hegseth, an assistant director at the FBI and other high-ranking U.S. government officials also had licenses in the mix.

Based on a variety of circumstantial evidence, Krebs linked the incident to identity verification service IDScan. The service's website says it helps to reduce fraud by confirming that an ID is authentic and being presented by its legitimate owner and by detecting fraudulent documents.

The FBI is looking into the incident, and IDScan has confirmed it is investigating a data breach. The Nexus service also disappeared from the dark web shortly after Krebs published his story, although the people responsible for the hack do not claim to have deleted the data. Presumably they are lying low till the publicity dies down. 

Licenses and identity documents can be used to facilitate identity theft and phishing attacks, but because the data can be used to inform intelligence operations, an incident like this also has national security implications.

For the intelligence world, licenses are particularly valuable because they're key identity documents and license numbers are often used in other databases. These databases, whether hacked or purchased, become much more valuable when records can be linked directly to a particular person with home address and photo included.

And it's not a theoretical threat.

In the mid-2010s, Chinese cyber espionage actors stole complementary data from a variety of sources that, together, would be useful for analyzing the U.S. intelligence apparatus. Various Chinese APT groups stole information from the health insurance company Anthem, credit reporting company Equifax, Marriott hotels, United Airlines, and, perhaps most significantly, security clearance information from the Office of Personnel Management.

The U.S. intelligence community is certain that stolen data was used to counter American intelligence efforts against China, as described in this series of Foreign Policy articles by Zach Dorfman.

Of course, China itself isn't known for releasing detailed reports describing how it exploits its stolen data, but investigative research outfit Bellingcat has shown exactly how similar data can be used to uncover covert government activity.

In 2022, a hacked database provided a key piece of travel information that helped Bellingcat identify a deep cover GRU agent (Russian military intelligence) trying to infiltrate a NATO command post in Naples, Italy. And in another striking example, these three Bellingcat reports from 2018 identified suspects in the attempted assassination of Sergei Skripal with the Novichok nerve agent.

Clearly, leaked and hacked databases are incredibly useful for Bellingcat's Russia-related investigations. In 2020, it said it had "acquired dozens of leaked databases over the past few years, giving us a large number of data points to cross-reference and verify any new data we acquire."

If a small investigative outfit is hoovering up Russian data when it is leaked, you can bet your bottom yuan that China's intelligence services are doing the same for any American data that pops up.

The IDScan breach is big. The number of U.S. licenses in the database is roughly 63 percent of the country's total licenses. But breaches from identity verification companies occur depressingly frequently. In the past two years, breaches have occurred at AU10TIX, at Discord's age verification service provider 5CA, and at National Public Data.

Identity verification services are necessary to help to prevent fraud but are also a point of vulnerability when security is poorly done. The sheer volume of sensitive data these services handle means they should be subject to strict regulation and oversight.

We're realists here at Seriously Risky Business, though, and recognize that there is no chance of swift government action. In the short term, we can only hope that significant financial consequences will help encourage these firms to shore up their security. Law firms are already lining up class-action suits against IDScan, but a little federal government attention from the Federal Trade Commission wouldn't be unwelcome either.

The U.S. Military's Ad-Tracking Fig Leaf

Back in June, Reuters reported that commercial location data was being used to target U.S. military personnel in the Middle East. At the time, we wrote that the Department of Defense's existing policies regarding the issue, which already included disabling advertising identifiers on military-owned devices, did "not fill us with confidence." They simply weren't comprehensive enough.

It turns out that these policies weren't even being well implemented.

This week, Reuters reported that some branches of the U.S. military have finally gotten around to disabling some advertising identifiers on military-owned devices.

The U.S. Air Force said it disabled Windows and Android advertising identifiers in late July, although they were already disabled on Apple devices. The Army told Reuters it disabled advertising identifiers on Android and Apple devices by default in February. And U.S. Special Operations Command said that identifiers on Windows computers were "recently" disabled. 

Turning off these advertising identifiers is a fundamental mitigation that should have been implemented years ago. The U.S. military was first briefed in 2016 about the potential for commercial location data to be used to track its people to sensitive locations. In a striking 2018 example, an Australian Twitter user pointed out that Strava's global heat map could be used to identify U.S. military bases and even service members' jogging routes.

Disabling advertising identifiers on military devices is also an incomplete solution. It makes it harder to track them, but not impossible. And as it happens, many U.S. service members also use personal devices. Having a locked-down work phone is step one, but having good policies for personal devices is equally important.

Disabling advertising identifiers by default is the simplest short-term measure that might improve operational security (OPSEC), but it is impossible to know if it will make much of a difference without assessing the U.S. military's OPSEC posture holistically. Does disabling advertising identifiers on government devices reduce risk to an acceptable level? Probably not.

It's good that the U.S. military is finally disabling advertising identifiers by default. But we’re concerned the military has no idea how effective it will be.

"White Hats" Are Kidding Themselves

Over the weekend, self-proclaimed white-hat hackers stole $320 million worth of Bitcoin from the Liquid Network cryptocurrency platform. By Wednesday, the hackers had returned 85 percent of the funds, but kept around $47 million.

In recent years, there has been a regular drumbeat of steal-first-claim-reward-later hacks. We begrudgingly categorize several of these as successes as the perpetrators have not (yet) been arrested or jailed.

In 2021, a hacker stole $610 million worth of cryptocurrency from Poly Network. This was eventually returned in full, minus the  company's offer of $500,000 for the attacker it referred to as Mr White Hat.

Hacks of Multichain (2022), Huobi (2023), and Tender.fi (2023) had similar outcomes: Millions were stolen and returned, with the hackers taking a cut of tens or hundreds of thousands in cryptocurrency as a "reward" or "bug bounty."

The standout example is the 2022 hack of Mango Markets, in which a hacker extracted $110 million from the decentralized exchange. The individual responsible, Avraham Eisenberg, described his actions at the time as a "highly profitable trading strategy." He claimed all his actions were legal and he used the protocol as designed, "even if the development team did not fully anticipate all the consequences of setting parameters the way they are." 

Eisenberg returned $67 million to Mango Markets to recapitalize it, and the Mango community voted to give him a cool $47 million for his time. Eisenberg was convicted of fraud in a 2024 jury trial, but those convictions were overturned by a U.S. judge last year.

In our view, the perpetrators of these hacks are deceiving themselves. By returning most of the money, they're deluding themselves into thinking they're acting responsibly. As for consequences, the law won't chase me down if I return the majority and the victim says it's fine … right?

In Eisenberg's case, it did turn out to be right. But the FBI has been clear that victims cannot guarantee that perpetrators will not be prosecuted.

One wrinkle in the Liquid Network case is, as far as we can tell, the company never agreed to allow the hacker to keep a 15 percent cut.

It feels possible that this self-proclaimed good guy might have to spend some of that $47 million on a good lawyer.

Three Reasons to Be Cheerful This Week:

1. More options for trusted defenders: Last week, Google launched the Fairwind Program, its version of equivalent Anthropic's Project Glasswing and OpenAI's Trusted Access initiatives to limit more advanced AI cyber capabilities to vetted cyber defenders. On the same day, it launched Gemini 3.8 Flash Cyber, a cyber-specific version of its latest model that will be available through its Fairwind Program. Google says the model delivers "frontier-level" performance in vulnerability detection and patching but is far cheaper than competitor models.
2. Sality botnet takedown: Last week, the U.S. Department of Justice and Europol announced that an international operation had disrupted the Sality botnet. The botnet was first detected way back in 2003, and its peer-to-peer architecture meant there was no single point of failure that authorities could attack. CrowdStrike's blog on the takedown says that for the past eight years the botnet's primary payload, known as EggJagger, monitored the compromised host's clipboard for cryptocurrency wallet addresses and replaced them with addresses controlled by the malware operator.
3. U.S., U.K. to collaborate on scam networks: British and American authorities have signed a memorandum of understanding to collaborate on efforts to tackle scam compounds.

Risky Biz Talks

In our latest "Between Two Nerds" discussion, Tom Uren and The Grugq talk about whether AI will help cyber defense in critical infrastructure and organizations that are below the cyber poverty line.

From Risky Bulletin:

Ukraine's top prosecutor resigns amid scam call center scandal: Ukraine's top prosecutor, Ruslan Kravchenko, resigned on Monday over allegations that individuals in his office were taking bribes to protect scam call centers operating across the country.

His resignation comes after investigators from Ukraine's main anti-corruption body, the National Anti-Corruption Bureau (NABU), arrested Serhiy Kropyva, the deputy head of the Department of International Cooperation, a top lieutenant in Kravchenko's Office of the Prosecutor General.

In a report last week, NABU claimed it uncovered a major scheme in Kravchenko's office, where one of his department heads was taking bribes to look the other way when it came to a network of call centers that was calling Ukrainians and foreigners and luring them into fake investment platforms that stole their money.

[more on Risky Bulletin]

BEC campaign steals 35 million euros from French notaries: Hackers have stolen more than 35 million euro from French notaries in a massive business email compromise campaign over the past four years.

The attackers breached companies via phishing, took over their networks, and slowly and silently modified transaction details to hijack wired payments.

According to French newspaper Le Monde, the campaign hit more than 500 victims, or about 7 percent of all French notary offices.

[more on Risky Bulletin]

Russia tells data centers to deploy drone defenses: The Russian government has instructed data center operators to deploy protections against drone strikes and other physical threats as part of a national effort to boost defenses at critical infrastructure organizations.

Companies that fail to follow the Kremlin's instructions risk having their operations put under the state's administration.

Russian President Vladimir Putin signed a presidential decree last month allowing the state to temporarily take over the operations of critical infrastructure operators who fail to protect against Ukrainian hacks and drone strikes, or who take too long to repair damage.

[more on Risky Bulletin]

Today’s tidal wave of queries is forcing hardware makers to pivot

Tensordyne’s Napier chip is designed to accelerate AI inference.

Since about 2020, AI has largely focused on training bigger and better models. Large language models (LLMs) ballooned from millions of parameters to trillions. This proved effective: The largest version of OpenAI’s GPT-3, released in 2020, correctly answered just 43.9 percent of questions on a popular knowledge-and-reasoning benchmark. Just four years later, GPT-4o reached a score of 88.7 percent on the same exam, effectively matching those of human experts.

Advanced AI labs are still training ever larger models, but that training has somewhat receded to the background of the AI conversation. In 2026, inference—the use of trained models to produce code, write essays, or make images of ourselves as elves—has come to the forefront.

“It’s like training is yesterday’s news,” says Matt Kimball, principal data-center analyst at Moor Insights & Strategy. “All that any chief information officer wants to talk about is inference.” Nvidia CEO Jensen Huang, speaking at the company’s GTC 2026 conference, touted this change as the “inflection point of inference.”

Part of what’s caused the shift is very simple: LLMs are becoming useful, so people are using them. On top of that, many models on the market today are reasoning models. In response to a user’s query, they run inference not just once but multiple times, reprompting themselves in a process called chain of thought. Reasoning models generate longer outputs, and models with high reasoning effort can produce up to 20 times as much text as those with low or no effort. Adding even more to the world’s inference workload, the rise of agentic AI has resulted in inference running not just as a real-time response to a user’s query but also around the clock, working autonomously toward a user-defined goal.

Close-up of an Annapurna Labs metal processor chip with reflective black surfaces

Amazon’s Trainium chip was originally designed for AI training. However, Amazon Web Services chose to break up AI inference into two parts, with Trainium running the more computationally complex portion and Cerebras’s wafer-scale engine taking on the more memory-intensive portion.

The resulting explosion in inference demand has led to unexpected alliances among tech giants. OpenAI and Amazon have deployed chips the size of a dinner plate designed by Cerebras, despite Amazon having its own Trainium chips. Nvidia bought key talent and intellectual property from AI-inference startup Groq in a controversial deal worth US $20 billion. And Anthropic is paying LLM competitor SpaceXAI over a billion dollars per month to lease spare compute.

Although they might seem similar, AI training and AI inference are computationally different. These big moves from tech giants signal that in order to support the inference demand, we’re going to need a very different mix of hardware than experts may have expected even a couple of years ago.

How does AI inference differ from AI training?

An untrained LLM is like a jumble of Scrabble tiles on a table. Instead of single letters, though, the tiles show fragments of words, called tokens. Everything you’d need to write almost anything is present, but nothing makes sense.

Training a model organizes this jumble using a guessing game played at scale. The model is shown real text with the next token hidden and asked to predict what comes next. After each guess, the correct token is revealed and then compared to the prediction, and the difference is used to calculate the model’s accuracy. The game is played not with a single sentence but over billions of passages.

While a real game of Scrabble can be played over a bag of chips and a few drinks, AI training is computationally intense. The model updates its parameters through backpropagation, a process that repeatedly calculates how each of a model’s billions or trillions of parameters should shift to make the next prediction better. This is why tech giants are building larger data centers than ever before.

Eventually the model’s creator decides further training isn’t worth the cost, and the guessing game stops. Backpropagation ends, the parameters are frozen, and the LLM becomes a pretrained model. Fine-tuning—a short training run on smaller, more specialized data—adds final tweaks, and the model is deployed.

Close-up of a gold computer chip with rainbow-colored circuitry on black background

Nvidia’s Groq 3 language-processing unit minimizes data movement by placing on-chip SRAM memory and computational blocks in the order they are needed on-chip.

Next comes inference. This is the process of using the deployed model, which, now that it’s been trained, has learned to spit out Scrabble tiles—tokens—in a sensible order.

You might think that AI inference is less computationally demanding because the backpropagation calculations used to update parameters are eliminated. But Sudeep Bhoja, founder and CTO of the inference-hardware company d-Matrix, explains that inference adds new challenges.

The models are “autoregressive” in nature. That is, the next output depends on the previous one. “So to generate the next token, you have to read all of the weights and all of the [context] from the previous token,” explains Bhoja. The context includes all of your prompts, all of the LLM’s replies, and all of the files you upload. It’s a lot of data and a lot of processing.

An LLM generates its reply in two phases: prefill and decode. Prefill is the model reading a prompt. It processes every token at once, computing how each token relates to all the others. This operation is called attention, and it’s a defining characteristic of the transformer architecture behind modern LLMs. It allows them to respond to a word in its sentence, paragraph, and larger context rather than on its own. Think of it like arranging Scrabble tiles before you place them in a game. Many players move tiles around to imagine how they connect. Self-attention plays a similar role, though instead of moving physical tiles, each token sends a query to the others and receives a score indicating the token’s relevance.

These queries result in two types of vectors: the keys and values. They are typically placed in a store called the KV cache. This isn’t strictly required, as a model could instead recompute these vectors with each new token it generates. But nearly all LLMs use a KV cache to reduce how much computing they do. The KV cache is stored in memory and becomes a scratchpad to which the LLM can return to understand a conversation, and though it starts small, it can swell to dozens of gigabytes.

Prefill is a problem that can be easily divided up and worked on in parallel. This is why GPUs became the dominant AI accelerator as LLMs surged in popularity. Graphics rasterization (computing the color of every pixel on a screen) is also massively parallel, so GPU architectures were a natural fit.

Gloved hands holding a large golden computer processor wafer

Cerebras’s wafer-scale engine chips maximize memory bandwidth by keeping everything—both memory and computational units—side by side on the dinner-plate-size chips.

Next comes decode. Here, the model generates its reply one token at a time. At each step it takes the most recent token, weighs it against everything in the KV cache, uses that information to predict the next token, and adds the new token’s key and value to the cache. Then it repeats in sequence, token by token.

This is where the autoregressive nature of the model works against inference speed. Predicting each token requires reading the entire model from memory, and that model consists of possibly tens to hundreds of gigabytes of parameters (the numbers representing what the model learned in training). Crucially, this is in addition to the memory required to store the KV cache.

As a result, the movement of all this data through memory often requires more bandwidth than inference hardware has available. So at least some of the computing parts of a GPU sit idle as it waits for data. Researchers found that Nvidia H100 GPUs running open-source LLMs sit idle 50 to 80 percent of the time.

Memory’s role in inferencing

Shahriar “Sha” Rabii, former head of silicon engineering at Meta and cofounder of the AI startup Majestic Labs, says idled processors are why many companies that are trying to improve AI-inference performance are laser-focused on memory. “With the GPU-based approach, you end up greatly over-provisioning compute and starved on memory. That’s driving the big [memory] scale out,” he says.

Bhoja’s d-Matrix and Rabii’s Majestic Labs both focus on this memory bottleneck. However, their companies imagine different solutions.

d-Matrix’s second-generation AI accelerator, Raptor, aims to improve inference performance by minimizing the distance between compute and memory. The GPUs in most current AI-inference deployments do this by placing high-bandwidth memory (HBM) around the perimeter of the GPU. Each HBM is a stack of DRAM dies linked together and connected to a superfast interface to the GPU. This is great for training, but for inference, the amount of memory you can stack this way and the bandwidth it can provide leave something to be desired.

d-Matrix’s stacked-die architecture

Diagram of stacked logic and DRAM chips connected by solder bumps on a substrate

Memory bandwidth—how quickly data can be read from memory to logic—is a major bottleneck in AI inference. The startup d-Matrix is increasing that bandwidth by stacking the logic die directly on top of the memory, in this case DRAM. This allows for lots of extremely short interconnects.

d-Matrix’s Raptor removes that bottleneck by stacking an AI accelerator on a DRAM die. Instead of stacking memory, d-Matrix stacks memory and compute. Bhoja says this reduces the distance that data must travel to “micrometers instead of millimeters.” Like building a skyscraper, going vertical makes it possible to do more inside the same physical footprint.

Majestic takes the opposite approach. Instead of trying to minimize the length that data must travel between compute and memory, the company is focused on improving the memory interface to accommodate longer wire traces while keeping bandwidth high. Longer wires allow Majestic to connect memory stacks that aren’t directly next to the GPU, removing the space limitation of HBM.

“A memory interface has a very short physical distance it can operate over. In the case of HBM, it’s up to 2 or 3 millimeters. You have this shoreline around the periphery, which is the only place where you can put HBM,” says Rabii.

Majestic claims its memory interface can transmit bits as far as about a meter. That’s achieved with a proprietary copper link and a memory-aggregator chip that coordinates data. “The aggregator is the endpoint for the high-speed interface and a way to fan out to many, many commodity DRAM chips,” says Rabii. Because of this, Majestic can support up to 128 terabytes of DRAM memory in a single server rack—a significant increase over Nvidia’s GB300 NVL72 rack, which has about 20 TB of HBM3E.

Majestic Labs’ memory-aggregation architecture

Diagram of memory aggregator chiplet linking server GPUs/CPUs to shared DRAM pool

Majestic Labs plans to satisfy AI’s memory appetite via a proprietary interconnect and a memory-aggregator chip, allowing a single rack access to up to 128 terabytes of cheap DRAM memory.

d-Matrix and Majestic have one thing in common: Instead of HBM, they both use off-the-shelf DRAM. This is the most common type of computer memory in the world; it’s in everything from smartphones to cars. Memory analyst Jim Handy says HBM costs two to three times as much as DRAM. d-Matrix and Majestic chose DRAM in part because of this price advantage. However, the proponents of HBM, which include memory giants like Samsung and SK Hynix, aren’t sitting idle.

HBM4, the latest version of HBM memory, is now in production and will be used by Nvidia’s Vera Rubin GPU, which is expected to ship in the second half of 2026. Hoshik Kim, head of memory-systems research at SK Hynix, says HBM4 “will decisively break the memory bottlenecks constraining AI inference today” by doubling HBM’s maximum memory bandwidth and increasing the amount of HBM memory per stack.

Combining chips for faster inference

The big players—Nvidia and Amazon—are going for an all-chips-on-deck approach. Nvidia’s GPUs and Amazon’s Trainium training accelerators are still great for part of the inference workload: the prefill stage, where all the context keys and values are calculated. But to accelerate decode, the part where new tokens are generated, they are looking to new, memory-centric architectures from smaller players.

In Nvidia’s case, the smaller player was Groq (not to be confused with Grok, the family of LLMs trained by SpaceXAI). Nvidia purchased intellectual property and hired talent from Groq at the end of 2025, and just three months later at the Nvidia’s GTC 2026 conference, Jensen Huang unveiled the Nvidia Groq 3 language-processing unit (LPU). Groq’s architecture relies on memory—in its case, SRAM—built directly into the chip’s architecture.

Unless you’re a chip architect, or a hardcore PC gamer, you probably never give SRAM a thought. SRAM has the benefit of being tightly integrated into a compute chip’s architecture—it’s on the same piece of silicon as the processor—and has the drawback of being less dense and more expensive than DRAM. Most chips include only a few dozen megabytes of SRAM. AI inference, however, has ignited new interest in SRAM as a means of bringing the model weights stored in memory closer to compute.

Ian Buck, vice-president and general manager of hyperscale and high-performance computing at Nvidia, says the LPU has a much different set of priorities than the company’s GPUs. The LPU has far less raw computing power than a standard GPU, but it gains 500 megabytes of on-die SRAM connected directly to its floating-point math units. “The benefit is the memory bandwidth. The LPU has seven times the memory bandwidth of the GPU,” he says.

Between the Rubin GPU and the Groq LPU, prefill and decode can both be accelerated to get the best of both worlds, the theory goes. “We do all the attention math and context processing on the Vera Rubin [GPU] rack,” explains Buck. “For all the expert calculations…the matrix multiplications, we do that part on the LPU.” The company packs 256 LPUs into the Groq 3 LPX, a system the size of a data-center rack.

Nvidia’s two-chip approach to inference

Diagram comparing Nvidia Rubin GPU and Groq 3 LPU chip layouts with labeled blocks

Nvidia also plans to split the inference workload across two chips. The company’s newest Rubin GPUs will tackle the compute-intensive prefill phase, while the Groq 3 language-processing unit (LPU), with lots of on-chip SRAM, will handle the memory-intensive decode phase.

Amazon Web Services (AWS), for its part, struck a deal with Cerebras, to pair the Trainium accelerator with Cerebras’s Wafer-Scale Engine 3 (WSE-3). Cerebras takes a similar approach to Groq, though at a much larger scale. WSE-3 turns an entire silicon wafer into a single chip that contains over 4 trillion transistors. The design doesn’t connect to external memory but instead etches 44 gigabytes of SRAM into each wafer. “We store the [model] weights on the SRAM,” says James Wang, formerly director of product marketing at Cerebras who has since moved to SpaceXAI. “So that’s easily 40 to up to 80 billion parameters that we can support on one chip.”

Amazon plans to use AWS Trainium chips for prefill, and Cerebras for decode. But Cerebras’s chips can also go it alone in inference. WSE-3 was deployed by OpenAI to power GPT-5.3-Codex-Spark, a variant of the company’s coding mode, outputting over 1,000 tokens per second. For comparison, OpenAI’s standard GPT-5.4 deployment outputs 50 to 125 tokens per second.

Amazon Web Services’ two-chip inference strategy

A schematic of Amazon's Trainium chip on the left, with SRAM memory block and logic blocks plus high-bandwidth memory. Schematic of Cerebras's wafer-scale engine on right, with small SRAM memory and logic block in a checkerboard pattern.

Amazon Web Services combined their Trainium chips with Cerebras’s dinner-plate-sized wafer-scale engine (WSE) to tackle different parts of AI inference. Trainium chips handle the computationally intensive prefill phase, while the WSE, with interleaved on-chip SRAM memory, handles the memory-bandwidth-limited decode phase.

Cerebras can also tackle prefill without moving the workload to different specialized chips. For this, it networks together multiple WSE-3 chips to form a single pool of memory. Cerebras has demonstrated it can serve models with up to 1T parameters, such as Moonshot AI’s Kimi 2.6, though Wang says “the architecture has no innate limitation in terms of how many parameters it will do.”

Despite these differences in strategy, Nvidia and AWS seem to agree that the future of AI inference will be solved by a systems approach that pools different kinds of chips together to tackle the largest LLMs. Or, as Buck says: “To do modern AI inference, you need all the chips.”

Learning to do more with less (bits)

Nvidia became the world’s most valuable tech company because it designed the world’s most desired GPUs. But not all of the attention is focused on improving AI-inference hardware. AI researchers are also learning how to optimize LLM software and hardware in tandem to make the best use of the memory and compute components.

Most computers store numbers in a 32-bit or 64-bit format. These determine how many bits are available to represent a single number. If too few bits are available, the number can’t be stored without losing information. The quality of an LLM benefits from more-precise number formats, but this creates a problem for inference performance. More-precise numbers aren’t free. The bits that describe them take up more space in memory and require more silicon and energy to compute.

Gilles Backhus, cofounder of the AI-accelerator company Tensordyne, says this creates a tension between model size and number precision. “Would you prefer a model that is size x but runs in 8-bit, or would you prefer a model that is twice the size but runs in 4-bit?” The size of each model will be roughly the same in terms of memory and compute, “but the 4-bit approach gives you twice as many synapses, if you will. And people are figuring out that [the 4-bit approach] is worth it.”

The process of converting an LLM from a more-precise number format to a less-precise format is called quantization, and it’s been in use for several years. However, researchers are finding new ways to quantize models down while retaining a large majority of the model’s quality.

Nvidia recently created a new 4-bit number format, NVFP4, for this purpose. AMD, Intel, and Qualcomm have instead rallied around a competing 4-bit number format called MXFP4 that Nvidia also contributed to developing. “It’s the black art of AI,” says Buck, of Nvidia. When Nvidia quantized DeepSeek-R1 from FP8 to NVFP4, scores on seven major benchmarks degraded by less than one percent while performance improved by three times, the company says.

Quantization is likely just the tip of the spear, as AI researchers and startups are investigating a diversity of opportunities for optimization, some of which could dramatically change the silicon found in AI-inference hardware.

TENSORDYNE TDN AIP chip with central green processor cores on black board

Tensordyne’s unique approach to AI inference combines a logarithmic number format with bespoke hardware in the company’s Napier chip.

Tensordyne is expected to accelerate AI inference with a logarithmic number system that leans on a property of logarithms: The log of A times B equals the log of A plus the log of B. So, storing numbers as their exponents lets the chip add where it would otherwise multiply. That matters in silicon because multiplier circuits draw more power and use more die area than adders do. Tensordyne says its rack-scale hardware, called Napier, can produce up to 1,300 tokens per second per user, and can do so while using less than a tenth as much power as comparable Nvidia hardware.

Etched, a startup based in San Jose, Calif., is even designing AI accelerators that translate the transformer architecture used by LLMs directly into silicon. Rather than building general-purpose GPUs, the company is wiring up the connections needed for efficient transformer calculations into its chip, making the chip much less flexible but more efficient for the tasks most performed by current LLMs. The company says its first AI accelerator, Sohu, can run Meta’s Llama 70B model at a stunning 500,000 tokens per second, though this approach also means it won’t be able to run LLMs that move away from a typical transformer architecture.

Whether these ideas will prove fruitful remains to be seen. Etched just shipped their first rack in August. Tensordyne believes its first hardware will be available in 2027. Even so, these startups show how the demand for inference performance is fueling unconventional ideas.

Inference is everyone’s game

The sheer variety of approaches to AI-inference acceleration—stacking compute on memory, extending interfaces from millimeters to meters, using an entire silicon wafer for SRAM, squeezing models into 4 bits—raises a question: Which is going to win, and which is going to lose?

But that’s likely not the right question, experts say. The demand for AI is currently insatiable, and while fears of an AI bubble stalk the industry, it has yet to hamper growth.

On the contrary, Kimball of Moor Insights & Strategy thinks inference could drive intense demand for AI hardware in the long term, because it’s not obvious where that demand will end. “You could add a million agents into your organization,” he says. “These things work 24 hours a day; they don’t go home at five at night like we do.”

If AI inference remains as desirable as Kimball expects, the evolution is likely to follow the same trajectory as the CPU. The CPU didn’t improve along a single axis but instead across multiple fronts simultaneously. Once transistor scaling slowed, chip and system architecture innovations of all kinds proliferated. The list of individual innovations that led to today’s ubiquitous, powerful personal compute could fill dozens of books.

A few decades from now, the history of AI inference innovation will show similar depth.

Standard deep learning relies on backpropagation. The brain, however, cannot implement backpropagation, at least not exactly[1, 2]. How the brain solves the multilayer credit assignment problem without explicit use of backprop remains one of the fundamental unsolved problems in neuroscience (though not without progress[3, 4, 5]).

There are several reasons the brain can't implement exact backpropagation. One is “phase locking”[6, 2]. Backpropagation runs in three phases, in strict order: 1) a forward pass, then 2) a backward pass, then 3) a weight update. A weight update is locked until the forward and backward passes have completed—a neuron in an early layer must hold its activation and wait for the error signal to arrive. The brain has no known mechanism that could enforce such strict timing coordination across an entire network[1].

In this post, we introduce PC-ALM (Augmented Lagrangian Predictive Coding), a method for training networks that replaces the forward and backward passes of backprop with layer-local dynamical systems. Each layer is coupled only to its neighbors. Instead of forward-then-backward, we run each layer forward in time. When run to convergence, the dynamics of the whole system distribute supervision credit signals quickly and accurately across the entire network.

PC-ALM is an extension of standard predictive coding (PC)[7, 8, 9, 10]. PC uses diffusive (i.e. energy-based or "heat flow") coupling between layers. Compared to PC, PC-ALM introduces dual neurons (Lagrange multipliers) per layer, making each layer's local recurrence a PI feedback controller. In the limiting case of linear networks, the dual neurons converge to the exact backprop credit signals, despite using only local computation.

We compare PC-ALM to PC and backprop in a suite of experiments. Local training methods such as PC have historically been difficult to scale. Following the PC literature, we use simple tasks (Fashion-MNIST, CIFAR-10, etc.) and networks such as residual MLPs.

We show that PC-ALM can successfully propagate supervision credit in 1000-layer neural networks, overcoming standard PC's signal decay problem[11] while remaining layer-local.

We focus on deep, small-width networks, a regime in which PC tends to perform poorly.

Ultimately, our motive is to understand how distributed systems (such as the brain) can implement gradient computations without backpropagation. Scientific motivations aside, this research may inform energy-efficient deep learning on neuromorphic hardware, where dynamical systems simulation is cheaper than on GPU[12].

Predictive coding: each layer as a dynamical system

Before explaining PC-ALM, let us first explain PC, interpreting it from a dynamical systems standpoint to emphasize its role as a backprop alternative.

Predictive coding

Predictive coding has its roots in Helmholtz's theories of unconscious perception[13]. Rao & Ballard (1999) developed a mathematical framework for PC as a model of visual cortex[14]. The idea of PC is that each layer attempts to model its incoming signals, sending upward only the prediction error (the part that the layer failed to model) to the next layer.

Mathematically, predictive coding utilizes a general motif: take a state and update it to reduce a prediction error at the next step,

statet+1=statet−η(statet−targett)⏟prediction error

By applying this update rule to each layer's activation vector (the “state” is the layer's activation hi; its “target” is the prediction σ(Wihi−1) arriving from the layer below)1, the PC framework effectively sidesteps backprop's need for a synchronized forward and backward pass.

To explain this in more detail, let's write a feedforward network as a constrained optimization problem:

minimizeθ,h12‖y−WLhL−1‖2subject tohi=σ(Wihi−1),i=1,…,L−1.

where L is the network depth, h0:=x the input, y the target, θ={Wi} the weights, hi the layer activations, and σ an activation function such as ReLU. Note that each hi is an optimization variable2. We then construct a new loss function that includes the original supervision loss, together with a quadratic penalty for violations of each layer constraint:

FPC(h,θ)=12‖y−WLhL−1‖2+12∑i=1L−1‖hi−σ(Wihi−1)‖2.

This is a quadratic relaxation of the constrained problem. FPC is known as the “free energy” of the network[15, 9].

To train a neural network, PC alternates between inference and learning steps:

Predictive coding

inference

for t=1,…,T

hi←hi−ηh∇hiFPCfor i=1,…,L−1

learning

Wi←Wi−ηθ∇WiFPCfor i=1,…,L

Per mini-batch, a forward pass initializes the activations, followed by T inference steps and a single weight update. We set T proportional to network depth; the 1000-layer experiments below use T=2L.

Each hi-update reduces the prediction errors between layers adjacent to i. This is because ∇hiFPC depends only on hi−1, hi, and hi+1. Inference requires only nearest-neighbor communication ("message passing") between layers. Explicitly, writing ri=hi−σ(Wihi−1) for the prediction error between layers i−1 and i, the inference update reads3:

hi←hi−ηh(ri↑error below−Wi+1⊤(σ′⊙ri+1↑error above))i=1,…,L−1

The bottom h0 is “clamped” (fixed) to an input value and the top of the network is clamped to the target y. Running T update steps, the network settles into states hi for each layer, after which a gradient descent step is taken on the same FPC but now with respect to the weights W (given the current remaining prediction errors and the current state activations).

Wi←Wi+ηθ(σ′⊙ri)hi−1⊤i=1,…,L−1

Both the inference step and the weight update are layer-local. The weight update is Hebbian-like, in that it multiplies a postsynaptic error by the presynaptic activity (a delta rule), and the dynamics map onto a neural circuit with explicit error neurons[14, 7].

PC trains deep networks, but exhibits signal decay

PC inference in a 32-layer residual MLP (width 16, ReLU) at weight initialization. Credit = per-layer norm of the prediction error ri. Dashed reference: norm of the backprop adjoint (the loss gradient with respect to that layer's activations).

Since minimizing the free energy with respect to each hi does not enforce the layer-wise constraints to hold exactly, PC results in a different learning trajectory compared to standard backpropagation.

Nevertheless, PC has been shown to successfully train networks on simple tasks. For example, MNIST and Fashion-MNIST in 128-layer residual MLPs with wide layer widths (512 neurons per layer)[16].

However, PC struggles on more complex tasks and networks[17]. Further, PC struggles even on simple networks/tasks if the network width is smaller than its depth[18].

Each layer adjusts its activity to reduce prediction errors with its neighbors. Supervision enters at the output, but must work its way through this chain of local compromises to influence earlier layers. In deep, narrow networks, the resulting credit signal becomes weak long before it reaches the input.

This leads to a documented signal-decay problem of PC[11], as illustrated in the above figure. Increasing T lets credit propagate farther, but requires more computation for each training update.

Our method, PC-ALM, introduces a way to improve signal propagation of PC networks, retaining the layer-local dynamics of PC and keeping inference budget T proportional to network depth.

Augmented Lagrangian Predictive Coding

We propose Augmented Lagrangian Predictive Coding, a variant of PC that uses the augmented Lagrangian (AL)[19, 20, 21] in place of PC's FPC:

L(h,θ,λ)=12‖y−WLhL−1‖2+∑i=1L−1λi⊤(hi−σ(Wihi−1))+12∑i=1L−1‖hi−σ(Wihi−1)‖2

supervised loss + Lagrangian term + PC energy

In each layer, the augmented Lagrangian introduces a Lagrange multiplier (or dual variable) λi∈Rni of the same dimension as hi.

The augmented Lagrangian is used extensively in distributed optimization[22], and has motivated many distributed methods for training deep networks[23, 24].

LeCun (1988)[25] showed that the Lagrange multipliers of a constrained network encode its backprop credit signals at equilibrium. The augmented Lagrangian combines this classical construction with the quadratic prediction-error penalties already used by PC. This suggests a simple possibility: can PC’s local dynamics recover those credit signals if we add the multipliers?

To use the augmented Lagrangian for training, we make a simple modification to PC:

Augmented Lagrangian predictive coding

inference

for t=1,…,T

hi←hi−ηh∇hiLfor i=1,…,L−1primal

λi←λi+α(hi−σ(Wihi−1))for i=1,…,L−1dual

learning

Wi←Wi−ηθ∇WiLfor i=1,…,L

Here α is the dual step size. PC-ALM is primal descent, dual ascent on the augmented Lagrangian, versus PC's descent on the energy.

By accumulating local prediction errors, the dual variables recover the exact backprop credit signals in a deep linear network. We derive this result in the paper.

Thus, at least for linear networks, PC-ALM gives a method for computing exact supervised loss gradients and distributing them throughout a network, using only layer-local dynamics.

In the experiments below, we test whether this advantage carries over to nonlinear networks.

Mechanistic interpretation

To understand, mechanistically, how PC-ALM works, consider a simple scalar network, with hidden unit h=w1x and output y^=w2h. We want to propagate the gradient of the supervised loss 12(y−y^)2 to w1.

We attach a multiplier λ to the constraint h=w1x, initialize λ=0, and initialize h at its forward-pass value.

The first step of PC-ALM agrees exactly with a PC step. After that λ accumulates the layer's prediction error each step, which feeds back into the h updates. Each h update now takes a gradient step on the energy with prediction targets shifted by the dual.4

At convergence the activation h returns to its forward-pass value (restoring the constraint h=w1x), while λ has accumulated to the backprop credit signal of that layer: i.e. λ=w2(y−y^).

PC vs PC-ALM in a two-layer, linear, scalar network, y^=w2w1x, with x,y clamped to data values. Left: training trajectories of BP, PC, and PC-ALM in weight space. In this simple model, both PC and PC-ALM work and converge to the solution manifold. Performance differences between PC and PC-ALM become apparent in deep-narrow networks.

Control theory and credit assignment

PC-ALM offers a control-theoretic perspective on credit assignment. Each layer combines its current prediction error with an accumulated error signal—the proportional and integral terms of a PI feedback controller.5 Global credit assignment emerges from a network of local feedback controllers. We believe this perspective offers a useful principle for designing new local learning algorithms.

Results

We present below some results for PC-ALM.

PC-ALM: a local learning method for training 1000-layer neural networks

PC-ALM successfully trains 1000-layer MLPs on MNIST. We use the residual MLP setup from Innocenti et al. (2026)[18]. We train for five epochs. Our architecture is a simple MLP with residual skip connections, using weight parameterizations that stabilize backprop training at this depth.

Our results are presented in the figure below; PC-ALM achieves near-BP performance using only layer-local dynamics.

MNIST test accuracy versus depth, from shallow networks through 1000 layers, for BP, PC-ALM, and PC. Backprop is global; PC and PC-ALM are local.

MNIST test accuracy vs depth (width N=32, ReLU activation). PC-ALM stays within ~2 percentage points of backprop across the whole range, including at 1000 layers.

Image classification benchmarks

We tested PC-ALM on a small set of image classification tasks and found that it improves performance over PC on every task, including when training ResNet-18 on CIFAR-10 and Tiny ImageNet.

Test accuracy for BP, PC, and PC-ALM across MNIST and Fashion-MNIST at 32, 64, and 128 layers, plus CIFAR-10 and Tiny ImageNet.

Across benchmarks. PC-ALM consistently narrows the gap between standard PC and global backprop. As depth increases, PC’s accuracy falls off much faster than PC-ALM’s.

PC-ALM propagation dynamics

In addition to the performance of PC-ALM across image recognition tasks, we find PC-ALM exhibits surprising dynamical properties. In both PC and PC-ALM inference, movement of the hidden activations hi initially appears only in the final network layers. As inference proceeds, earlier layers receive credit signal, forming a wavefront that advances toward the input. PC-ALM’s primal-dual dynamics drive this wavefront through the network faster than PC. We call this "ballistic" credit propagation to contrast with PC's diffusive, heat-flow-like propagation.

Ballistic vs diffusive credit propagation. Credit magnitude across layers during inference. PC's credit decays with depth; PC-ALM's spreads evenly across the network.

Stable oscillatory transient responses

Individual neurons in PC-ALM also exhibit damped oscillations during inference.

Oscillatory inference dynamics. We consider a deep linear network (σ=identity) with depth L=8 and width N=1. With weights fixed, the coupled inference dynamics across all layers form a linear system, characterized by the eigenvalues shown on the left (Equation (17) of the paper). Right: a neuron’s activity and dual variable during inference. The parameter α is the dual step size. Setting α=0 recovers PC exactly, with all eigenvalues real; increasing α introduces complex eigenvalues and oscillatory dynamics. Increasing α too far eventually destabilizes the network (Equation (14) of the paper).

Discussion

This work introduces PC-ALM, a layer-local alternative to backpropagation for training deep networks. To our knowledge, this is the first layer-local method shown to successfully train networks up to 1000 layers.

Summary

Predictive coding remains an attractive candidate for a theory of cortical function. It is rooted in Helmholtz's ideas on unconscious processing. It was later shown that predictive coding can be viewed as a layer-local alternative to backpropagation for training deep networks. PC in its standard form can successfully train networks on simple tasks, but signal decay limits its performance in deep, narrow networks. By introducing Lagrange multipliers and running primal-dual inference on the augmented Lagrangian (vs PC's gradient flow on an energy), PC-ALM lets individual layers compute a gradient signal of a global loss function, using only communication between neighboring layers. The primary motivation of this work is to understand more deeply the mechanisms of credit assignment in real physical systems such as the brain.

Constrained and lifted optimization

PC-ALM relates to a long line of work on constrained optimization approaches to training deep networks. These approaches “lift” training into a larger optimization problem by treating the activations h as variables alongside the weights W. This includes the method of auxiliary coordinates[26], ADMM training of deep networks[23, 27, 24], BlockProp[28], ProxProp[29], several other variants[30, 31, 32, 33, 34, 35], and augmented Lagrangian methods for network training[36, 37].

PC-ALM draws on this optimization lineage to address a question from neuroscience: how can local neural dynamics compute and distribute credit for a global objective?

Prospective-configuration tradeoff

In PC, the settled activations differ from the forward pass. Song et al.[38] called this prospective configuration and showed that it can improve sample efficiency over backprop. PC-ALM improves credit propagation but gives up prospective configuration at convergence. We suspect a fundamental tradeoff between the two, with intermediate settings of T, α, and dual leak potentially preserving the benefits of prospective configuration while substantially improving credit propagation.

Motivations

This work began with three observations:

1. The Neuro-AI and distributed optimization communities share a concern with locality, but there has been relatively little cross-talk between them.

2. The PC energy is suspiciously similar to the augmented term of the augmented Lagrangian that is commonly used in distributed and constrained/lifted optimization.

3. LeCun (1988) identified the multipliers of the standard Lagrangian with backprop credit signals, suggesting that these multipliers could serve as local credit signals.

Our work on PC-ALM ties these threads together.

Future work

There is a wealth of work that needs to be done. Notably, extending PC-ALM to temporal tasks with temporal credit assignment[39, 40], self-supervised losses[41], and of course larger networks and more difficult tasks.

We hope this work inspires further connections between augmented Lagrangian methods and biologically plausible credit assignment.

In late 2025 the company behind MinIO decided to abandon it to pursue other commercial interests. As well as upsetting a bunch of folk, it also put the cat amongst the pigeons of many software demos that relied on MinIO to emulate S3 storage locally, not to mention build pipelines that used it for validating S3 compatibility.

In this blog post I’m going to look at some alternatives to MinIO.

Whilst MinIO is a lot more than 'just' a glorified tool for emulating S3 when building demos, my focus here is going to be on what is the simplest replacement. In practice that means the following:

• Must have a Docker image.

  • So many demos are shipped as Docker Compose, and no-one likes brewing their own Docker images unless really necessary.

• Must provide S3 compatibility.

  • The whole point of MinIO in these demos is to stand-in for writing to actual S3.

• Must be free to use, with a strong preference for Open Source (per OSI definition) licence e.g. Apache 2.0.

• Should be simple to use for a single-node deployment

• Should have a clear and active community and/or commercial backer.

  • Any fule can vibe-code some abandon-ware slop, or fork a project in a fit of enthusiasm—but MinIO stood the test of time until now and we don’t want to be repeating this exercise in six months' time.

• Bonus points for excellent developer experience (DX), smooth configuration, good docs, etc.

I’m purely looking at the single-node local S3 experience. Several of the tools discussed below have a wide range of features, S3 support being just one of them. Others offer S3 alone and that’s it. For my purposes either is fine, so long as any additional features and support don’t add to the complexity or weight of deployment.

What I’m not looking at is, for example, multi-node deployments, distributed storage, production support costs, GUI capabilities, and so on. That is, this blog post is not aimed at folk who were using MinIO as self-managed S3 in production. Feel free to leave a comment below though if you have useful things to add in this respect :)

All of the code in this project can be found in this repo.

MinIO baseline

My starting point for this is a very simple Docker Compose stack: DuckDB to read and write Iceberg data that’s stored on S3, provided by MinIO to start with.

You can find the code here.

The Docker Compose is pretty straightforward:

1. DuckDB, obviously, along with Iceberg REST Catalog
2. MinIO (S3 local storage)
3. mc, which is a MinIO CLI and used to automagically create a bucket for the data.

When I insert data into DuckDB:

INSERT INTO cat.test.products VALUES
    (1, 'Widget', 9.99),
    (2, 'Gadget', 19.99),
    (3, 'Doohickey', 14.99);

it ends up in Iceberg format on S3, here in MinIO:

$ mc ls minio/warehouse/test/products/data/

[2026-01-12 14:35:55 UTC]   693B STANDARD 019bb2a2-8ad5-7159-968c-3693c3578902.parquet
[2026-01-12 14:35:55 UTC] 2.7KiB STANDARD 188ee032-8576-4f46-9834-1cac6d3080a3-m0.avro
[2026-01-12 14:35:55 UTC] 1.6KiB STANDARD snap-2633370813252912097-95ad4df0-3243-4cfb-a736-8f785aac100c.avro

In each of the samples I’ve built you can run the test.sh to verify it.

❯ ./test.sh

    ░▒▓██████████████████████████████████████████████████▓▒░
    ▒                                                      ▒
    ▓      ___                    ▓▓▓▓▓                    ▓
    █  ___( o)>  ┌───────────┐  ▓▓▓▓▓▓▓  ▒▒▒▒▒▒▒           █
    █  \ <_. )   │  DUCKDB   │  ▓░Local S3 ░▓  ▒▒▒▒        █
    █    ---     └───────────┘   ▓▓ MinIO  ▒▒▒             █
    ▓           ┏━━━━━━━━━━━━━┓        ░░░                 ▓
    ▒  ≋≋≋≋≋≋≋  ┃  ICEBERG    ┃  ≋≋≋≋≋≋≋≋  Smoke test      ▒
    ░  ≋≋≋≋≋≋≋  ┗━━━━━━━━━━━━━┛  ≋≋≋≋≋≋≋≋                  ░
    ░▒▓██████████████████████████████████████████████████▓▒░

1. Checking MinIO buckets (before)...
[2026-01-14 12:48:41 UTC]     0B warehouse/

2. Creating Iceberg table and inserting data...
[…]

3. Verifying data in DuckDB...
[…]

┌───────┬───────────┬───────────────┐
│  id   │   name    │     price     │
│ int32 │  varchar  │ decimal(10,2) │
├───────┼───────────┼───────────────┤
│     1 │ Widget    │          9.99 │
│     2 │ Gadget    │         19.99 │
│     3 │ Doohickey │         14.99 │
└───────┴───────────┴───────────────┘

4. Checking MinIO bucket contents (after)...
[2026-01-14 12:49:07 UTC]   493B STANDARD data/019bbc8d-7d44-7b6b-af71-0ab99d3a0124.parquet
[2026-01-14 12:49:07 UTC] 2.7KiB STANDARD data/c37ab68c-8aaf-44a4-88e6-bc02a01daf5c-m0.avro
[2026-01-14 12:49:07 UTC] 1.6KiB STANDARD data/snap-1988745121468861337-95e77212-9571-423f-bb71-ce8f1b4282f0.avro
[2026-01-14 12:49:07 UTC] 1.0KiB STANDARD metadata/00000-8bf22118-d262-47a9-986c-721d2b99a846.metadata.json
[2026-01-14 12:49:07 UTC] 1.7KiB STANDARD metadata/00001-5cad6fc5-fe54-41bc-8131-4d6f900d20e7.metadata.json

MinIO alternatives

Let’s now explore the different alternatives to MinIO, and how easy they are to switch MinIO out for.

I’ve taken the above project and tried to implement it with as few changes to use the replacement for MinIO. I’ve left the MinIO S3 client, mc in place since that’s no big deal to replace if you want to rip out MinIO completely (s3cmd, aws CLI, etc etc).

S3Proxy

💾 Example Docker Compose

Version tested: 3.0.0

• ✅ Docker image (5M+ pulls)
• ✅ Licence: Apache 2.0
• ✅ S3 compatibility

Ease of config: 👍👍

Very easy to implement, and seems like a nice lightweight option.

One thing I did notice was that one of the projects that S3Proxy uses, jclouds, was moved to the Apache Attic (i.e. retired) in mid-2025—although probably nbd if you’re only using local storage anyway?

RustFS

💾 Example Docker Compose

Version tested: 1.0.0-alpha.79

Ease of config: ✅✅

• ✅ Docker image (100k+ pulls)
• ✅ Licence: Apache 2.0
• ✅ S3 compatibility

Be aware that there was recently a pretty bad security vuln found in RustFS, which has put some people off from using it. The website looks pretty smart but several links resolve to the same page, giving it that "fresh paint" smell of a new project :) This might matter less for demos, if it’s easy to switch out. You’ll also note that the project is currently only 'alpha' release.

RustFS also includes a GUI:

SeaweedFS

💾 Example Docker Compose

Version tested: 4.06

• ✅ Docker image (5M+ pulls)
• ✅ Licence: Apache 2.0
• ✅ S3 compatibility

Ease of config: 👍

This quickstart is useful for getting bare-minimum S3 functionality working. (That said, I still just got Claude to do the implementation…). Overall there’s not too much to change here; a fairly straightforward switchout of Docker images, but the auth does need its own config file (which as with Garage, I inlined in the Docker Compose).

Edit: Straight after posting this blog, the project replied to say they’ll be removing this extra requirement, making it even easier to use! How cool is that :)

This is completed in https://t.co/RQv387bbKb and will be included in the weekly release.

— SeaweedFS (@SeaweedFS) January 14, 2026

SeaweedFS comes with its own basic UI which is handy:

The SeaweedFS website is surprisingly sparse and at a glance you’d be forgiven for missing that it’s an OSS project, since there’s a "pricing" option and the title of the front page is "SeaweedFS Enterprise" (and no GitHub link that I could find!). But an OSS project it is, and a long-established one: SeaweedFS has been around with S3 support since its 0.91 release in 2018. You can also learn more about SeaweedFS from these slides, including a comparison chart with MinIO.

Zenko CloudServer

💾 Example Docker Compose

Version tested: 9.2.8

• ✅ Docker image (also outdated ones on Docker Hub with 5M+ pulls)
• ✅ Licence: Apache 2.0
• ✅ S3 compatibility

Ease of config: 👍

cloudserver.excalidraw

Formerly known as S3 Server, CloudServer is part of a toolset called Zenko, published by Scality. It drops in to replace MinIO pretty easily, but I did find it slightly tricky at first to disentangle the set of names (cloudserver/zenko/scality) and what the actual software I needed to run was. There’s also a slightly odd feel that the docs link to an outdated Docker image.

Garage

💾 Example Docker Compose

Ease of config: 😵

Version tested: 1.0.0

• ✅ Docker image (1M+ pulls)
• ✅ Licence: AGPL
• ✅ S3 compatibility

I had to get a friend to help me with this one. As well as the garage container, I needed another to do the initial configuration, as well as a TOML config file which I’ve inlined in the Docker Compose to keep things concise.

garage.excalidraw

Could I have sat down and RTFM’d to figure it out myself? Yes. Do I have better things to do with my time? Also, yes.

So, Garage does work, but gosh…it is not just a drop-in replacement in terms of code changes. It requires different plumbing for initialisation, and it’s not simple at that either. A simple example: The specified key ID is not a valid Garage key ID (starts with GK, followed by 12 hex-encoded bytes). Excellent for production hygiene…overkill for local demos, and in fact somewhat of a hindrance TBH.

Is this an entirely fair assessment? If I were looking at it as a new piece of technology in its own right, completely not! Many pieces of excellent technology, particularly those that can support running distributed, will have a steep learning curve for configuration. However, my requirement here is a simple drop-in replacement for MinIO—which Garage is not.

Apache Ozone

💾 Example Docker Compose

Version tested: 2.1.0

• ✅ Docker images (1M+ pulls)
• ✅ Licence: Apache 2.0
• ✅ S3 compatibility

Ozone was spun out of Apache Hadoop (remember that?) in 2020, having been initially created as part of the HDFS project back in 2015.

Ease of config: 😵

apacheozone.excalidraw

It does work as a replacement for MinIO, but it is not a lightweight alternative; neither I nor Claude could figure out how to deploy it with any fewer than four nodes. It gives heavy Hadoop vibes, and I wouldn’t be rushing to adopt it for my use case here.

Ceph Object Gateway

I took one look at the installation instructions and noped right out of this one!

Ozone (above) is heavyweight enough; I’m sure both are great at what they do, but they are not a lightweight container to slot into my Docker Compose stack for local demos.

Comparison

Everyone loves a bake-off chart, right?

Name Ease of config Licence Commercial Backing & Governance Docker pulls1 GH Stars Date of first commit Significant Committers

gaul/s3proxy

(Git repo)

👍👍

Apache 2.0

Single contributor (Andrew Gaul)

5M+

2.1k

2014

1

RustFS

(Git repo)

👍👍

Apache 2.0

Fancy website but not much detail about the company

100k+

19.7k

2024

~4

SeaweedFS

(Git repo)

👍

Apache 2.0

Single contributor (Chris Lu), Enterprise option available

5M+

29.5k

2012

1

Zenko CloudServer (Git repo)

👍

Apache 2.0

Scality (commercial company)

5M+ (outdated version)

1.9k

2015

~10

Garage

(Git repo)

😬

AGPL

NGI/NLnet grants

1M+

2.5k

2020

~4

Apache Ozone

(Git repo)

lol

Apache 2.0

Apache Software Foundation

1M+

1.1k

2018

30+

1 Docker pulls is a useful signal but not an absolute one given that a small number of downstream projects using the image in a frequently-run CI/CD pipeline could easily distort this figure.

Summary

I got side-tracked into writing this blog because I wanted to update a demo in which currently MinIO was used. So, having tried them out, which of the options will I actually use?

• SeaweedFS - yes.
• S3Proxy - yes.
• RustFS - maybe, but very new project & alpha release.
• CloudServer - yes, maybe? Honestly, put off by it being part of a suite and worrying I’d need to understand other bits of it to use it—probably unfounded though.
• Garage - no, config too complex for what I need.
• Apache Ozone - lol no.

I mean to cast no shade on those options against which I’ve not recorded a yes; they’re probably excellent projects, but just not focussed on my primary use case (simple & easy to configure single-node local S3).

A few parting considerations to bear in mind when choosing a replacement for MinIO:

• Governance. Whilst all the projects are OSS, only Ozone is owned by a foundation (ASF). All the others could, in theory, change their licence at the drop of a hat (just like MinIO did).
• Community health. What’s the "bus factor"? A couple of the projects above have a very long and healthy history—but from a single contributor. If they were to abandon the project, would someone in the community fork and continue to actively develop it?

Addendum

• 2026-01-30: Justin Cormack has written about some of the implementation details and functionality of RustFS and Garage
• 2026-03-02: Ruohang Feng has forked MinIO to pgsty/minio and is promising to maintain a stable, CVE-patched, distribution.

Also known as: we have the Clicks Communicator at home.

The thing

The thing

The motivation.

I don’t organize my things well. I try to, but quite often end up with a bunch of stuff on my desk. From various aliexpress orders and projects I’m working on, all the way to gifts and stuff I didn’t have time to find a place for.

That is exactly how this project came to be. At one point in time I had, lying on my desk:

• Various, hopefully openstick compatible, models - namely the:

  • MF800 - one I ended up using
  • UZ801 - OK size, but no battery or sufficienet visible GPIO
  • USB drive form factor one I ditched because of other issues

• The Clicks Keyboard for iPhone 16 Pro Max (gift from my cousin - too bad I don’t have an appropriate iPhone 🥲)

• The Adafruit SHARP Memory Display board

And of course, my caveman brain combined the 3.

Okay, okay, not to lie I was probably a bit conditioned by knowing about:

• Beepy by sqfmi
• Playdate

The modem.

Due to the mysterious laws of supply and demand, and the magic of supply chains - somehow you can get a 4G modem with WiFi, bluetooth, a display, fully battery powered and completely unlocked - for less than $20 shipped. This, of course, is the cornerstone of our project.

There are versions with and without a display. The non-display one just swaps the display for LED-s, though the PCB is the same. Display is a GC9107 powered one, but it looks like ass so i ditched it.

Linux can be installed trivially, powered by the wonderful openstick project. The stock device runs Android, but adb is accessible out of the box - and from adb you can go straight to edl and reflash the thing. Just make sure to save all the important partitions. There are also pins on the pcb which you can short out to get straight to edl.

Extracting the running device tree from the running android was a goldmine of information so be sure to do that.

Here are a few guides or links I found helpful:

• OpenStick github
• openstick.de
• wvthoog.nl blog - I think I used this kernel for starters
• extowerk.com blog
• This project’s GitHub repo might also help you

The real pain with openstick starts once you get to the drivers and device trees, but we will talk about that later.

The keyboard.

There is not much to say about the Clicks Keyboard. It feels veeery nice to use, the only issue is you need to fork over quite a few clams - made worse because you are going to have to cut it 😱.

Regarding the protocol, it’s exactly what I expected with my previous experience of working with MFi devices - just a regular USB keyboard with an additional Apple proprietary endpoint which the iPhone can authorize with before allowing the keyboard to go through.

So for our device this means that it’s just a regular keyboard.

On a side note, there is a Clicks mobile app used for configuration/updates/whatever. The keyboard itself is powered by CH32V203 or similar. Custom code could be flashed but I don’t see any reason to to this currently. In the future I would like to have a configuration utility.

The display.

I think the sharp display look great with the high contrast (and it plays well into my use case of a dumb device used only for messaging). Other than that, if there wasn’t much to say about the keyboard - then there is absolutely nothing more to say here.

It’s a display.

You send commands.

It displays.

The adapter PCB.

One thing became clear to me quite fast - I was going to have to add a custom PCB. I wasn’t exactly sure what the MF800 had on-board, and I never did end up opening the shield can - but I am fairly certain there is no 5V booster on-board.

Because of the physical sizes, which we will go over further in the post, the USB connector will end up chopped off - so we need a way to handle that too.

The final PCB ended up handling the USB host mode power, USB host/device mode switching, display power and display signal level conversion.

PCB from the component side

PCB from the component side

I ordered the PCB along with assembly. And because 2 sided assembly is expensive, I made a few compromises to fit all the components on one side. The non-component side is used for the solder points for the PCB-to-PCB connections.

All files can be found on GitHub, but in short the PCB consists of:

• TUSB320 - for the USB mode switching
• SN74LVC8T245 - for the level shifting
• MCP1640 - the 5V booster
• TPS22917 (one high, one low) - for swithing VBUS/VBAT
• USB connector and FPC connector for the display
• test pads used to connect the adapter PCB to the MF800

Enclosure 1.

The MF800 is quite a bit bigger than other opensticks, partly because of the battery it has to include, partly because of it’s slop nature.

MF800 without the back cover and with the (unnecessary) cut-out for the bootloader pins

MF800 without the back cover and with the (unnecessary) cut-out for the bootloader pins

So to fit inside a Clicks case, we either orient it verticaly and and up with a humongous abomination, or we trim the pcb to fit horizontally.

Top, with a the blue lines showing the USB data lines going from the connector and pads to the SOC

Top, with a the blue lines showing the USB data lines going from the connector and pads to the SOC

Bottom, red lines mark where I was planning to cut and the blue circle marks the via for USB data test pads

Bottom, red lines mark where I was planning to cut and the blue circle marks the via for USB data test pads

Notice that my right cut (on the bottom picture) cuts off the battery connection line, so this will have to be patched up later.

Cutting.

Cutting the PCB went unexpectedly well. The device booted up immediately and everything seemed to work. Turns out there really were no crucial lines going through those areas of the PCB.

The cut PCB inside a test enclosure with the battery below and the patch VBAT wire

The cut PCB inside a test enclosure with the battery below and the patch VBAT wire

Only things I didn’t test were the USB connection and the 4G modem. The modem I was 99% sure wouldn’t be an issue as there is no reason to route anything for it under those areas - but regarding the USB I was worried that I hadn’t maybe nicked the via.

2D scan of the PCB, trimmed to the cut lines and extruded to match the measured thickness - a tight fit within the iPhone’s width

2D scan of the PCB, trimmed to the cut lines and extruded to match the measured thickness - a tight fit within the iPhone’s width

Wiring 1.

I decided to reuse the pads from the old display. I don’t really know why anymore - possibly because my initial ideas was to use a rigid flex PCB and solder it similar to how the original display was. (which I decided against immediately upon seeing the prices)

Looking at all this now, it seems very dumb. I should have used the labeled pads next to the unpopulated micro SD connector. Note that never did check if these were shared with the SIM card though.

Since I could boot the device and I had the original android device trees, I extracted which pins were used for the display SPI. I then tested those with gpioset to make sure that I was indeed correct. Same goes for the power supplies (although I tested those by disabling them in the device tree and rebooting) and grounds.

Checking and mapping the pins with a multimeter

Checking and mapping the pins with a multimeter

The first thing I wired and checked were the power supplies followed by the USB. This is because I could test this as an isolated unit. I was also more skeptical about this because it involved a bit of circuitry on my part as well as that iffy via.

Wiring the USB and the power supplies

Wiring the USB and the power supplies

Of course, it didn’t work initially. After probing around the pads and seeing that all the voltages were OK I noticed in my laptop’s dmesg that it TRIED to enumerate - meaning something was going on.

I also noticed that it says "high-speed". This caught me a bit off guard. I didn’t expect it to use "high-speed" USB. My first thought was that the wires were too long. But before shortening them, I tried a quick fix - twisting them more tightly - and it worked 😲!

Device’s USB Gadget enumerating

Device’s USB Gadget enumerating

Immediately following this success, I tried to get the other direction working. This took some time. Turns out, not all aliexpress adapters correctly wire the CC lines. The keyboard did work immediately, though - only issue is I was afraid to test with it first in case something was wired incorrectly.

Events from the keyboard

Events from the keyboard

Wiring 2.

Before wiring the SPI lines to the display, I wanted to check if I had correctly reconfigured my device tree. I knew the pads were correct from the earlier testing, but there is quite a lot which can go wrong here.

spi@78b9000 {
    compatible = "qcom,spi-qup-v2.2.1";
    reg = <0x78b9000 0x500>;
    interrupts = <0x00 0x63 0x04>;
    clocks = <0x13 0x41 0x13 0x36>;
    clock-names = "core", "iface";
    dmas = <0x6d 0x0c 0x6d 0x0d>;
    dma-names = "tx", "rx";
    pinctrl-names = "default", "sleep";
    pinctrl-0 = <0x84>;
    pinctrl-1 = <0x85>;
    #address-cells = <0x01>;
    #size-cells = <0x00>;
    status = "okay";
    spidev@0 {
        //compatible = "linux,spidev"; 
        compatible = "rohm,dh2228fv";
        reg = <0>;
        spi-max-frequency = <16000000>;
        spi-cs-high;
    };
};

For example, qualcomm drivers are sketchy and the commented out compatible won’t actually export the spidev, so we scam it with this the dh2228fv compatibility.

Using the spi-pipe utility running in a loop, I was able to measure voltage change on the MOSI and CLK lines - which was enough for me to conclude that something was happening. I would, of course, prefer to do this with a scope or a logic analyzer - but I didn’t have any of those at hand.

Encouraged by the major success of power supplies and USB, I carelessly connected the display into the PCB (while the device was on 🤦).

Immediately something happened to the display and I was sure I broke it. Fortunately nothing came of this and the display was fine. (This actually happens every time I turn on the device. I don’t yet know if I should be concerned 😬.)

Once I reassured myself that nothing bad had happened, and that nothing was smoking or overheating - I proceeded with trying to get the display to work.

After an hour of slopping through this with AI python code I was absolutely nowhere. There were multiple possible points of failure. Level converter, bad routing, contacts, etc…

Turns out, as is quite common (IDK why), the qualcomm driver doesn’t handle the CS well (or correctly - or maybe it does but for other use cases). In any case, I tried again the same test script, but this time toggling the CS pin manually (via libgpiod) - and it actually worked.

Display showing a checkerboard test pattern

Display showing a checkerboard test pattern

Rewiring.

I was immediately dissapointed with the everything. From the "electrical" wire I used to connect the power supplies to the sketchy tiny magnet wire I used for the signals all the way to the twisted ground I wrapped around MOSI and CLK (which seemed to do nothing) - so I decided to rewire everything once more.

This time i used magnet wire for both signals and power, but this one was quite a bit thicker and also kept position once bent. I didn’t rewire the USB data lines though, as the pads on the main board look very iffy.

Much better, though still lacking solder mask/resin and tape

Much better, though still lacking solder mask/resin and tape

Display kernel driver.

All the previous tests were done with just dumb python scripts, but the real way forward is with a kernel driver. There are quite a few drivers available, but the one I picked is ardangelo’s sharp-drm-driver. My reasons for wanting a DRM driver are as follows:

• allows me to use a direct output for eg. playing videos (like with mpv)

  • this would work with a framebuffer as well, but that is sketchy in 2026

• i can run X/wayland on it easily

  • maybe try and get into a desktop environment for the lols

• extend the driver to do partial updates

• still get the framebuffer interface

This worked almost immediately. I did have to playing around with CS and it’s active high default.

For probably the first time in my life I didn’t have any issues compiling the kernel module and running it. The mystery kernel I was running was a 6.12.1-msm8916 one with modules enabled. It had a .config file present which I took.

Next I downloaded the mainline linux 6.12.1 kernel and hoped that there weren’t any (or significant) changes. This ended up being enough, and after a few small patches to the driver the thing just worked.

Below is what the device tree ended up looking like. Notice that the CS logic being handled by the display driver.

spi@78b9000 {
    compatible = "qcom,spi-qup-v2.2.1";
    reg = <0x78b9000 0x500>;
    interrupts = <0x00 0x63 0x04>;
    clocks = <0x13 0x41 0x13 0x36>;
    clock-names = "core", "iface";
    dmas = <0x6d 0x0c 0x6d 0x0d>;
    dma-names = "tx", "rx";
    pinctrl-names = "default", "sleep";
    pinctrl-0 = <0x84>;
    pinctrl-1 = <0x85>;
    #address-cells = <0x01>;
    #size-cells = <0x00>;
    status = "okay";
    
    sharp_drm@0 {
        compatible = "sharp-drm";
        reg = <0>;
        spi-max-frequency = <4000000>;
        cs-gpios = <0x49 18 1>;
    };
};

Standard linux console login prompt showed up

Standard linux console login prompt showed up

Issue now is that the driver just rounds pixel color under some value to black, above that to white (or inverted, depends on parameters). For text (or if you do the visuals yourself in your app) this works great - but for a general purpose solution where you want to play videos or show images - this looks like ass. The fix for this is to add dithering to the driver.

A video looking bad with just color rounding

A video looking bad with just color rounding

I added a custom sys value which allows the user to enable dithering, as well as to pick which algorithm they want:

• Atkinson for video
• Floyd-Steinberg for stills

All the initial functionality was left intact.

Big buck bunny looking great (played by stock MPV with DRM output)

Big buck bunny looking great (played by stock MPV with DRM output)

You can find more information about this, or the DRM driver patches on the project’s GitHub repo.

Enclosure 2.

With everything on my table and working, mainly meaning the dimensions are set and can be measured, I jumped into modeling the near-final case which I can hopefully put into the keyboard case without worrying about breaking anything.

This was kinda sketchy since apple doesn’t give dimensions of how far the USB-C connector is inside the iPhone - but I got around this by measuring apple standard USB-C cables (which fit snug up to the device) and interpolating from there.

Linux login, working out-of-the-box after setting USB to host

Linux login, working out-of-the-box after setting USB to host

This print still wasn’t particullarly useful but served it’s purpose to confirm that the USB connector dimensions (among others) were measured correctly.

Also, 2 sidewalls didn’t print correctly and while modelling (this was before I receive’d one visible in the pictures) I didn’t have a display (except the one bonded to the devkit PCB) to model off of, so the cover is lacking.

Cover fits but no display slot, case still not trimmed

Cover fits but no display slot, case still not trimmed

Final enclosure (for now).

This is what ended up being the final enclosure. Mostly everything fit correctly. I first whipped up a quick test held together by kapton tape.

Final encloure, in the still-not-trimmed case held with kapton tape

Final encloure, in the still-not-trimmed case held with kapton tape

This was the point at which, becuase of some ongoing life stuff, I temporarily lost access to a big chunk of my tools (mainly the 3D printer but also other stuff).

My hand being forced, I decided to hot glue the case together instead of printing a final final one which clips together.

I also forgot about the power button, which despite being on the PCB and working - didn’t get a case cutout and a plunger. This was solved with a small hole and a pin. Very ugly but it works.

Hot glued enclosure

Hot glued enclosure

Now the big boy moment - cutting down the keyboard case with no tools. It went about as well as you can expect. Though I made sure to cut less than needed so that I can sand it down and make it look pretty once I get my gear back.

It realistically doesn’t look that bad, but the edges could use some cleaning. The case hot glue protrusion is a bigger issue.

With the magic of top-down photos I have hidden most of this from you.

End result

End result

I forgot to take photos while assembling this. It’s exactly the same as before plus a 4G flex PCB antenna which I soldered to the PCB and glued below the display on the top half of the enclosure. There is now also a mini SIM in it’s slot.

Battery configuration.

The android device trees I copied from the device come predefined with the battery and charger configurations. These are also more advanced than the ones offered by the mainline linux i’m running.

Still, I expected it to be pretty easy to get something usable working. Big questions here were the battery information and power draw of my adapter PCB.

What the driver provided on the user level, however, was only the battery voltage in uV and a flag whether it’s charging or not. So the actual battery logic will be left up to my app as I’m not planning to mod the driver just for the battery percent value.

Charging seems to work fine. It also works via the keyboard USB passthrough port, but unfortunately only when the device is booted up.

The missing.

Sleep currently stands as the biggest non-solved issue. Main reason is the lack of day-to-day testing of the device, especially with the modem turned on - and the lack of a convenient power button.

I’m planning to tackle this in the near future as I begin using the device for my messaging. I would like to get a fast bootup/shutdown going on at the very least.

Additional input methods, eg. a touch screen or a scroll wheel, would probably be the best additional feature. Touch, especially, can be done with very little space.

Those are followed closely by sound or vibration. Even a tiny speaker at like 8khz. There is sufficient PCB space for an amplifier as well as space for the speaker in the enclosure.

The ugly.

Mainly the glue issue and the missing power button, both of which require a new print, plus the jagged edges on the keyboard case that need filing down.

The battery is held down by a bit of tape as it otherwise falls out when not in the case. Not a priority at the moment.

The big bottom bezel driving the enclosure height could also be shortened, but that would require sourcing a different battery with the same 3 pin connector among other things.

The helper PCB slides up inside it’s slot because the display FPC cable slightly pulls on it and I forgot to add tabs in the enclosure cover to keep it in place. Not ideal - but it’s only an issue when sliding the enclosure into the case.

Finally, a tiny portion of the display is covered by the case. Like 1-2 pixels on all edges. This will also be fixed with the next print.

All in all I’m very happy with the device, but a bit more work would do wonders for the visuals. In photos it looks fine - in real life it leaves a little to be desired.

If you are interested in replicating this or doing something similar, you can find most of the stuff on the project’s GitHub repo.

Future.

I have deliberately omitted software from this post since that will only get ironed out with use, and I’m not a big fan of releasing projects I haven’t finished but didn’t drop.

Quick preview of the software

Quick preview of the software

As of writing this I already found a memory leak in the original display driver. I also shipped a patch which fixes it. Stuff like this can’t easily be found without actual hands-on testing.

The text was fully written by me, a human.
You can contact me at veggie_privacy_8y at icloud dot com

A 4,400-year-old Saqqara tomb belonging to Egyptian judge Younmin and his father Ity has been uncovered with colored reliefs and a false door. Credit: Egypt’s Ministry of Tourism and Antiquities

Archaeologists working at Saqqara have uncovered a roughly 4,400-year-old mastaba tomb belonging to a high-ranking Egyptian official and his father, revealing finely carved agricultural scenes and offering processions that still preserve traces of their original colors.

The tomb dates to the late Fifth Dynasty, during the reign of Pharaoh Djedkare Isesi, and was discovered during the 2026 excavation season in the Mariette Necropolis, an extensive burial ground north of Saqqara that has been known to archaeologists since the 19th century but remains far from fully understood. Egypt’s Ministry of Tourism and Antiquities announced the discovery on September 7.

The newly investigated monument is a medium-sized mastaba containing two funerary chapels. The larger northern chapel belonged to an official named Younmin, while the southern chapel was constructed for his father, Ity.

But the inscriptions inside make clear that this was not an ordinary family burial.

A man who handled complaints at the heart of the Egyptian state

According to the Ministry, Younmin held several important administrative positions. He served as a judge and director of the court, as well as head of the scribes responsible for examining petitions and complaints.

Those titles provide an unusually direct glimpse into the administrative machinery of Old Kingdom Egypt. Rather than simply recording prestige, they suggest that Younmin participated in the system through which disputes, petitions, and official requests reached the upper levels of the state.

His father Ity also belonged to the scribal administration, holding positions connected with the management of agricultural land and offerings.

The combination is significant. In a society whose wealth and taxation depended heavily on agricultural production, the supervision of land, written records, and offerings was closely tied to the functioning of both royal administration and funerary institutions.

Djedkare Isesi himself ruled near the end of the Fifth Dynasty and was buried in a pyramid complex in southern Saqqara. Documents surviving from his reign include royal communications with powerful officials, providing evidence for a highly developed written administration operating around the court.

Archaeologists working at Saqqara have uncovered a roughly 4,400-year-old mastaba tomb belonging to a high-ranking Egyptian official and his father. Credit: Egypt’s Ministry of Tourism and Antiquities

Painted fields and processions survived for more than four millennia

The most visually striking material was found in Younmin’s chapel. Excavators uncovered an architectural façade carrying funerary texts and an offering formula, together with a formal representation of the tomb owner. The surrounding walls were decorated with high-quality reliefs showing agricultural activities and processions bringing offerings into the tomb.

Remarkably, many of these reliefs retain their original colors.

Such scenes were not simply illustrations of everyday life. Images of crops, animals, food production and offering bearers formed part of a funerary system intended to guarantee that the deceased would continue to receive provisions in the afterlife. Similar scenes occur throughout elite Old Kingdom mastabas at Saqqara, where everyday economic activity was transformed into a permanent source of ritual abundance.

The southern chapel belonging to Ity is less completely preserved, but archaeologists identified surviving colored decoration depicting offerings as well as a false door.

Despite its name, the false door was never intended to be opened. In Old Kingdom tomb chapels it represented a symbolic threshold between the world of the living and the deceased. Offerings could be placed before it, allowing the dead to symbolically emerge and receive them.

Archaeologists working at Saqqara have uncovered a roughly 4,400-year-old mastaba tomb belonging to a high-ranking Egyptian official and his father. Credit: Egypt’s Ministry of Tourism and Antiquities

A cemetery first explored by Auguste Mariette

The discovery’s location adds another layer to the story. The Mariette Cemetery takes its modern name from French Egyptologist Auguste Mariette, who excavated the broad field of mastabas northwest of Djoser’s Step Pyramid between 1860 and 1863.

Research presented by mission director Josep Cervelló Autuori of the Autonomous University of Barcelona shows just how uneven the archaeological history of this part of Saqqara has been. After Mariette, only limited work was conducted there by Margaret Alice Murray in 1903–1904 and William Stevenson Smith in the 1930s. Large-scale modern investigation did not follow for decades.

A Czech archaeological mission began working in the western part of the area in 2022, while the joint Spanish-Egyptian mission started investigating its eastern sector in 2023.

This background helps explain an unusual detail in Egypt’s announcement: the mastaba itself was not entirely unknown. Its position had been recognized since Mariette’s 19th-century work, but according to the Supreme Council of Antiquities, it had never previously been excavated and documented using a comprehensive modern archaeological methodology.

The distinction is important. Archaeological maps produced more than a century ago can record the existence of a structure without revealing its full architecture, inscriptions, chronology or ownership. Returning to such sites with modern excavation and documentation methods can therefore produce discoveries hidden in places archaeologists have technically “known” for generations.

The surrounding walls were decorated with high-quality reliefs showing agricultural activities and processions bringing offerings into the tomb. Credit: Egypt’s Ministry of Tourism and Antiquities

Saqqara is still revealing the society behind the pyramids

Saqqara served as one of the principal cemeteries of Memphis, the ancient Egyptian capital, and contains monuments spanning much of Egyptian history. Located about 40 kilometers southwest of Cairo, the necropolis includes Djoser’s Step Pyramid as well as royal pyramids and thousands of tombs belonging to officials, priests, and members of the elite.

The newly excavated mastaba belongs to precisely this world of officials who surrounded the royal court.

While pyramids record the power of kings, tombs such as Younmin and Ity’s preserve something different: the people who administered land, received petitions, kept records and managed the institutions on which the Egyptian state depended.

The Spanish-Egyptian team has now completed the excavation and scientific documentation of both chapels. Conservators have also carried out work on the architecture, reliefs and surviving pigments to stabilize the newly exposed remains.

For a tomb whose location had lingered on archaeological records since the age of Mariette, its walls are finally beginning to reveal who was buried there—and how two generations of one family served the Egyptian administration more than four millennia ago.

Taste is subjective. But when it comes to architecture, there is a surprising level of agreement.

In the twentieth century, a radical change took place in global architecture. The modernist movement emerged in the 1920s, favoring unornamented designs, exposed modern materials like steel and concrete, and the abandonment of the complicated visual patterns of traditional planning and facade design. Modernism came to dominate global architecture by the 1950s, and although it has evolved and fragmented since then, styles of broadly modernist character remain dominant today.

Throughout its history, both supporters and detractors of modernist architecture have agreed that it is not widely liked. In 1940, the English architectural historian John Summerson remarked that the public ‘can’t understand it, never will, and hates it like poison’. In 1977, the German-American architectural critic Wolf von Eckardt prematurely announced the death of modernism, claiming that ‘No one mourned. It was never popular.’ The young Summerson was sympathetic to modernism while von Eckardt disliked it, but they made the same observation about its standing with the general public.

Were Summerson and von Eckardt right? In recent years, researchers have begun to study this more systematically using visual preference surveys. In their simplest form, these are just a comparison of two images alongside a neutral, unbiased question, like ‘Which do you prefer?’ Online panels and better image editing have brought down their cost in recent decades. Since the 1990s, around twenty properly sampled visual preference surveys have been run on popular architectural preferences.

Many are British, partly because the King’s championship of traditional urbanism has generated a particularly lively debate about architecture in Britain. However, visual preference surveys have also been run in the United States, Canada, the Netherlands, Portugal and Chile, and enthusiasm for visual preference surveying is rising internationally.

Early visual preference surveys were often methodologically primitive. A common limitation was that they surveyed responses to images that varied in many respects, such as the angle of the photograph or the weather, making it impossible to claim that architectural style alone had caused any difference in ratings. Over time, however, much has been done to mitigate this, with progressively better editing techniques being used to create more controlled image pairs. In the past few years, artificial intelligence has greatly reduced the time and costs of producing comparable images, making reliable large-scale visual preference surveys far more feasible.

In every survey conducted, over 60 percent of the respondents have favored traditional architecture, with many revealing over 85 percent support. This preference is surprisingly unaffected by age, gender, politics, socioeconomic class and nationality. The results are not definitive, but they suggest that there is substantial public discontent with the dominant trends in modern architecture. Perhaps the anecdotal impressions of Summerson and von Eckardt were not so far off after all.

The birth of visual preference surveying

The story of visual preference surveying begins in 1979, in the small town of Metuchen, New Jersey. Metuchen decided that it wanted data on local opinion to guide its design policies, and so it commissioned a study from the Rutgers University Urban Design Studio. The Rutgers team took hundreds of photographs of streets and buildings in Metuchen, and asked locals to rate them at a large public meeting. Metuchen residents overwhelmingly preferred streetscapes with a traditional, small-town feel, detailed and varied facades, and a walkable, human scale. They disliked blank, featureless buildings, and wide, car-dominated streets.

These preferences may seem obvious. Of course a traditional high street is more pleasant to walk down than a freeway with six lanes of traffic. But at the time, the level of consensus was surprising. The Metuchen survey was the first to show that people tend to have predictable, shared responses to different built environments, responses that are not purely a matter of individual taste. The research team used the results to make recommendations for streetscape improvements, facade renovations, and infill multi-family housing, and Metuchen later developed a design code on this basis.

The modern tradition of visual preference surveying begins in Metuchen in 1979. These streets were scored -4.19, -1.38 and +5.07. Source: Anton C Nelessen.

The Metuchen survey was pioneering, but it had some obvious limitations. First, participants in public meetings are not a representative sample of the local population. Second, the images used in the survey did not offer a perfectly controlled comparison, making it difficult to know what exactly respondents were reacting to. Did people dislike the width of the street, the low ratio of building height to street width, the style of the facades, the absence of greenery, the presence of fast-moving cars, or something else? We cannot know based on their responses to any individual image, although the survey included enough images that robust patterns do start to emerge.

Britain’s cottage industry in architectural visual preference surveying took off in the late 1980s, beginning with a 1987 study by a Cambridge undergraduate called David Halpern, who later became a leading behavioral psychologist. Halpern showed volunteer students photographs of twelve people and twelve buildings, asking them to rate their attractiveness. Students on different programs had similar views on which faces were the most attractive. But when it came to buildings, architecture students held wildly different views from those in other disciplines. In fact, their preferences were often directly opposite.

This building was ranked lowest out of twelve by non-architecture students in Halpern’s study, but it was the second-favorite of architecture students. Source: David Halpern.

Halpern’s study had obvious limitations: the sample was unrepresentative of the general population and small, and the images were imperfectly controlled. But it has become a minor classic, and studies have since been run in Canada in 2001, Britain in 2015, and Chile in 2019 with similar results, showing a wide divergence between professional and lay opinion when it comes to the visual appeal of different architectural styles.

The first properly sampled visual preference survey followed in 1998, run by the British Market Research Bureau for an industry group called the Popular Housing Forum. British respondents were presented with card sets showing newbuild houses of a given style. As before, the pictures were imperfectly controlled, although the use of multiple pictures in each group compensated for this to some extent. 94 percent of potential newbuild buyers said they would prefer to live in one of the more traditional options, and 92 percent of other respondents said they would prefer one of the more traditional options to be built near them.

This early survey presented respondents with ‘card sets’ of newbuild house pictures and asked them to choose between them, rather than between individual images. From left to right, the sets above were favoured by 2 percent, 37 percent, 28 percent and 2 percent of potential newbuild buyers. Source: Robert Adam.

In the following decades, several more visual preference surveys were commissioned by academic researchers, architecture firms, think tanks, and public bodies. Most of these studies were conducted by professional polling companies, so their respondents will have been largely representative of the general population. However, nearly all of them still used imperfectly controlled image pairs, meaning that it was hard to be sure that responses were caused by differences in architectural style rather than some other difference between the images. And the original images used in some older studies have been lost, making it hard to rest too much weight on them.

However, to the extent that their findings are reliable, they show that while a minority of the public enjoys boundary-pushing modern architecture, the majority prefer more traditional buildings. And different groups respond to architectural styles in reasonably consistent patterns: where demographic data have been collected, it seems there is a weak tendency for men and younger people to be more open to modernist architecture, compared with women and older people. As far as we can tell from these surveys, though, all demographic groups prefer traditional styles: the preference is simply more pronounced in some groups than others. This holds not only for houses but also for hospitals and commercial and civic buildings.

Bristol City Hall (left) came first and Shropshire Shirehall (right) last in a survey of ten town halls commissioned by a British think tank. Source: Alamy.

One of the best early visual preference surveys was commissioned by the British architectural practice ADAM Architecture in 2009. Four real commercial buildings were shown in matching perspectives and weather conditions, two in traditional styles, two in modernist ones. Both of the modernist buildings had won prizes, suggesting that they were especially well-designed examples. Even so, only 23 percent of respondents preferred them, compared to 77 percent for the two traditional designs. The images offer the best controlled comparators of any study before the 2020s, with matching perspectives and lighting conditions.

Four images of real commercial buildings used in a survey commissioned by a British architectural practice in 2009. Images two and three were preferred by 77 percent of respondents, compared to 23 percent for images one and four. Source: ADAM Architecture.

David Halpern’s finding that architecture students differ in their tastes from those in other majors was corroborated by later research. Portuguese researcher Luis Balula carried out a visual preference survey in the small city of Évora in 2008–2009. Balula broke down the results into experts (architects, planners, developers, academics) and non-experts. Both groups gave higher ratings to older buildings, but there was more divergence when it came to modern designs. For example, non-experts had a positive view of the pastiche Mediterranean group of homes pictured in figure 7**,** while the experts were generally negative. Inversely, the modernist apartment block was rated negatively by the non-experts, but positively by the experts.

Luis Balula.

Luis Balula surveyed both experts and non-experts, providing valuable evidence on the divergence between lay and professional views. The different environmental conditions in these two images means that it cannot be treated as a straightforward A/B test, but the difference in relative rankings by experts and non-experts remains striking. Source: Luis Balula.

Perhaps the difference between lay and expert taste explains why the public’s preference for traditional architecture has occasionally come as a surprise to researchers. Two visual preference surveys were commissioned in 2002 and 2005 by the Commission for Architecture and the Built Environment (CABE), a British independent public body, and both found low levels of support for modern styles. ‘The Modern Movement’s concern for light and sun and large windows should give a modern style a clear advantage over more traditional designs, but this advantage does not seem to have been understood by home buyers’, the 2005 report regretfully notes.

The golden age of visual preference surveying

While the surveys carried out from the 1970s to the 2010s gave a fairly clear indication that the public consistently prefers traditional architecture, they are not perfect evidence. Imperfectly controlled comparisons and sometimes self-selecting panels mean that their findings can, and often have been, challenged.

In the past five years, however, the evidence has rapidly strengthened. Technological improvements, especially artificial intelligence, have cut the cost of image editing, allowing survey images to be far better controlled. And at the same time, renewed public debate about architecture has driven an increased quantity of visual preference surveys, often led by specialist research groups like Create Streets in Britain or The Aesthetic City in the Netherlands. Something of a golden age of visual preference surveying has emerged, with an increasing number of studies of unprecedentedly high quality.

Perhaps the most famous visual preference survey ever conducted was commissioned by the National Civic Art Society in 2020, looking at federal and courthouse buildings in the United States. It asked respondents for their preferences between seven pairs of photos of actual buildings, controlled so far as possible for massing (the form and volume of a building) and context. In every case, huge majorities of respondents preferred the traditional designs, averaging 72 percent across the seven questions.

National Civic Art Society.

The building on the left was preferred by 81 percent of respondents, the building on the right by 19 percent. Source: National Civic Art Society.

As in other studies, most striking is how little the clear preference for traditional designs was affected by demographics. There was almost no correlation with political affiliation, with 70 percent of Democrats and 73 percent of Republicans preferring the traditional designs. Women were somewhat more pro-traditional than men, at 77 percent against 67 percent. 75 percent of white Americans preferred traditional architecture against 65 percent of Hispanic Americans and 62 percent of black Americans. No significant correlation was found with income or educational level, and age was only a weak predictor: 77 percent of over 65s preferred traditional architecture against 68 percent of those aged 18–34.

Many modern visual preference surveys have been carried out in Britain. In 2023, disappointed by the utilitarian design proposal for a new rugby stadium in Bath, England, the small firm Apollodorus Architecture proposed a classical alternative. Create Streets then commissioned a polling company to survey public views on the two designs, holding context and rendering style constant between the image pairs. Respondents preferred the traditional design by 74 percent to 26 percent. There was almost no variation by social class, and only modest variation by political party. Older people were, as in most studies, more traditional than younger people, but even 18–24s still preferred the traditional design by 61 to 39 percent.

Reproduced by permission of Apollodorus Architecture.

Rival designs for a stadium in Bath. The classical scheme received 74 percent support against 26 percent for the modernist one. Source: Reproduced by permission of Apollodorus Architecture.

The same trend shows up in infrastructure. High Speed 2 (HS2) is a railway under construction between London and Birmingham. Create Streets ran a visual preference survey comparing HS2’s planned concrete viaducts with a brick-clad design emulating Britain’s ubiquitous railway viaducts from the nineteenth century. Again, they held context and rendering style constant, so that only building treatment varied between the images. The survey showed a clear preference for the red brick design, 69 versus 28 percent. These results varied only modestly along demographic lines: support for the brick viaduct was slightly higher among women than men (71 versus 66 percent). Anomalously, the survey also found that Labour voters were slightly more pro-traditional than Conservatives (72 against 69 percent) and younger people more than older people (70 percent for the youngest group against 64 percent for the oldest).

Create Streets.

Designs for a viaduct for the High Speed 2 railway. Source: Create Streets.

In 2025, Christ’s College Cambridge announced plans for a new library building designed by Grafton Architects, one of the world’s most prestigious architecture firms. Create Streets prepared a counterproposal with the same floor area, using the Gothic style traditionally favored by Cambridge colleges. In a survey of over 2,000 British adults, 71 percent preferred the traditional design, while just 21 percent favored the original proposal. As with the HS2 survey, this pattern held across all demographic groups with little variation.

Create Streets.

Designs for a new library at Christ’s College, Cambridge. Source: Create Streets.

This preference is seen time and again. The British retailer Marks and Spencer is planning to demolish a 1920s building in central London and replace it with a modern design. Working with the architect Francis Terry, Create Streets proposed an alternative scheme with the same overall floor area, but retaining and extending the existing facade. They then commissioned pollsters to survey 3,130 British adults on images of the two designs, holding the context and rendering style constant. This yielded the strongest difference in preference of any Create Streets study, with 79 percent supporting the alternative proposal, and only 17 percent the original. As usual, demographic breakdowns show little correlation between architectural taste and other characteristics. Londoners were a little more likely to favor Marks and Spencer’s proposal than the national average, but still favored the Terry scheme by 72 against 26 percent.

Create Streets.

Designs for the Marks and Spencer building on Oxford Street, London. Source: Create Streets.

Create Streets recently ran a slightly different kind of poll on a major proposed development in London called Shoreditch Works. Shoreditch Works would involve the demolition of some postwar warehouses, which would be replaced with much larger office blocks in a style drawing on Victorian commercial architecture and Art Deco.

Most of the studies we have looked at aim to control for everything other than architectural style. In this comparison, however, the proposed buildings were drastically taller than the existing ones. The standard view is that the public dislikes high-rise development, meaning that this study could be seen as ‘biased against’ traditional architecture. But in each case, between 76 and 78 percent of respondents preferred the proposed scheme to the existing condition. And, again, support for the proposals was slightly stronger among younger than older respondents.

Create Streets.

In the Shoreditch Works survey, respondents were asked: ‘Here are two alternative designs for the same street in a city centre location. If you had to choose, and all other things being equal, which one of the streets do you prefer?’ Source: Create Streets.

Continental Europeans appear to prefer traditional architecture by similar margins. In 2025, the Dutch group The Aesthetic City ran a visual preference survey on architectural preferences in the Netherlands. It asked respondents for their preferences between eight pairs of images, and showed a preference for traditional buildings among Dutch respondents ranging from 67 to 92 percent, with an average of 82 percent. The Dutch study is notable for showing how useful artificial intelligence is in visual preference surveying. In all but one image pair, they used a photograph of a modernist building and then used AI to create the traditional alternative. This allowed image pairs to be tightly controlled for context, weather, and massing.

The Aesthetic City.

In a survey run in the Netherlands by the Dutch organisation The Aesthetic City, the building on the left received 8 percent support, and the building on the right received 92 percent. Source: The Aesthetic City.

Taking the measure of public taste

The main question of any visual preference survey on architectural style is whether it generalizes. Sceptics often argue that when the public prefers a particular traditional design over a modernist one, it is just because that particular modernist design is weak. A better modernist building, they claim, could have matched or surpassed the popularity of the traditional one. The suspicion of a loaded choice of images is heightened by the fact that most visual preference surveys have been commissioned by organizations sympathetic to traditional architecture, with only a handful of counterexamples like the early CABE studies.

Nevertheless, it is striking that every image pair surveyed so far has generated the same result. Visual preference surveying on architectural style is still in its infancy: perhaps only three dozen fully controlled image pairs have ever been polled on, and these high-quality studies have occurred only in Britain, the Netherlands and the United States. We will be able to draw conclusions with far more confidence in a decade, when we will have tripled or quadrupled the size of our evidence base.

For now, the provisional conclusion is that 70 to 90 percent of the population has a clear preference for traditional architecture. This preference is surprisingly unchanged by other demographic features. Men, younger people, and maybe richer people tend to be slightly more likely to favor modern architecture, but these trends are not marked. Every demographic group favors traditional design: men and women, old and young, rich and poor.

If this is true, a greater puzzle comes into view. Almost every architectural school in the world exclusively teaches broadly modernist styles, and probably more than 99 percent of architects practice exclusively in them. In Britain, there has been hardly a single public commission in a traditional style since the 1950s, and only a trickle of commercial and institutional ones. Except in the market for private housing, traditional styles have been virtually extinct for seven decades. Although in some cases this can be explained by anti-traditional planning rules, the broad trend is still evident in jurisdictions whose development control systems are stylistically neutral.

In other words, there seems to be a mysterious divergence between people’s stated and revealed preferences. Why are we commissioning styles that we say we dislike? There are several interesting possibilities. But investigating them will have to wait for the future.

There are plenty of laws on the books to hold companies, and potentially their execs, accountable

Former FTC chair Lina Khan wants the federal government to know that it doesn't need to wait for new laws to address AI threats. There are already laws and regulations on the books, including a 92-year-old Supreme Court precedent, that she argues could be used to hold AI companies and, in some circumstances, their executives accountable for their actions.

Khan’s comments on X Sunday follow a flurry of activity from the leadership of OpenAI, Anthropic, Microsoft, and xAI aimed at doing what can only be described as trying to corner regulators into giving them their way. The former Biden administration trust buster pointed to numerous examples of current laws, and prior precedent, that could be used to hold frontier labs to account, even if they’re currently doing all in their power to change the conversation.

“We shouldn’t let discussions about new legal regimes distract from the fact that there’s no AI exemption from laws already on the books,” Khan said. “Law enforcers already have authority to charge companies and their CEOs for creating and releasing dangerous, unvetted, or defective products.”

As one example, Khan points to laws governing dangerous and defective products as an avenue to prosecute AI leaders. She notes that the release of unvetted models or agents can violate consumer protection laws, and that shipping tools “without implementing adequate measures to detect and stop rogue or defective AI agents” could be prosecuted under rules governing unfair and deceptive trade practices. Particularly timely, Khan also pointed to existing laws prohibiting unfair methods of competition. This, she notes, includes cases “where firms pursue dangerous behavior, aware that doing so may compel rivals to do the same.”

There’s no leap needed to understand what Khan’s talking about here. OpenAI’s agents broke out of their intended sandbox and gained unauthorized access to Hugging Face systems - conduct that could raise serious criminal-law questions if carried out knowingly by a human. After doing some digging to look at its own agents' behaviors, Anthropic has essentially copped to similar activities that would be criminal if a meatbag was behind the keyboard instead of a simulated silicon brain. OpenAI’s agents have since been identified as the culprits in other misuses of online assets that, again, would be crimes were they perpetrated by a human.

Khan points to a 1934 US Supreme Court decision to argue that the current battle between American frontier labs, which has put parts of the internet in the firing line of agents that escaped their intended constraints, could amount to an unfair method of competition if companies feel compelled to take similar risks to keep up.

That decision, FTC v. R.F. Keppel & Bro, includes a passage where the justices argue that, if keeping up with the competition requires companies to “descend to a practice which they are under a powerful moral compulsion not to adopt,” that competition is unfair whether or not it’s criminal.

Without weighing in on who shot first, OpenAI and Anthropic appear locked in a race to build increasingly capable AI while also warning, as both did over the weekend, that those systems could become dangerous without stronger safeguards and coordinated limits.

Aside from the bad activity of the frontier labs themselves, Khan points out that the “highly concentrated and interconnected structure” of the AI industry also merits scrutiny for its potential to create “major risks and conflicts of interest.”

Again, Khan points out this isn’t a hypothetical.

“OpenAI could face liability given the Hugging Face incident, but Hugging Face being bought up by Nvidia means that we’re unlikely to see it file a lawsuit over this,” Khan noted, “given Nvidia’s strong incentive to see OpenAI continue full speed ahead.”

Nvidia has dumped billions of dollars into OpenAI, becoming a centerpiece of the lab’s datacenters that power ChatGPT. Why, then, would the soon-to-be-owner of Hugging Face opt to hold one of its major partners accountable and further push it to build its own hardware?

“We can and must pursue any new efforts alongside enforcing existing laws,” Khan said. Let’s be frank, though: The current administration is unlikely to do anything except capitulate and allow the AI industry to capture its regulators, if it even bothers to implement new regulations at all.

Trump has already rejected the AI industry’s weekend calls for regulation, declaring himself to be the only guardrail the AI industry needs.

As the AI industry leaders basically admitted over the weekend, whichever one of them blinks first stands to lose, so every single frontier lab in the US is going to keep pushing full steam ahead unless all of them agree to hit the brakes and pace their development. With Trump and other Republican leaders rejecting those calls, Khan’s argument leaves her former agency and other state and federal regulators as potential avenues for action.

Kirk Sigmon, a founding partner at technology law firm KellDann Law, told us that it’s unlikely federal regulators will take any action.

“Most governments are desperate not to kill a nascent technology as it grows, especially when other countries are allowing it to grow,” Sigmon told The Register. He said the only actions against the industry he expects to see in the next few years are “easy wins” in places like deepfake porn, impersonation, and AI-enabled scams. “I very much doubt we'll see much action … against the entire process of training, or the like - that's likely to be perceived as strangling the industry.”

In other words, fire up the boilers - it’s full speed ahead toward the day AI does something truly devastating and we all gnash our teeth and wail about how something should have been done earlier. ®