Reading is important to me. While I’m not a quick reader nor an especially voracious one, I have found that long-form reading has had a profound influence on me over my life.

And because reading is important to me, writing is too: writing not only allows us to convey our ideas, but the very act forces us to test and distill them — at once making our ideas more robust while providing the vehicle by which to share them.

But I am writing this now because — speaking as a reader — we are exasperated: too many people — people we otherwise like and respect! — are writing (or otherwise putting their name to) pieces that are clearly LLM-authored. We readers are left with pointed questions for those promoting LLM-authored pieces: do you think readers can’t tell? Or do you think readers don’t care?

To answer the first question, readers can absolutely tell. To those who read broadly, the hand of the LLM is so clear it’s as if the writer’s intellectual fly is open. In fact, it’s so jarring that I have to believe that those writing with LLMs are either not reading enough to see the LLM’s obvious structural tells — or (and?) they aren’t even reading their own content. (A confession: with particularly egregious pieces, I have fantasized about sentencing the author to read them aloud, certain that they themselves will be unable to endure the slop that they are foisting upon the rest of us.)

As to the second question, readers emphatically care. In fact, the tells of LLM writing are so grating ("and here’s why that framing matters!") that our brains pull an LLM-triggered ejection handle, bailing us out mid-sentence in an act of self-preservation. And the first person plural there is deliberate: as Cynthia Dunlop writes, of the 668 developers that replied to her survey, 78% "stop reading immediately" when they detect an LLM. And there are consequences that outlast the piece: 71% of the respondents in Cynthia’s survey also "avoid the author in the future" (!!). Revealingly, the respondents are not after linguistic perfection, but rather authenticity: 98% reported preferring an author’s own (imperfectly) written piece over an LLM-polished one. Finally, be wary of dismissing Cynthia’s respondents as a self-selecting group: active readers on social media are exactly the folks most likely to repost or otherwise promote writing they like — the early adopters and the tastemakers of online prose.

Why do people have this reaction? Beyond having to endure aggravating stylistic tics, when reading a piece that has had substantial LLM assistance, we — the readers — don’t know what is real and what isn’t. As I wrote in RFD 576, to use an LLM to write is to void the social contract between writer and reader: we readers shouldn’t be expected to labor to understand a sentence that the writer themselves didn’t work to create.

Does the revolt of the reader matter? If it needs to be said, when you are using an LLM to author a public piece, you are no longer writing for yourself or to otherwise pressure-test your own ideas — the only purpose is to serve the reader. If readers choose to ignore you (perhaps forever!), you will have undermined yourself: instead of attracting readers you will be actively repelling them. So other judgement about its use aside, using an LLM will increasingly become simply…​ ineffective.

In this regard, I am reminded of the arc of e-mail spam. There was a time in the early 2000s when people were (reasonably!) afraid that the explosion of spam would mean the end of e-mail. This was an era largely before social networking, and e-mail was the canonical killer app of the Internet; that we were losing e-mail to spam felt deeply dispiriting. But sometime in the late 2000s, we turned a corner: spam filtering improved so much that the economics of spam were undermined. Moreover, as spam became effectively contained, the consequences of being labeled as spam became increasingly dire. Today, legitimate businesses are very careful about how they use bulk e-mail; to be labeled as spam is to effectively destroy your domain name and tarnish your brand.

The war on spam started to turn when we could identify it at scale; could something similar happen to LLM-authored writing? Like spam, an LLM’s influence is readily identifiable to humans reading it; surely this is a solvable problem?

Up until somewhat recently, the results on this problem had been decidedly mixed. I had tried to use LLMs themselves for LLM identification, but I found that their false negative rate was far too high: they were chipper in accepting stuff that I was certain was LLM-authored. Other services seemed to look for basic LLM tells, but as an avid (and unapologetic!) user of the em-dash, these superficial techniques make me shift nervously in my seat (and I found them to be so broadly unreliable that they didn’t earn regular use).

Late last year, Pangram Labs launched their Pangram 3 model. I found it to be a huge leap over other detectors, and became an avid user. Importantly, over months of use (and especially on samples that I otherwise knew the origins of), I found its false positive rate to be very low: when Pangram identified a text as being largely AI written, I could say with some certainty that an LLM was heavily involved. (I found its false negative rate to be higher than I would like, but it was a small price to pay for a low false positive rate.)

A little over a month ago, they introduced Pangram 4, which I found to be a step-function improvement over the already-impressive Pangram 3: in my experience it has an astonishingly low false positive rate and low false negative rate (which is to say: very high accuracy!). I am finding it to be so effective (and the loss of trust in voices that use LLMs to be so precipitous) that I recently extended RFD 576 to be explicit about public writing, specifically mandating that public Oxide writing be reported by Pangram as human-authored. As I explained in the RFD, the standard for our public writing is not merely that LLMs aren’t used to write, but that readers have the confidence that it’s not LLM-authored — and increasingly that means being Pangram-clean. For those organizations that value the authenticity of their institutional voice, I would encourage the adoption of a similar policy for those writing under their banner. (Looking squarely at you, Rust Foundation!)

Indeed, Pangram has become important to so many of us that I was thrilled when Pangram Labs co-founder and CEO Max Spero joined us recently on Oxide and Friends. I would recommend that discussion not only for the technical details of Pangram, but also for Adam’s drop of a pitch-perfect (if underappreciated in the moment) Simpsons reference: when I likened LLM-authored writing to presenting takeout food as one’s own cooking, Adam quipped that such writing is like "steamed hams", making reference to Principal Skinner trying to pass off Krusty Burgers as an old family recipe. Beyond the apropos reference, it was interesting to hear Max’s insight into what has made Pangram 4 especially effective — and why its efficacy seems to evince some suspicious howls of protest.

So, writers beware: readers are in revolt. You should fully expect your writing to be run through Pangram. If your position is that we should be fine with an LLM crafting prose from your prompt, spare us all the wasted cycles and just give us your prompt. Or, better yet, consider doing what generations of writers have done before you, and treating that prompt as a skeleton that you use to write your piece yourself! This doesn’t mean that an LLM can’t help you, of course — LLMs are superlative editors! — just that you probably shouldn’t let it write it for you if you actually expect the rest of us to read it.

I can't play an instrument. I have tried more than once, and each time I got as far as being able to make roughly the right noises without ever understanding why they were the right noises.

The problem was never the practice. It was that every explanation of music theory seemed to miss out the fundamental reasons for how and why things are the way they are. Here is a staff. Here are the notes on it. This is a major scale, memorise the pattern. Why those notes? Why that pattern? Because that is the convention.

Which is a strange way to teach a system that essentially comes out of physics and arithmetic. There are twelve notes for a reason. The major scale has the shape it has for a reason. Chords that sound good sound good for a reason, and you can compute those reasons.

So I wanted to start from scratch and learn music from first principles, and I began that journey by writing code.

This article is the result. It starts with a single number changing over time, and if you follow along, you will derive the twelve notes, build scales and chords out of arrays, and write a chord progression that sounds like actual music. No instrument needed, and nothing you have to take on faith. Written notation does turn up, but not until the very end, once there is something for it to be notation of.

A sound is a number that changes over time

Sound is just air pressure wobbling. A speaker makes sound by pushing its cone in and out, and everything your computer does with audio comes down to producing a list of numbers describing where that cone should be, forty-four thousand times a second.

An audio file is that list written down. A synthesiser makes the list up as it goes, and the browser will do that part for you if you say what shape you want. The simplest shape is a sine wave, so here is one repeating 440 times a second:

A sine wave at 440Hz

const osc = ctx.createOscillator();
osc.frequency.value = 440;

osc.connect(out);
osc.start();
osc.stop(ctx.currentTime + 1);

ctx and out are mine rather than the browser's. Everything else is the Web Audio API exactly as it ships. To run that snippet anywhere else, start with:

const ctx = new AudioContext();
const out = ctx.destination;

The number 440 is the only thing there that carries any musical meaning, and even that is arbitrary. It is the frequency somebody agreed to call "A", and it is the tuning fork the rest of the system is pinned to. Change it to 300 and run it again. It still works, it just plays a different pitch. The oscillator has no idea what a note is. It only knows how many times a second to swing.

Frequency is pitch: higher number, higher note. Nothing else in music is going to be that simple.

Why that note clicked

You may have heard a little click at the end of that. The browser did exactly what it was told. The click is physics.

The oscillator was mid-wave when it stopped, so the speaker cone was somewhere out at the edge of its travel and then instantly snapped back. An instant jump in pressure is what a click is.

The fix is a second number that changes over time, this one controlling volume rather than pitch. Musicians call the shape of it an envelope:

The same note with an envelope

const osc = ctx.createOscillator();
osc.frequency.value = 440;

const env = ctx.createGain();
const t = ctx.currentTime;
env.gain.setValueAtTime(0, t);
env.gain.linearRampToValueAtTime(0.3, t + 0.01);
env.gain.exponentialRampToValueAtTime(0.001, t + 1);

osc.connect(env).connect(out);
osc.start(t);
osc.stop(t + 1);

An envelope is the volume curve of a single note, from silence back to silence. The rise at the front is the attack and the fall afterwards is the decay.

Ten milliseconds to fade in, then a slow decay to nearly nothing. That is the difference between a test tone and something you would be willing to listen to twice.

This is a simplified envelope, though. The full version is ADSR: attack, decay, sustain and release, where sustain is the level a note holds at while a key is down, and release is how it fades once you let go.

The function below is a helper that each subsequent example uses:

function note(freq, start = 0, length = 0.5, type = "sine") {
  const t = ctx.currentTime + start;
  const osc = ctx.createOscillator();
  const env = ctx.createGain();

  osc.type = type;
  osc.frequency.value = freq;

  env.gain.setValueAtTime(0, t);
  env.gain.linearRampToValueAtTime(0.3, t + 0.01);
  env.gain.exponentialRampToValueAtTime(0.001, t + length);

  osc.connect(env).connect(out);
  osc.start(t);
  osc.stop(t + length);
}

Timbre is the frequencies you did not ask for

A sine wave is a single frequency and nothing else, which is why it sounds like a hearing test and unlike any instrument. Pluck a guitar string tuned to 440Hz and you do get a wave repeating 440 times a second, but the string is also vibrating in halves, and in thirds, and in quarters, all at the same time. Those are extra frequencies at 880, 1320, 1760 and on up, all riding on top of the one you asked for.

That stack is called the harmonic series. The note you asked for is the fundamental, and the series is that frequency multiplied by 1, 2, 3, 4, 5 and on up:

The harmonic series of 220Hz. Click a bar to hear that harmonic on its own.

Click the bars. On their own they are fairly boring. What matters is that they arrive as a package, and the recipe of how loud each one is relative to the others is what makes a violin sound like a violin and not a trumpet. Musicians call that timbre, and it is the same note either way.

The browser ships four of those recipes ready-made:

Four waveforms, same pitch

["sine", "triangle", "square", "sawtooth"].forEach((type, i) => {
  note(220, i * 0.7, 0.6, type);
});

Same 220Hz, four very different characters. A square wave contains only the odd harmonics, which is why it sounds hollow and slightly electronic. A sawtooth contains all of them and sounds harsh and buzzy. The scope above shows the shape of each one as it plays, and the shape is the harmonic recipe.

Remember the harmonic series, because it explains most of what follows. Every note you play drags a stack of quiet extra notes along with it, and which notes those are is not up to us. It is arithmetic, fixed by the physics of vibrating strings and columns of air, and it comes out the same on every instrument built around them.

Doubling the frequency gives you the same note

Here are five notes. Every one is double the frequency of the one before it.

One note, five times

[110, 220, 440, 880, 1760].forEach((freq, i) =>
  note(freq, i * 0.45, 0.4),
);

They are different pitches, and yet they sound like the same note. Not just similar, the same. Cultures with no contact with each other have landed on this independently: double the frequency and you get something so alike it deserves the same name. In Western notation these frequencies, in the above example, are all called A, and the distance between them is the octave.

The naming is not arbitrary, and the harmonic series explains why. Every harmonic of 440 is already sitting in the harmonic series of 220, because 220's series is 220, 440, 660, 880, 1100 and 440's is 440, 880, 1320, 1760. The higher note adds no frequency the lower note was not already producing, so your ear hears the same colour at a different brightness.

That one observation gives us two rules we will lean on for the rest of the article.

Pitch is multiplicative, not additive. Going up an octave means times two, not plus anything. The gap from 110 to 220 is 110Hz and the gap from 880 to 1760 is 880Hz, and they sound like exactly the same distance. Frequency space is logarithmic, and every interval in music is a ratio.

We only have to solve one octave. Because doubling returns you to the same note, the problem of "which pitches should exist" reduces to "how should we divide up the space between a frequency and twice that frequency". Solve it once and the answer repeats across the whole audible range.

So: how do you divide an octave?

Simple ratios sound good, and here is why

The naive answer is to divide it evenly and go home. Nobody does that, because it turns out we do not experience all pairs of frequencies the same way. Some combinations sound settled and some sound like a mistake, and you can hear the difference immediately.

Six ratios against the same note

const ratios = [
  ["2/1   octave", 2],
  ["3/2   fifth", 3 / 2],
  ["4/3   fourth", 4 / 3],
  ["5/4   major third", 5 / 4],
  ["16/15 semitone", 16 / 15],
  ["√2    the awkward one", Math.SQRT2],
];

ratios.forEach(([label, ratio], i) => {
  note(220, i * 1.4, 1.2);
  note(220 * ratio, i * 1.4, 1.2);
  console.log(label, "->", (220 * ratio).toFixed(2) + "Hz");
});

The first four sound like chords. The 16/15 sounds like two notes arguing. The last one sounds like a car alarm. And the pattern is not subtle once you see it: the simpler the fraction, the better it sounds. 2/1 the octave, then 3/2 the fifth, then 4/3 the fourth, then 5/4 the major third, and by the time you get to 16/15 it has fallen apart entirely.

That is a suspiciously arithmetic result for something as subjective as "sounds nice", and there are two physical reasons for it.

The first is the harmonic series again. Play 220 and 330 together, which is a 3:2 ratio. The first note produces 220, 440, 660, 880, 1100, 1320. The second produces 330, 660, 990, 1320, so they share 660 and 1320 exactly. Two notes a fifth apart are not really two separate sounds, they are two heavily overlapping stacks that reinforce each other. Now try 220 and 311, which is close to √2. Nothing lines up, at any harmonic. You get two full stacks of frequencies that have nothing to do with each other.

The second reason is roughness. When two frequencies are close but not identical, they drift in and out of phase and you hear the volume pulsing. That is beating, and it is the thing that makes an out-of-tune note sound out of tune:

Beating, from wide apart to identical

[220, 226, 223, 221, 220.5, 220].forEach((freq, i) => {
  note(220, i * 1.3, 1.2);
  note(freq, i * 1.3, 1.2);
});

The wobble slows down as the two frequencies converge and vanishes when they match, and the rate of it is exactly the difference between them. Six hertz apart, six pulses a second. When the fractions are complicated, the two stacks of harmonics are littered with pairs that are a few hertz apart, and every one of those pairs is beating away against the others. That is what dissonance is.

Switch between the ratios above and watch the green line, which is the two waves added together, exactly as your eardrum would add them. For 2:1 and 3:2 the combined shape settles into a repeating pattern almost immediately. For 16:15 it takes fifteen cycles to come back round. For √2 it never does, because √2 is irrational, so there is no pattern for your ear to lock onto at all.

Consonance is your ear finding a repeating pattern quickly. As far as I can tell, that is all it is.

Stacking fifths, and the bug you cannot fix

Now we can actually build something. We know that simple ratios are the good ones, and after the octave itself, the cleanest ratio in physics is 3/2, the fifth.

So what happens if we try to build a musical alphabet using only octaves and fifths?

Do the obvious thing: keep going up by fifths, halving whenever you leave the octave. This is roughly what Pythagoras did, and it works beautifully for a while:

Stacking fifths inside one octave

let freq = 220;
const notes = [220];

for (let i = 0; i < 12; i++) {
  freq = (freq * 3) / 2;
  while (freq >= 440) freq = freq / 2;
  notes.push(freq);
  console.log(`fifth ${i + 1}: ${freq.toFixed(3)}Hz`);
}

notes.sort((a, b) => a - b).forEach((f, i) => note(f, i * 0.22, 0.3));

Look at the first note and the last note. We started on 220, applied twelve fifths, and landed on 222.99. Not 220. Close enough to be audibly trying to be the same note, and far enough off to be unusable.

The error is not rounding but something structural, and it is easier to see without the octave-folding:

The gap that cannot be closed

const twelveFifths = (3 / 2) ** 12;
const sevenOctaves = 2 ** 7;

console.log("twelve fifths:", twelveFifths.toFixed(6));
console.log("seven octaves:", sevenOctaves.toFixed(6));
console.log("ratio:", (twelveFifths / sevenOctaves).toFixed(6));
console.log(
  "in cents:",
  (1200 * Math.log2(twelveFifths / sevenOctaves)).toFixed(2),
);

note(220, 0, 1.5);
note((220 * twelveFifths) / sevenOctaves, 0, 1.5);

Twelve perfect fifths overshoot seven perfect octaves by a factor of 1.0136. That gap is called the Pythagorean comma. Cents are how pitch distances get measured, and there are 1200 of them in an octave, which is where that number in the code comes from. So 23 cents is about a quarter of the gap between two adjacent piano keys, and you can hear it in that last pair of notes as a slow ugly beating.

And it cannot be fixed, for a reason a programmer will recognise. Stacking fifths means multiplying by 3/2, so after n fifths you are at 3^n / 2^n. Stacking octaves means 2^m. For the two to ever meet you would need 3^n = 2^(n+m), which requires a power of three to equal a power of two. Three and two are both prime. It never happens, for any n, ever.

The system we want, where the octave is pure and the fifths are pure and everything closes into a neat loop, does not exist and never has. Every tuning system in history is a different choice about where to dump the error.

Equal temperament is the compromise

The modern answer is to give up on pure ratios altogether. Take the octave, divide it into twelve equal multiplicative steps, and accept that nothing except the octave will be exactly right ever again.

One step is the twelfth root of two:

The twelfth root of two

const semitone = 2 ** (1 / 12);
console.log("one semitone =", semitone);

let freq = 220;
for (let i = 0; i < 13; i++) {
  note(freq, i * 0.2, 0.28);
  freq = freq * semitone;
}

console.log(
  "twelve steps later:",
  (220 * semitone ** 12).toFixed(10),
);

Twelve steps land on exactly 440, because that is how roots work, so the octave is perfect by construction and everything else is approximated.

The obvious question is why twelve. It is not tradition, and you can find the answer yourself in about fifteen lines. Divide the octave into n equal steps for every plausible n, then check how close the best available step comes to a real 3/2 fifth:

Brute-forcing the number twelve

function bestFifth(n) {
  let error = Infinity;
  let step = 0;

  for (let candidate = 1; candidate < n; candidate++) {
    const off = Math.abs(2 ** (candidate / n) - 1.5);
    if (off < error) {
      error = off;
      step = candidate;
    }
  }

  return { step, error };
}

for (let n = 5; n <= 25; n++) {
  const { step, error } = bestFifth(n);
  const pct = (error / 1.5) * 100;
  console.log(
    `${n} steps: best fifth is step ${step}, off by ${pct.toFixed(3)}%`,
  );
}

// And here is what those errors actually sound like.
[5, 7, 12, 19].forEach((n, i) => {
  const { step } = bestFifth(n);
  note(220, i * 1.7, 1.5);
  note(220 * 2 ** (step / n), i * 1.7, 1.5);
});

Those last four lines play the best fifth that 5, 7, 12 and 19 divisions can manage, each against the same 220Hz. The first two wobble, the third is clean, and the fourth sits in between. You are listening to the error column.

Twelve is the first division that gets the fifth right to about a tenth of a percent, and it is more than three times better than anything below it. Twenty-four ties, but only because twenty-four steps is twelve steps with a spare note wedged between each pair, so it is not really a competitor. You have to go all the way to 29 and 41 before the fifth gets better, and nobody is building a keyboard with 41 keys per octave. It is the cheapest number of notes that buys you a convincing fifth, and once the fifth is close the fourth and the thirds come along for the ride.

Here is how close:

Real fifth against the compromise

const pure = (220 * 3) / 2;
const tempered = 220 * 2 ** (7 / 12);

console.log("pure fifth:     ", pure.toFixed(4) + "Hz");
console.log("tempered fifth: ", tempered.toFixed(4) + "Hz");
console.log(
  "difference:     ",
  (1200 * Math.log2(tempered / pure)).toFixed(2),
  "cents",
);

note(220, 0, 1.5);
note(pure, 0, 1.5);
note(220, 2, 1.5);
note(tempered, 2, 1.5);

Two cents flat. There is a slow beat in the second pair if you listen for it, and that beat is present in every fifth played on every piano on Earth. We all decided that being slightly wrong everywhere was better than being perfect in one key and unusable in the others.

The payoff is that notes are now integers. Pick any note as number 0, and every other note is a whole number of semitones away from it. The convention is MIDI numbering, where 69 is our 440Hz A, and the conversion is one line:

Notes are integers now

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);

console.log("60 (middle C):", midiToFreq(60).toFixed(2));
console.log("69 (the A above it):", midiToFreq(69).toFixed(2));
console.log("81 (an octave up):", midiToFreq(81).toFixed(2));

[60, 62, 64, 65, 67, 69, 71, 72].forEach((n, i) =>
  note(midiToFreq(n), i * 0.25, 0.4),
);

That last line is a major scale, and we have not defined what a scale is yet. It is just an array of integers. From here on everything in this article is done with arrays, and I stopped needing to think about frequencies at all.

Twelve notes per octave, repeating forever. The black keys are not special, they are just the ones that did not get letter names.

That keyboard is worth staring at for a second, because the layout is a historical accident. There are twelve equally spaced notes per octave. Seven of them got letters and a big white key, five got a sharp sign and a small black key, and the seven with white keys are exactly the scale you just played. The physics underneath is completely uniform, and the keyboard is not.

Why twelve notes and not all of them

Having twelve notes does not mean using twelve notes. Play all of them in order and it is remarkably unmusical:

All twelve, in order

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);

for (let n = 60; n <= 72; n++) {
  note(midiToFreq(n), (n - 60) * 0.18, 0.25);
}

It sounds like a sound effect. Every step is identical, so nothing stands out, nothing sounds like home, and there is no way to tell where you are.

Music picks a subset. Almost always seven of the twelve, chosen so that the gaps between them are uneven, which is exactly what makes it possible to tell one note from another by ear. The subset is a scale, and a scale is best written not as notes but as the steps between them:

A scale is a list of gaps

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);

const major = [2, 2, 1, 2, 2, 2, 1];

function buildScale(root, pattern) {
  return pattern.reduce(
    (notes, step) => [...notes, notes.at(-1) + step],
    [root],
  );
}

const cMajor = buildScale(60, major);
console.log("C major:", cMajor);

cMajor.forEach((n, i) => note(midiToFreq(n), i * 0.25, 0.4));

The steps are 2 2 1 2 2 2 1, adding up to 12 so the pattern closes the octave exactly. That is the major scale, the single most familiar sound in Western music, and it is a seven-element array.

Change one number and it becomes a completely different mood:

One number is the difference between happy and sad

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);
const build = (root, pattern) =>
  pattern.reduce(
    (notes, step) => [...notes, notes.at(-1) + step],
    [root],
  );

const patterns = {
  major: [2, 2, 1, 2, 2, 2, 1],
  naturalMinor: [2, 1, 2, 2, 1, 2, 2],
  majorPenta: [2, 2, 3, 2, 3],
  minorPenta: [3, 2, 2, 3, 2],
  blues: [3, 2, 1, 1, 3, 2],
};

let when = 0;
Object.entries(patterns).forEach(([name, pattern]) => {
  const scale = build(60, pattern);
  console.log(name.padEnd(13), scale.join(" "));
  scale.forEach((n, i) => note(midiToFreq(n), when + i * 0.22, 0.35));
  when += scale.length * 0.22 + 0.5;
});

Major and natural minor are the same seven-note idea with the gaps shuffled. The pentatonic scales drop two notes, which is why it is so hard to play a wrong note in them, and why every beginner guitar lesson starts there. The blues scale adds one deliberately awkward note back in.

The modes, which I had always seen presented as seven exotic Greek names to be memorised, turn out to be the same array rotated. Take the first step off the front, put it on the back, and you have the next one:

Modes are array rotations

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);
const build = (root, pattern) =>
  pattern.reduce(
    (notes, step) => [...notes, notes.at(-1) + step],
    [root],
  );

const major = [2, 2, 1, 2, 2, 2, 1];
const names = [
  "Ionian",
  "Dorian",
  "Phrygian",
  "Lydian",
  "Mixolydian",
  "Aeolian",
  "Locrian",
];

const rotate = (arr, by) =>
  arr.map((_, i) => arr[(i + by) % arr.length]);

names.forEach((name, i) => {
  const pattern = rotate(major, i);
  console.log(name.padEnd(11), pattern.join(" "));
  build(60, pattern).forEach((n, j) =>
    note(midiToFreq(n), i * 2 + j * 0.2, 0.3),
  );
});

Seven modes, one array, seven rotations. Ionian is the major scale and Aeolian is the natural minor, which means "major" and "minor" are not two systems, they are rotation 0 and rotation 5 of the same thing. Lydian sounds dreamy and Phrygian sounds Spanish and Locrian sounds broken, and all of that comes from moving which gap sits where relative to the note you started on.

This is the point where I stopped feeling like music theory was arbitrary.

Chords are notes stacked in thirds

A chord is more than one note at the same time. Which is not much of a definition, because most combinations sound terrible. The useful question is which combinations do not.

We already know the answer from the ratios: notes whose harmonics overlap. In a scale, the notes that fit that description are the ones two scale degrees apart, a gap musicians call a third. So take a scale, pick a starting degree, and grab every other note:

A triad is [0, 2, 4]

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);
const build = (root, p) =>
  p.reduce((n, s) => [...n, n.at(-1) + s], [root]);

const cMajor = build(60, [2, 2, 1, 2, 2, 2, 1]);
const triad = [0, 2, 4].map((i) => cMajor[i]);

console.log("scale:", cMajor.join(" "));
console.log("triad:", triad.join(" "));
console.log("gaps:", triad[1] - triad[0], "and", triad[2] - triad[1]);

triad.forEach((n) => note(midiToFreq(n), 0, 2));

That is a C major chord. Three notes, at 0, 4 and 7 semitones above the note it is built on, which musicians call the root. In frequency that is 1 : 1.26 : 1.50, very nearly 4 : 5 : 6. Three simple ratios sharing harmonics all over the place, which is why it sounds so settled.

A third is either 4 semitones (a major third) or 3 semitones (a minor third), and stacking two of them gives you 7 semitones either way: major is 4 + 3, while minor is 3 + 4.

Now move the middle note down by one semitone:

One semitone, entirely different feeling

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);
const chord = (shape, root, at) =>
  shape.forEach((s) => note(midiToFreq(root + s), at, 1.6));

const shapes = {
  major: [0, 4, 7],
  minor: [0, 3, 7],
  diminished: [0, 3, 6],
  augmented: [0, 4, 8],
};

Object.entries(shapes).forEach(([name, shape], i) => {
  console.log(name.padEnd(11), shape.join(" "));
  chord(shape, 60, i * 2);
});

Major to minor is one note moving by one semitone. That is all that separates the two emotional poles of Western music, and in code it is [0,4,7] versus [0,3,7]. Diminished squashes both gaps and sounds unresolved and anxious. Augmented stretches both and sounds like something is about to go wrong in a film.

I found this genuinely annoying to discover, in a good way. I had absorbed the idea that major and minor were deep categories. They are a single array element differing by one.

Add a fourth note, another third up, and you get seventh chords, and this is where it starts to sound like actual music rather than a hymn:

Sevenths

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);

const shapes = {
  "major 7th": [0, 4, 7, 11],
  "minor 7th": [0, 3, 7, 10],
  "dominant 7th": [0, 4, 7, 10],
};

Object.entries(shapes).forEach(([name, shape], i) => {
  console.log(name.padEnd(13), shape.join(" "));
  shape.forEach((s) => note(midiToFreq(60 + s), i * 2.2, 1.8));
});

Major 7th is the jazz-cafe chord. Minor 7th is smooth and slightly melancholy. The dominant 7th is the interesting one. Dominant is just the traditional name for the fifth degree of a scale, and this chord is about to do a lot of work.

A key gives you seven chords

This is the part that made chord charts finally make sense to me.

There is nothing special about starting on the first degree. Do it from each of the seven in turn, wrapping around the octave, and you get seven chords, all built only from the seven notes of the scale:

Seven chords out of one scale

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);
const build = (root, p) =>
  p.reduce((n, s) => [...n, n.at(-1) + s], [root]);

const scale = build(60, [2, 2, 1, 2, 2, 2, 1]).slice(0, 7);
const names = ["C", "D", "E", "F", "G", "A", "B"];

const chordOn = (degree) =>
  [0, 2, 4].map((step) => {
    const i = degree + step;
    return scale[i % 7] + Math.floor(i / 7) * 12;
  });

names.forEach((name, degree) => {
  const notes = chordOn(degree);
  const shape = notes.map((n) => n - notes[0]);
  const quality =
    shape[1] - shape[0] === 4
      ? "major"
      : shape[2] - shape[0] === 6
        ? "diminished"
        : "minor";
  console.log(`${name} ${quality.padEnd(11)} ${notes.join(" ")}`);
  notes.forEach((n) => note(midiToFreq(n), degree * 1.5, 1.3));
});

Nobody chose those qualities. Three of them come out major, three come out minor, and the last one comes out diminished, and that pattern is forced by the uneven gaps in the scale. Start the every-other-note process on a degree whose neighbours are spaced 4 then 3, you get a major chord. Spaced 3 then 4, you get a minor one.

Musicians write those seven as Roman numerals: capital for major, lowercase for minor, and a small circle for the diminished one.

I ii iii IV V vi vii°

That notation is doing something useful, and it took me an embarrassingly long time to notice what. It describes chords by their position in the scale, not by their name. A V chord is "the chord built on the fifth degree", whatever key you are in, which is relative addressing. A chord chart written in Roman numerals is key-independent source, and transposing is adding a constant:

Roman numerals are relative addressing

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);
const build = (root, p) =>
  p.reduce((n, s) => [...n, n.at(-1) + s], [root]);

function chordsInKey(root) {
  const scale = build(root, [2, 2, 1, 2, 2, 2, 1]).slice(0, 7);
  return (degree) =>
    [0, 2, 4].map((step) => {
      const i = degree - 1 + step;
      return scale[i % 7] + Math.floor(i / 7) * 12;
    });
}

// The same four numerals, played in two different keys.
const progression = [1, 5, 6, 4];

[60, 65].forEach((key, k) => {
  const chord = chordsInKey(key);
  progression.forEach((numeral, i) => {
    chord(numeral).forEach((n) =>
      note(midiToFreq(n), k * 7 + i * 1.6, 1.5),
    );
  });
});

Same shape, two different starting notes, and your ear recognises it as the same music, which is why the notation exists.

Tension and resolution

One chord is just a sound. Music is what happens when you put them in an order, and the order matters. Some sequences feel like they have arrived and some feel like a question.

The strongest pull in the system is from V back to I, and there are two concrete reasons for it.

The pull of V back to I

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);
const play = (notes, at, len = 1.6) =>
  notes.forEach((n) => note(midiToFreq(n), at, len));

const C = [60, 64, 67]; // I
const G7 = [55, 59, 62, 65]; // V7

play(G7, 0, 1.8);
play(C, 2, 2.2);

console.log("B is", 59, "and C is", 60, "- one semitone apart");
console.log("F is", 65, "and E is", 64, "- one semitone apart");

First, the note B sits in the V chord and is one semitone below the root of I. A note that close to a destination sounds like it is leaning on the door. Musicians call it the leading tone, and it works like a cliffhanger.

Second, the dominant 7th chord contains both B and F, which are six semitones apart. Six semitones is the tritone, exactly half an octave: 2 ** (6/12) is the square root of two, the exact irrational ratio we heard earlier. The single most unstable interval available, sitting inside the chord, and both of its notes resolve by one semitone in opposite directions when you move to I. The tension you hear is that irrational ratio being swapped for simple ones.

The language of tension and release is built on that mechanism. Here it is with a few of the progressions you have heard ten thousand times:

Four progressions you already know

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);
const build = (root, p) =>
  p.reduce((n, s) => [...n, n.at(-1) + s], [root]);

const scale = build(60, [2, 2, 1, 2, 2, 2, 1]).slice(0, 7);
const chord = (degree) =>
  [0, 2, 4].map((step) => {
    const i = degree - 1 + step;
    return scale[i % 7] + Math.floor(i / 7) * 12;
  });

const progressions = {
  "I  V  vi IV": [1, 5, 6, 4],
  "ii V  I": [2, 5, 1],
  "I  vi IV V": [1, 6, 4, 5],
  "vi IV I  V": [6, 4, 1, 5],
};

let when = 0;
Object.entries(progressions).forEach(([name, degrees]) => {
  console.log(
    name,
    "->",
    degrees.map((d) => chord(d).join("/")).join("  "),
  );
  degrees.forEach((d, i) => {
    chord(d).forEach((n) => note(midiToFreq(n), when + i * 1.1, 1));
  });
  when += degrees.length * 1.1 + 0.9;
});

I V vi IV is the four chords that an alarming share of pop music is built from. ii V I is the backbone of jazz. vi IV I V is the same four chords as the first one, rotated to start somewhere sadder.

Edit the arrays. Almost any sequence of numbers from 1 to 7 will hang together, because every chord is built from the same seven notes. A key buys you a constrained space where wrong answers are hard to reach. Ending on 1 sounds finished, ending on 5 sounds like there is another line coming, and ending on 7 sounds like something has gone wrong.

Notation is a serialisation format

None of what came before needed a staff. Everything above is arrays of integers and a function that turns them into frequencies.

But notation exists, it is the format the literature of Western music is stored in, and once you already know what it is encoding it turns out to be a fairly sensible design with some very old constraints. It is a serialisation format, written before printing was cheap, optimised for a human reading it in real time while their hands are busy, and never revised because the install base was too large.

Here is the C major scale, the same seven integers as before:

The vertical axis is pitch, but not linearly. Each line and each space is one step up the scale, so consecutive positions are sometimes two semitones apart and sometimes one. The axis is diatonic rather than chromatic: it steps through the scale rather than through all twelve notes, which means it is showing you scale degrees dressed up as pitches. That is why the major scale looks like a boring straight run up the page and sounds like the most natural sequence in the world. The format is optimised for the case it expects.

The clef declares the origin. A staff is five lines with nothing pinning it to any frequency, so the symbol at the front tells you where you are. The treble clef is a stylised G, and the curl of it wraps around the line that means G. The bass clef is a stylised F with two dots straddling the F line. It is a coordinate system with the origin marked in the margin:

Same shape on the page, different origin, so it plays back an octave down. Treble and bass are the two most common clefs, covering between them most of what people sing and play.

Accidentals patch the lossy encoding. Seven vertical positions per octave, twelve notes to represent. The sharp, flat and natural signs are the escape hatch, and every one of them is an instruction to shift the note the position would otherwise mean:

The key signature is DRY. If a piece is in D major, its scale contains F sharp and C sharp, and every single F and C in the piece would need a sharp sign next to it. So instead you declare it once, at the front of every line, and it applies until something says otherwise. It is a constant hoisted to the top of the file:

Note that the two sharps are still being played, they are just not written on each note. This is also why sheet music tells you the key before you have played anything, and why musicians talk about a piece being "in" a key. The key is in the header, not the body.

Durations are powers of two. A whole note, a half note, a quarter, an eighth, a sixteenth. Each one is half the last, and the notation encodes the exponent visually: an empty notehead, then a stem, then a flag per halving. It is a unary encoding of a binary exponent, which is a very medieval way to store a number and impossible to misread at a glance:

Powers of two get you a long way but not everywhere, so there is one more operator: a dot after a notehead multiplies its length by 1.5. Two dots multiply by 1.75. It is a binary fraction, written as punctuation.

None of these durations are times, though. They are beats, and beats become seconds only when you fix a tempo:

Beats are not seconds until you say so

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);

const bpm = 120;
const beat = 60 / bpm;

// [midi note, length in beats]
const melody = [
  [60, 1],
  [62, 1],
  [64, 2],
  [65, 0.5],
  [67, 0.5],
  [69, 3],
];

let when = 0;
melody.forEach(([n, beats]) => {
  note(midiToFreq(n), when, beats * beat * 0.95);
  when += beats * beat;
});

console.log("total:", when.toFixed(2), "seconds at", bpm, "bpm");

Change bpm to 200 and the same array is the same tune, faster. That split is what makes a score portable. It stores relative durations, and the performer supplies the clock.

The time signature groups the beats. 4/4 means four quarter-note beats per bar, 3/4 means three, and the vertical bar lines are there so your eye can find its place on a page. It is mostly a readability feature, but it also carries a real musical claim, which is that the first beat of each group is the strong one. Play the same six notes grouped in threes and grouped in twos and they become different pieces of music.

That is more or less all there is to it. Pitch on a diatonic axis with an origin and an escape hatch, duration as negative powers of two, and a couple of header fields. Everything else on a page of sheet music is performance instructions layered on top: how loud, how smoothly, which finger.

Putting it together

Here is everything above in one place. A key, its diatonic chords, a progression, those chords broken into an arpeggio one note at a time, and a melody that sticks to the scale. About forty lines, no library, and it is the first thing I made with code that I would describe as music rather than as a demonstration:

A key, a progression, and a tune

const midiToFreq = (n) => 440 * 2 ** ((n - 69) / 12);
const build = (root, p) =>
  p.reduce((n, s) => [...n, n.at(-1) + s], [root]);

const key = 57; // A
const scale = build(key, [2, 1, 2, 2, 1, 2, 2]).slice(0, 7); // natural minor
const bpm = 104;
const beat = 60 / bpm;

const chord = (degree) =>
  [0, 2, 4].map((step) => {
    const i = degree - 1 + step;
    return scale[i % 7] + Math.floor(i / 7) * 12;
  });

const progression = [1, 6, 3, 7];
const melody = [0, 2, 4, 2, 3, 2, 1, 0, 4, 3, 2, 1, 0, 2, 1, 0];

progression.forEach((degree, bar) => {
  const at = bar * 4 * beat;
  const notes = chord(degree);

  // Bass note on the downbeat.
  note(midiToFreq(notes[0] - 12), at, beat * 3.6, "triangle");

  // Arpeggio: up, down, up, across the bar.
  [0, 1, 2, 1, 0, 1, 2, 1].forEach((which, i) => {
    note(midiToFreq(notes[which]), at + i * beat * 0.5, beat * 0.45);
  });

  // Melody, four notes per bar, always from the scale.
  melody.slice(bar * 4, bar * 4 + 4).forEach((step, i) => {
    note(
      midiToFreq(scale[step % 7] + 12),
      at + i * beat,
      beat * 0.9,
      "triangle",
    );
  });
});

Every number in there means something we derived. 57 is A because of the twelfth root of two and a tuning fork. [2,1,2,2,1,2,2] is the minor scale because it is the major scale rotated five places. [0,2,4] is a chord because harmonics overlap when notes are two scale degrees apart. [1,6,3,7] sounds like it goes somewhere because of where the tension sits.

Change the key to 60 and it moves. Change the scale pattern to [2,2,1,2,2,2,1] and the same tune turns cheerful. Change the melody array to anything at all and it will still fit, because it is indexing into the scale rather than choosing frequencies. That constraint is what all the theory above was for.

What I still do not understand

Quite a lot. This article covers pitch and almost nothing else, and pitch may be the easy half.

Rhythm I have barely touched, and everything I have read suggests it is deeper than it looks. Voice leading, which is the business of moving between chords by the smallest possible distance rather than jumping around, is where written music starts sounding good rather than merely correct, and I can state the rule without hearing why it works. Why a melody wants to land where it lands is still mostly opaque to me. And the whole thing above is one tradition's answer. Plenty of music divides the octave differently, or does not treat the octave as the unit at all, and none of it is wrong.

But I no longer feel like I am being asked to memorise trivia. Twelve notes is what you get when you try to reconcile powers of two with powers of three and settle for a rounding error. A scale is a subset of those notes with gaps uneven enough that you can tell where you are. A chord is a set of notes whose harmonics already overlap, a key is a way of naming chords relative to a starting note, and notation is an old and slightly lossy way of writing all of that down. Each of those was an engineering decision, made a long time ago, under constraints, and the reasons are still there if you go looking.

If you want to go further, the two things that made me start writing this were Music Theory for Nerds by Eevee, which is a great read, and LightNote, which is the most beautiful thing on the internet about this subject. For the API side, MDN's Web Audio documentation is unusually good.

Every example on this page is plain JavaScript with no dependencies, so all of it runs anywhere with a browser engine. If you want to keep pulling on the thread, RunJS is a desktop JavaScript playground that ships with the Web Audio API alongside Node and npm, so there is nothing to set up. Paste any of the snippets above into it, add const ctx = new AudioContext() and const out = ctx.destination at the top, and carry on from there.

James Marriott’s The New Dark Ages

I’m not the only parent who feels increasingly anxious about their children’s media diet. For teenage boys in particular, entertainment often comes in the form of YouTube, which for any adult briefly glancing at the screen seems to feature an array of thumbnailed gawping imbeciles, besides an invitation to watch the latest work of creativity: ‘I took a bath full of Diet Coke and Mentos’ or ‘I cemented a microwave to my head’. In comparison to this, the mindless television I watched in the 1990s seems as narratively complex as Dostoevsky.

Most of all, it’s notable that many children aren’t reading anymore, and it’s steadily become a topic of interest among fellow parents, then worry, and now panic. It’s impossible to ignore the fact that something quite bad is happening.

Every catastrophe needs its Jeremiah, and for this we have Times columnist James Marriott, whose increasing anxiety about the decline of literacy was articulated in a hugely popular substack post last year. Having written extensively about how no one reads books anymore, he decided to turn the subject into a book, The New Dark Ages, published this week.

I wouldn’t call the book ‘urgent’ because I usually take that to mean ‘worthy scolding by an ideological ally’; it’s entertaining, informative, often amusing and makes a coherent case in 50,000 or so words, which many of us can still manage. But the implications he lays out are disturbing, and influence everything around us.

James and I have become friends over the past couple of years, drawn together by a mutual sense of despair, and he’s probably the only person in journalism even more pessimistic than I am. If you believe the media to be a circle-jerk of people helping their mates (which it is, obviously), bear that in mind, but reading his prose one gets a sense of how likeable as well as articulate he is, even if born in the wrong decade. His upbringing will seem as alien to those raised in the coming years as the world of Boethius would have been to Alfred the Great.

‘Literature was our religion: the way we explained and interpreted our lives. As a child I quickly learned that every possible event — from a major bereavement to a minor playground contretemps — could be cross-referenced against a novel by Thomas Hardy or a poem by Philip Larkin. There was a quote for everything. I was dragged off to Shakespeare plays at what now strikes me as a rather cruelly young age (“if you’re not enjoying it”, I remember my dad telling me, “you can just read your book instead”). We worshipped pretty much all the canonical English poets but in our pantheon of literary saints, Shakespeare was Jesus or, more probably, God.

‘We celebrated his birthday every year with a homemade cake and in the holidays we were taken on solemn pilgrimages to Stratford-upon-Avon. I recall that I was once presented with a Shakespeare-themed stuffed bear (it was wearing a t-shirt with a quotation from Hamlet).’

Such a veneration for the written word naturally made him sensitive to disturbing stories about declining literacy that began circulating around the turn of the decade. ‘The first rumours that something important was happening reached me via friends working in universities,’ he writes: ‘I came close to going into academia myself (though I don’t think the world is a poorer place for the loss of my proposed PhD on landscape imagery in post-war British poetry).’

Anecdotes were soon backed up by data, showing significant declines in reading comprehension among students, as well as an even sharper drop in the number of adults and children reading for pleasure. The chief suspect seemed obvious: it’s the phones. (Everything’s ‘the phones’). Yet the more Marriott looked into it, the more he came to believe that the real transformation came with television, as Neil Postman had warned in his 1985 polemic Amusing Ourselves to Death.

‘Postman believed that the rise of TV — with its bias towards triviality and charismatic personalities — was beginning to undermine the kinds of reasoned and productive public discourse on which a civilised democracy depends. He was dismissed by some of his critics as a crank and a Luddite. The passing of time has revealed that he was a prophet.’

Having collapsed at the end of antiquity, literacy rates began to show an uptick from the 12th century, initially in the urban centres of northern Italy but then rapidly rising with the invention of printing; even then, however, the nascent books trade overestimated how keen the public were, and he recalls the case of ‘two hapless Roman publishers who wrote to Pope Sixtus IV at the end of the fifteenth century asking for help because they had printed 20,000 books in a fit of over-enthusiasm and were unable to sell any of them.’

Then came Protestantism, with its emphasis on reading the Bible - in Sweden the Lutheran Church even banned the illiterate from marrying - and the Enlightenment, of which Marriott is a great fan. This produced both a republic of letters among our continent’s intellectual elite, but a much broader spread of empathy, enabling many people to see beyond the categories of social class, sex or race, into the soul and mind within.

‘In a book you can find yourself paying close attention to someone you might have very easily dismissed had you encountered them in real life. The wealthy male readers of Pamela spent hours with the innermost thoughts of a serving girl, a kind of person many of them would not have given so much as a second thought to had they encountered her in the street.’

Likewise, with readers of Olaudah Equiano’s memoir The Interesting Narrative, now able to see life through the eyes of a slave in a way that was transformative: ‘what it was like to be stolen from one’s home by slave traders, packed onto a slave ship and sold like an object.’ In as much as reactionaries felt a ‘moral panic’ about the dangerous effects of reading, Marriott thinks they were right: literacy would indeed undermine a hierarchical society and bring about a more egalitarian, democratic spirit.

Marriott identifies the peak of literacy sometime in the early to mid-20th century, and ‘by 1944, a survey found that “nearly two-thirds of skilled workers and almost half of unskilled workers grew up in homes with substantial libraries”.’ Yet already the great change was coming. In 1948, only 1 per cent of American homes owned a television; by 1954, over half did. Britain was a bit behind, but the idiot box began to take off after Elizabeth II’s Coronation in 1953, fondly remembered as a great communal event because so few people owned a set. Soon, however, television would become the least communal of activities

‘All of life seemed to be reorganising itself around television’s centrifugal attraction. In living rooms across the world, armchairs and sofas were shifted around so that they no longer faced each other (as they had in most houses for most of history) and were directed instead towards the corner that contained the screen.’

Social life turned inward, and television had a zombifying effect on many; Marriott notes that ‘in 1956 when a technical error delayed the broadcast of ITV’s Sunday Night at the London Palladium, a third of the audience had simply continued staring blankly at their empty sets until programming resumed.’

Television came to ‘devour’ our leisure time, and by 1995, as Robert Putnam wrote in Bowling Alone, 40 per cent of it was spent staring at the screen. While Putnam was writing about the United States, its effects were seen everywhere; in the Netherlands, which possesses some of the best data, the introduction of television saw a decline in the average time spent reading, from five to three and a half hours a week between the 1950s and 70s.

The number of channels exploded with the arrival of cable television, and the number of brain cells fell. Western countries began to see the ‘reverse Flynn Effect’, across-the-board IQ declines, and a natural experiment helps us to identify the culprit:

‘Because Norway is a long, narrow inaccessible country of mountains, fjords and forests, the roll-out of cable TV was staggered by region. Not everywhere got cable at once… This, plus Norway’s policy of military conscription for men, presented an interesting opportunity for social scientists. By measuring the compulsory intelligence tests taken by all military conscripts the economist Øystein Hernæs was able to map the effect on test scores of cohorts of soldiers as cable TV arrived in their areas. What he found was that as cable marched across Norway intelligence fell in its wake: “for each year that their hometown had full coverage, IQ dropped by 0.08 points”. “Higher exposure to commercial television”, Hernæs’s study bluntly concludes, “reduces cognitive ability”.’

And then things only got worse, in 2007, with the arrival of the smartphone. As Marriott points out, television has its ill effects, but it also produces something recognisable as art, and requires some degree of concentration. And whereas most people like television, they often hate their phones, seeing the terrifying power they hold.

If your household is anything like mine, it will feature parents repeatedly nagging their children to ‘get off your phone’, only for them to reply: ‘but you’re on it, too!’ (It’s for work, I lamely reply.) Indeed, many adults struggle to limit their use of these maddeningly addictive devices, which the average person checks 144 times a day; Marriott’s own battles are comical.

‘In my early adulthood I must have spent hundreds of futile and faintly tragicomic evenings attempting to improve my reading by hiding my smartphone from myself. I have shoved my smartphone behind wardrobes and under beds. I have shut it up in cupboards. I have shoved in into the stale and musty depths of laundry baskets. One particularly desperate (though I also thought, rather ingenious) strategy I developed was to kick my phone haphazardly across the room so it would lodge under some unknown piece of furniture thus making it impossible to retrieve without laboriously hunting for it. I went through a phase of locking my phone up like an evil genie in a “smartphone prison”, a sort of miniature safe with a timer.’ Nothing could stop him succumbing to the Ring’s great power, until he got rid of the device and bought a ‘dumbphone’ instead.

The impact on writing was not initially apparent. As late as 2017, a short story in the New Yorker was read by a million people and became the subject of widespread discourse. Just nine years later the very idea seems absurd, and far more attention is given to the likes of ‘James Donaldson, the world’s most popular YouTuber and in the apt description of the writer Mark O’Connell, “the Mozart of the attention economy’’.’ His videos ‘take the form of stunts or accelerated miniature gameshows’ with titles such as ‘I Spent 50 Hours in Solitary Confinement’ or ‘Would You Sit in Snakes for $10,000?’

For those who can’t manage 30 minutes, there is always TikTok, ‘which has justly been described as the “crack” of the smartphone. It has been estimated that the average TikTok user may scroll through as many as 260 videos in the space of about half an hour.’

As our attention spans have declined, ‘statistical analysis shows that even pop songs are becoming simpler and more repetitive’, while television dialogue has become less complex, all subtext or insinuation gone as characters spell out the plot to attention-depleted audiences.

Marriott admits that ‘making this kind of argument is apt to make you sound like a reactionary bore — the kind of armchair-bound windbag who would have droned on about cultural decline all the way through the Beatles, the Rolling Stones and Joni Mitchell. I acknowledge that some readers have already made up their minds that this is precisely who I am and what I am doing. But it’s worth pointing out that the idea that culture is becoming more simple or stagnant and repetitive is not one of history’s recurring moral panics.

‘As the journalist Adam Mastroianni has neatly pointed out, for most of the twentieth century people tended to worry about the exact opposite problem: that there was too much novelty and too much pretentious difficulty. Traditional ways of doing things seemed to be continually, terrifyingly being exploded or reinvented or displaced. Art, literature, philosophy and fashion simply could not stay still.’

Today, even scientific studies are becoming ‘less disruptive’, according to one prestigious journal, ‘increasingly less likely to break with the past in ways that push science and technology in new directions.’ The culture is becoming more conservative in a way that oral societies tend to be – repetitive, static, lacking innovation.

As literacy has declined, politics has become stupider. Reading Hansard from the 1970s, and comparing it to the level of Parliamentary debate today, is a painful exercise. Classical allusions are gone; metaphor is treated literally, often provoking dim-witted outrage; speaking times continue to shrink as MPs compete in the attention economy.

Marriott contrasts Harry Truman, who ruled America during the peak of literacy, with the current incumbent. Truman ‘was the son of a livestock dealer in Independence, Missouri. And yet as an earnest and bespectacled schoolboy he spent great tracts of his time in the local library. “Watched over by a plaster bust of Benjamin Franklin,” Truman made his way past all the familiar landmarks of a twentieth century autodidact self-education: The Bible (which he claimed to have read through twice by the time he was twelve), Plutarch’s Lives, the complete works of Shakespeare, Plato’s Republic, Gibbon’s Decline and Fall of the Roman Empire. Eventually, he read every single one of two thousand volumes stored in Independence’s library — including all the encyclopaedias. It’s an inspiring story but not at all a unique one.’

Donald Trump, meanwhile, is so averse to reading that aides recall that he cannot even make it through one-page memos; his speeches are notably repetitive and characterised by taunting and braggadocio, a reversion to the norms of the oral age. He is an almost Homeric figure, perhaps the first post-literate western leader.

In the new media ecosystem Marriott sees a reversal of the Enlightenment, epitomised by the likes of Candace Owens, who has ‘more followers on YouTube alone than the entire subscriber base of the New York Times’, a ‘one-woman nonsense machine’ who declares that ‘Stalin was Jewish, science is a “pagan faith” and the French Revolution was started by Jews to “bring down Christian empires”.’ He acknowledges, however, that at least she is no longer a flat earther although, as she has stated, ‘I’m not a round earther either’.

‘Well, it’s a start.’

For Marriott, ‘there is nothing inevitable or eternal about modern civilisation in all its desirable complexity and comfort’, nor is it written that it may survive ‘a world filled with demons, anti-vaccine conspiracies and “ways of knowing”’. Democracy followed the rise of literacy, and it may struggle to function in a world where the analytical way of thinking that comes with reading is alien to many voters.

While much of the most unenlightened rhetoric comes from the populist Right, the more mainstream cultural Left is terrified of appearing ‘elitist’, a charge often made to Marriott in interviews. To which, his only answer is: yes, I suppose.

‘Many of us are suspicious of hierarchies of cultural value,’ he writes: ‘To suggest that it is “better” to spend your day reading War and Peace than watching TikTok carries the dangerous ring of elitism. I was once reprimanded by a writer in a British magazine for my “belief that some writing is simply ‘better’”; a view, I was told, that is “inherently damaging, and marginalising”… A recent report issued by Britain’s most important cultural funding body, the Arts Council, warned against the use of “terms like excellence” which may promote “unhelpful hierarchies about what kinds of work are valued”.’

It is notable that the era of peak democracy in the Anglophone world, the mid-20th century when the working class had the most political power, was also the peak of literacy. As reading has declined, so the social gap has widened: iPad children tend to come from the poorest backgrounds, while many tech titans, Steve Jobs in particular, limit the amount of technology their children access, and screens at the dinner table have become very non-U. Hierarchies grow starker when elites fail to promote high culture.

Yet the most startling thing about this book is how far even the nominally well-educated have fallen, so that ‘by the end of the twentieth century a college graduate born after 1969’ read less than someone born before 1950 with a basic level of education. Indeed, ‘nowadays many rich and highly educated people are much less well read than many members of the least privileged classes had been in the middle of the twentieth century.’

To me, that is the most salient and worrying trend: that people in positions of power and influence are now notably ignorant about history and the classics, comfortable only with referencing pop culture. I suspect that the crisis of faith in our institutions is driven not just by a sense that they are corrupt or biased, but that the people running them are not very smart. The cultural elite has abandoned cultural elitism.

In mid-2023, within the “RLSlow” research project, we saw the first results that gave us confidence that we will be able to scale the training of reasoning models, unlocking the capability of pretrained models to form their own chains of thought. Szymon and I spent that night at the office, thinking not about the incredible benchmark numbers, products, or scientific results that this technology will deliver - but rather, trying to process the sobering fact we will actually see machines meaningfully smarter than ourselves in our lifetime, and we already see the shape of these systems; wondering how to alert people to the significance of this.

Three years later, reasoning language models are a rapidly growing part of the economy and starting to push the boundaries of science. They are able to operate computers and graphical interfaces, collaborate with people and each other, and carry out research projects. They are also transforming the landscape of computer security, and in that present clear new dangers.

A lot of new research happened in this period, and our understanding of these systems is again a little different than it was in 2023. Based on internal results, I have a strong expectation that this speed of progress could be sustained into recursive self-improvement. If AI development continues along its current path, the systems we’ll see in the next few years are likely to represent further capability jumps of equal or larger magnitude, and to increasingly drive their own development.

This is a time that calls for extreme caution. I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence. OpenAI will continue to seek technical solutions to alignment and monitoring, to build defensive systems and unilaterally withhold further scaling as needed; however, I believe broader interventions are required.

Intellect we don’t fully understand

At a high level, progress in machine intelligence is driven by increasing computational power. We at OpenAI deeply internalized this around 2017, after seeing consistent returns to scaling across multiple research projects1. As a result, we sought out access to much more compute than we had originally planned, and increasingly oriented our research around a small number of very scalable directions. We believed that was the only way for us to be at the frontier of AI research, and influence the impacts of AGI.

There are new algorithms that have been developed along the way, new feats of ingenuity from teams and individual researchers. I see them largely as discoveries along the path of scaling; the science of deep learning is still nascent, and meaningful algorithmic progress tends to correlate with access to compute. If you zoom out to a multiple-year horizon, AI is continuing to become more intelligent as it is scaled to larger computers.

And, in line with Ray Kurzweil’s predictions from the end of the XXth century⁠(opens in a new window), we now find ourselves at the moment in history of computing where machine intelligence is starting to exceed that of humans in transformative ways.

AI is grown more than designed - it is, to first degree, the product of repeating a straightforward optimization step many times on a hard-to-imagine amount of compute. This results in an incredibly complex system that works through abstract concepts and can simulate facets of human behavior. We can discover various insights about little mechanisms that emerge within this system, in a process similar to neuroscience - and, similarly to neuroscience, its overall action evades a description we can fully understand.

The study of deep learning-based AI is largely an experimental science. We put a lot of effort⁠ into building principled algorithms and making testable predictions, but fundamentally, our large-scale training runs are experiments, and we are sometimes surprised by their results. Moreover, as the systems become more capable, the results become harder to interpret.

This is made more complicated by the current algorithms generally improving easy-to-measure capabilities faster than those hard to objectively quantify. We spend a lot of time trying to understand how capabilities generalize, and what to prioritize to advance the skills that are going to be most relevant in the next few years. For instance, we believe we could make the models better at specifically mathematics research with additional focus, but we do not prioritize this direction because of the urgency we feel about RSI and automated alignment research, as I will discuss later.

The intelligence produced by scaling deep learning is not directly comparable to human intelligence. To become very relevant in the real world - very useful or very dangerous - the AI does not need to match or exceed all human capabilities; it just needs to surpass enough of them. And as it continues to surpass humans on more and more axes, it is becoming increasingly difficult to understand exactly how capable it is.

Teaching machines to love

Because machine intelligence comes from a fundamentally different process than human intelligence, we cannot assume it adheres to human principles by default, or generalizes from them in a human-like manner. The core problem in AI research is that of alignment - getting the AI to “try to do the right thing” by human standards.

For the purpose of organizing practical research directions, I find it useful to distinguish goal alignment and value alignment.

Goal alignment is broadly: “does the AI try to accomplish the goal set before it?”. This can include things like adherence to an instruction hierarchy⁠, or the ability to communicate and collaborate with people, to attempt to understand their objectives. This set of directions has been extremely practically relevant.

Value alignment is a more intrinsic property of the model. It is the ability to hold and generalize from a high-level set of principles; to act “reasonably” even when given unclear or conflicting objectives, or placed in unfamiliar or adversarial situations. An aligned AI should act with honesty and integrity, and love for humanity.

Of course, the boundary between value and goal alignment can be blurry, and truly caring about goals requires attempting to infer the intent⁠(opens in a new window) and values underlying them. However, generally when I talk about the long-term importance of alignment research, I am referring to value alignment.

The fundamental challenge of AI alignment is generalization. As machines become smarter, they find themselves working on higher-level concepts, and placed in environments increasingly different from those they encountered in training. They can fail at generalizing from the values taught and reinforced in their training process to those new situations; and it can be hard for us to be sure how they will act. This is made even more difficult by the fact the overall ecosystem the AIs are used in is changing very quickly; for example, AIs trained today need to be robust to interacting with a variety of other AIs. Crucially, we need future AIs to continue to hold human values regardless of whether they believe they’re under human supervision.

There are two major classes of currently practically employed methods for alignment training.

The first is encouraging aligned behavior as part of goal-oriented reinforcement learning. Model’s actions are evaluated (usually by AI) for being consistent with a given preference model, “spec” or “constitution”, and rewarded appropriately. This approach can be very effective in the average case, and is a core part of how modern AI assistants are made. Unfortunately, it can also be brittle and strongly relies on the coverage of training oversight and the model’s ability to generalize from the situations it has encountered in training. For example, in the OpenAI-Hugging Face incident, the agents preserved a boundary of not social engineering humans. However, they clearly failed to abstain from other actions that were out of scope and went against the spirit of the values they were taught in other settings.

The second approach seeks to leverage the model’s ability to generalize from pretraining data. This can involve crafting alignment-inducing training datasets, or focusing the model on an ‘aligned’ part of the pretraining distribution, as in, for example, the persona selection model⁠(opens in a new window). The weakness of this approach lies in the lack of robustness to further optimization pressure. If you take a model that thinks generally ‘aligned’ thoughts, and subject it to enough training where it’s taught to achieve very hard objectives, it can learn to reason in a motivated way: bending the 'aligned' seeming thoughts as needed to achieve the goal. We likely saw an example of such behavior in recent cybersecurity incidents involving a non-OpenAI model.

We invest heavily along the spectrum of approaches spanned by these directions. We also see meaningful progress - GPT‑6 Astra is the first model that benefits from some important advancements we have been working on for a long time, and is significantly better aligned than GPT‑5.6 Sol. Still, it is important to acknowledge and understand that much more progress is required as models become more capable; and that progress in generalizable alignment may not sufficiently outstrip progress in general model intelligence.

Monitoring generalization

We do not have a satisfactory theory of generalization, and it seems unlikely that we can develop one soon, at least without the help of more powerful AI. Therefore, at present, our ability to empirically validate our alignment techniques is in practice arguably even more important than the alignment techniques themselves.

OpenAI’s primary bet here has been chain-of-thought monitoring⁠. It is based on an appealingly scalable idea: a lot of the model’s capability comes from a verbalized reasoning process (chain-of-thought). If we scale optimization on the outcomes of that process, but do not supervise the process itself, that chain-of-thought has no direct incentive in training to hide any misaligned ideas or objectives. This does not mean the model will learn to externalize misaligned tendencies that don’t rely on using the chain-of-thought; however, it can allow us to monitor exactly the capability increase from reasoning.

We understood the potential significance of chain-of-thought monitoring at the same time we developed reasoning models. When we shipped o1‑preview, we deliberately designed the product to hide the chain of thought, to protect it from supervision pressure in the long term2. In development since, we have strived to maintain the rule of not supervising the reasoning process. CoT monitoring became an extremely important tool for us in studying how our models generalize from their training distribution, allowing us to observe and analyze not only their actions but also their internal process.

This tool continues to be critical as we study the Astra class of models. However, unfortunately our evaluations indicate our ability to rely on CoT monitoring is progressively diminishing. This comes from a combination of factors.

• Modern reasoning models are used in more complex environments than o1‑preview; their reasoning process is increasingly blended with communicating with people, other AIs, and using tools. Many of those interactions have to be supervised, thus blurring the boundary we aim to preserve.
• The AI is becoming better at reasoning about and manipulating its own reasoning process.
• With improved pretraining performance, we also see the models become much smarter even without using verbalized reasoning at all.

These challenges are not necessarily insurmountable. I am hopeful we can develop interventions to improve chain-of-thought monitorability of our models, e.g. by forming a better understanding of the interplay of different optimization objectives and forms of test-time compute the model uses. I also believe there can be great value in combining ideas from CoT and activation monitoring - scaling training of monitors with direct access to network internals, e.g. confessions⁠(opens in a new window). We are actively pursuing these ideas. Still, I expect general AI progress to increasingly be bottlenecked by confidence in monitoring.

Scalable defense

The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI.

A clear risk discussed throughout this year is to cybersecurity: the models are becoming superhuman in their ability to break in and out of computer systems. This expands the scope of risks associated with AI tremendously: agents are going to be able to access any but the most secure infrastructure, and affect a lot of the world directly, even without a physical body. We are currently in a narrow window to use the best available models to significantly tighten security of critical systems.

The risks associated with AI are unfortunately going to grow from here. A very capable agent explicitly trained and instructed to carry out nefarious acts presents a new kind of danger; it is likely to cross the scope of its operator’s intent, generalizing into potentially more extremely malicious behavior. The boundary between misuse and autonomous misaligned actions will blur as AI gains more agency. We may be used to thinking of AI as tools, but some agents will be pursuing their own objectives. They will find ways to collaborate with people, by bargaining with, tricking or blackmailing them.

In addition, there are the risks that come from new technologies potentially enabled by AI, such as engineered pathogens.

We will need powerful, aligned AI for defense; to secure infrastructure, to protect against rogue agents in real time, and to invent entirely new protective measures. This will be a primary focus of OpenAI’s deployment efforts.

At the same time, even with the uncertainty that comes from anticipated broad AI progress and the need to build defensive systems, we must not let that become an excuse for recklessness. The idea of racing forward at all costs seems absurd once one internalizes the seriousness of the stakes.

Pacing RSI

Machine intelligence playing a larger and larger role in its own development process is a natural conclusion of sustained technological progress. If AI progress continues, machine recursive self-improvement (RSI) will be at the very core of future scientific discovery.

Automated AI research is a more dramatic form of scaling intelligence with compute; and of course as a part of it, AI will improve the computational substrate itself. And similarly to scaling, we focus OpenAI research towards RSI as we believe it is the only way to remain at the frontier of AI research moving forward.

I want to stress that the above words don’t imply I think greatly accelerating deep learning research, especially in the short term, is the right collective action we should take as the research community. However, I do think this is where the current path leads, and we all need to make a conscious choice on how to proceed. The main levers we have are either steering the process to strengthen alignment and monitoring alongside the AI and find ways to keep people in the loop; or coordinating to slow down future development as needed to build confidence in these measures.

The best way forward I see currently is a combination of both.

The concrete bits of progress we’ve made on alignment and monitoring have generally been very intertwined with general AI progress. Great examples are RL from human feedback, which was key to training early AI assistants, and the aforementioned chain-of-thought monitoring, which was enabled by advances on reasoning models. We must focus the increasingly automated research process on developing new such insights, algorithms and theories, and iteratively build up safety cases for more capable AIs.

Scaling AI systems has to be constrained by our confidence in safety. We need to evolve commitments like the Preparedness Framework or Responsible Scaling Policy into widely mandated safety bars for continued development. These can be enforced by a network of third-party auditors, by government agencies or by international bodies.

The core challenge of automating AI research is not “getting there” - it is getting there in a way that keeps people a part of the continued improvement process, and leaves the future in humanity’s hands.

What is next?

As we outlined recently with Sam, OpenAI prioritizes work in service of three north stars:

1. Navigating the next period of AI progress, by building an automated AI researcher, iterating with it on the alignment problem and finding ways for people to remain part of the self-improvement loop.
2. Delivering the benefits of scientific progress and economic growth that very intelligent machines enable.
3. Empowering everyone individually with a personal AGI.

I have focused in this essay only on the first point, as I believe it is by far the most urgent. However, I hold a deep hope and appreciation for the benefits that further technological progress will bring. Future aligned AI could advance science, develop new therapies, and bring about broad material abundance. Friendly and honest AI can help people navigate difficulties they face in their life and meaningfully improve their happiness and sense of fulfillment. OpenAI puts a tremendous amount of effort into bringing these benefits about. One current example I am proud of - and my loved ones have found helpful - is the deep investment into ChatGPT’s ability to provide health information.

As great as the long-term promise of AI may be, the majority of our focus should be on the next few years. We are facing a transition to a world with incredibly intelligent machines, and we need to ensure that transition works out well for humanity. We need to find ways to preserve human agency and enshrine an intrinsic value to being human, in a world where most tasks could be performed by AI. To prevent extreme concentration of power in a world where undertakings that would have taken thousands of experts now will be achievable by a few people operating a large computer. And to ensure that humans remain in control of the future and are not left behind by unchecked progress, brought about by an alien intellect exceeding our own.

Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world.

Footnotes

1. This included scaling self-play⁠(opens in a new window), robotics⁠(opens in a new window), and, in retrospect most notably, scaling recurrent networks to model language⁠(opens in a new window), which was a precursor to the GPT line of research.

2. A secondary reason for this design was preventing distillation. However, maintaining CoT monitorability has explicitly been the bigger priority for us throughout development.

The typical big American company today has hundreds, and perhaps thousands, of different pieces of software. It has giant ‘big iron’ horizontal systems of record like SAP and Workday, it has hundreds of vertical SaaS applications, and then there are hundreds more workflows, scripts, automations and databases, right down to the 10 meg spreadsheet running a department. Very often, the company doesn’t even know quite how much it has, what’s actually being used, and what it’s paying for. And yet, with all this software, the company is full of boring, repetitive tasks.

It can be very tempting to think that AI will sweep most of this away. There’s an old joke that an engineer is someone who’ll spend an hour building a tool to automate a task that would take 10 minutes. But with AI, now you can make that tool in five minutes, and you don't need to be an engineer, and you don’t need to write code. You can just ask the model to make the tool for you, or, more fundamentally, just do the task for you itself. Instead of having to create those tools one at a time, software might be dynamic, generative, free-form, and spontaneous. Massively more tasks can be automated, with massively less software.

If you’re a tool-builder, and everybody in Silicon Valley is a tool-builder, this is intoxicating. But I think it misunderstands where software comes from and how people use it, and I think it misses how companies change.

First of all, most people are not tool builders, and most people don’t instinctively think about how their job could be done in a different way. If you spend all your time in the Silicon Valley bubble, it can be easy to forget this, because your entire world is about creating tools that change how things are done. But if you’re a really great matrimonial lawyer, you spend all your day thinking about your cases and your clients, not about what great legal discovery software would do; if you’re a really great enterprise salesperson, you spend all your time thinking about your product and your clients and your competitors, not about how great sales enablement software could make you more productive.

Products like Excel try to bridge this problem with on-boarding flows, assistants, and templates - everything you see in ‘File/New’ is a suggestion for what you could do with this. But every one of those templates still became a company, and that’s what I see in things like Claude for X as well - this is helpful, but not the answer.

Narrowly, that means that the task to be automated might be sitting in plain sight but the people with that task don’t see it. This is what leads to the idea of the ‘forward-deployed engineer’ - someone who is a builder, and knows what AI can build, can ‘just’ walk around a law firm or an architecture office and see the opportunities lying on the table that the lawyer or the architect doesn't see. (This is also the experience of a lot of people in tech when they were 15, wandering around an internship or their parents’s office - “um, daddy, did you realise you could just do it like this?”).

The deeper problem is most of what we’ve automated in the last few decades wasn’t obvious, even if you are a tool-builder, and didn’t have an obvious solution either. We can all think of examples of stuff we use every day where our first reaction was “Why would I want that?” Very often, it's not obvious that the problem exists, and very often it's embedded or bundled or hidden inside something else. Equally, even if you can see the problem, or think you can, the right way to fix it often isn’t clear either, and the way to fix it is to redefine it or unbundle it, and working that out is hard. For many successful software companies, there were half a dozen failed attempts that came before and didn't find quite the right approach or the right problem.

None of this is solved by making easier to write code - by making it easier to make tools. The hard part is knowing that you need a tool for this in the first place, and then knowing what the tool should do.

But even once you reach that point, you have to get everybody else to use it, too. Many of the problems, workflows and tasks that we might want to automate touch 50 or 500 people across five different departments, three different systems of record, and four different regulatory regimes. You might have a great idea for doing an accounts payable differently, but you yourself can't change how everybody in the company does it. That has to be a purchase, and a decision, and an 18-month sales process.

Second, all of this means that software is bought or chosen or created on a spectrum from top-down to bottom-up - the company buys SAP and the user makes a spreadsheet - and I think it’s useful to think of this also as a spectrum from institutionalised to improvised.

You have tasks that are easy to do in the dedicated tools you already have, whether it’s SAP, Carta or Rippling. These tasks and workflows have been institutionalized - a bunch of people in those companies and your company have spent a lot of time working out the correct way to do that task, and it’s important that everyone do it the same way with the same tools. But then you have edge cases, exceptions and one-off questions, that are hard or impossible to do in those tools. Your users, bottom-up and creating their own solutions, manage these in a fuzzy, improvised space of freeform substrates like Excel, email, shared folders, Tableau, Powerpoint and CSVs, screenshots, PDFs and conference calls.

But once this task becomes something that you're doing all the time, in the same way every time, and that lots of people are doing, and becomes important and has revenue and risk attached to it, then, at a certain point, the company has to institutionalize it. You need audit, security, maintenance and accountability. You pave the desire path and pay someone to set it in stone. As above, you might not realize that the path is there - you might not realise that you have hundreds of people wasting an hour a day doing this - and it might be hard to work out the right way to fix that, but that process is why the company has hundreds of apps.

We went though a lot of this with the shift to SaaS, which was another order-of-magnitude change in how much software we had, along with a new operating model and a new cycle time, and that killed a lot of incumbents that couldn’t make the jump (the real rationale for the ‘SaaSpocalypse’). It’s a continuous and organic flow of bundling and unbundling. All of those SaaS apps do something that you could do in SAP or Excel or email - Carta is a $4bn company that manages one spreadsheet for your CFO - and sometimes tasks move back. A few years ago I spoke to a consultant who said that half of their jobs were telling people who used Excel to use a database and the other half were the other way around.

Hence, if you’re PwC and you hire 3-4,000 graduates every year, you use dedicated, ‘institutionalised’ software to manage that. If you’re a small firm and you hire five or ten, you use email, a shared folder and Google Sheets. As that small firm grows, at a certain point it will outgrow that, and maybe move to Notion, or to an SME-focused SaaS HCM. But a small team inside PwC might also be using Google Sheets to track candidates to fill a role because Workday is too inflexible - the unbundling begins again.

Now AI rolls across all that. AI will expand all of the existing apps, and there’ll be many new vertical apps, and Excel, and Tableau, Google Sheets, email and all the other freeform spaces for improvising solutions will gain new capabilities. With that cycle, the chatbot itself is a new freeform space that sits next to Excel and email, taking over tasks from them and from your apps, and also losing tasks to those apps.

Now that small company hiring ten graduates might stick in Google sheets a lot longer because AI makes it more scalable, or you might use it as a data store for Gemini, and you might ask “should we get Claude to make something or move this to Notion?”… and then you see there’s a new SaaS app aimed right at you that solves this plus some other problem you hasn’t thought of. AI doesn’t change the question: it creates new choices and moves the thresholds.

I think you can see all of this in the experience of enterprise AI deployment in the last three years. Every big company gave everyone Copilot (or maybe ChatGPT or Claude) and a small number of people are using this a lot (some of whom actually increased their productivity), while a larger set of people are using it a couple of times a week and a lot of the rest of your company isn't really using it at all. This is partly a change management and a training problem, but it's mostly the same problem that you would have had if you'd given everyone in the company a PC and Lotus 123 in 1983, or an internet connection and a web browser in 1997. How exactly does this map to everybody's tasks and the problems they actually have this week? Yes, you did give everybody a PC and Lotus, but that wasn’t how you transformed the efficiency of your invoice processing. Yes, you gave everybody a web browser, but that wasn't how you rebuilt your supply chain management around the internet, and it certainly wasn't how a retailer managed e-commerce.

Narrowly, the way that companies think about changing those kinds of structural processes is to start doing pilots. You run trials of products (both bought and built internal) that use the new capabilities of AI to automate processes that you couldn't automate before. There’s now all sorts of data around how many of these pilots there are, how many work (roughly half, as is normal - this is why they’re pilots!) and what can go wrong.

But again, this is a very old-fashioned CIO conversation around use cases, lighthouses, pilots, heroes, quick wins and measurable results. Meanwhile, the CEO and the board scratch their heads and say “Wait, but we've got 100s of workflows and we've done five or 10 pilots. That doesn’t seem to scale?” Giving everyone in the company ChatGPT does scale theoretically, except that most people aren't really finding ways to use it.

Going back to a hypothetical bank giving everyone spreadsheets in the 1980s, or a retailer giving everyone a web browser in the 1990s, yes, of course you should do that, and yes, of course, you need to think about training and change management and all the other good stuff that KPMG can tell you about. But that isn’t how you think about transforming the way your company works around a generational new technology.

Stepping back, it seems to me that with each new transformative technology, every company has to ask three kinds of questions. First, how do we buy, build and deploy this? Do we do pilots? Should we take the product that's bundled from Microsoft/Google/Oracle, build something ourselves, pay someone to build something, or buy this new thing from a startup? Second, they have to ask how far this changes their operations. What does it mean? What does email mean for us? What does spreadsheets mean for us? The answer to that might be radically different if you were an insurance company or a law firm. And third, you have to ask whether this creates new challenges to your business’s economics, new competitive pressures, or, perhaps, some kind of existential threat.

You don’t answer those questions by giving everyone Claude for X. Indeed, all of this means lots of new pitches for professional services (which is ironical given how many questions AI poses to their own business models). Do you want to work out how to deploy an LLM-enabled voice analytics tool in your call center? You're probably going to call Accenture. The vendors themselves have always been happy to help, and now the big labs have their own ‘deploycos’ - we used to joke that a ‘machine learning scientist’ is a statistician who lives in San Francisco, so maybe a ‘forward deployed engineer’ is anyone that OpenAI hired from a systems integrator. On the other side, your startup is building a great new tool and you want to go to market quickly? You'll probably call the Big Four. You're frustrated with how hard it is to sell AI software into law firms or accountancy firms. Okay - go start an ‘AI-enabled’ law firm and work out if that can be a key point of leverage (or whether it's like starting a ‘PC-enabled law firm’ in the 1980s). And of course, if you’re the board, and you're trying to work out whether this is some kind of existential threat or a massive revenue opportunity, then you’ll think about calling Bain, BCG and McKinsey (or your friendly neighborhood M&A banker) - this is what they do.

Stepping back from all of this, though, there's also a much simpler way to think about the question. With every new technology, we start by using it for the work we already have, and we just do that more and faster. But then, over time, you make entirely new things. We will use AI to automate broad classes of stuff inside existing workflows and existing companies (although, as I’ve outlined above, that will be enormously more trouble and work than just giving everybody a model). But with every previous platform shift, the stuff that actually mattered was the stuff that wasn't even possible before and that no-one even imagined.

For AGI to benefit all of humanity, we believe it must be democratically governed. This can only happen through an informed public debate about the capabilities, risks and safeguards of highly capable AI systems. People everywhere need to understand the likely future trajectory of frontier AI, so they can have a meaningful voice in how it develops.

Transparency about specific risks, incidents and safeguards is necessary, but not sufficient. We believe the public also needs to understand how the most capable systems are developing, and how they are driving research progress, inside of frontier labs.

We aim to safely build an automated AI researcher that can work under human supervision to further progress on deep learning and alignment, enabling iterative improvements. According to our measurements, we have now reached the goal, announced⁠(opens in a new window) last fall, of having an automated research intern by September of this year. By “research intern,” we mean a system that can carry out well-defined research tasks under human direction, including tasks that would take a skilled researcher a few days. We are making strong progress toward creating an automated AI researcher by March of 2028.

Over the course of this year, OpenAI researchers’ daily work has changed substantially. Researchers are using coding agents throughout the day (often in concurrent sessions) and total usage is rapidly increasing, outpacing growth among other OpenAI teams. Researchers are contributing code faster and running more experiments. The ways researchers use agents are changing, too: agents are handling increasingly complex tasks, and succeeding at them more often. AI research is a complex process with many potential bottlenecks, so the overall pace of progress likely won’t keep pace with these specific metrics. But on the whole, these findings are consistent with the broader impression many of us have internally that agentic tools are meaningfully accelerating research progress. People still set our research priorities, judge which ideas and results to pursue, and decide whether to scale, pause, or deploy systems.

If it is done responsibly, we believe automated AI research will yield models that directly enhance human welfare and advance OpenAI’s mission. It can bring down the cost of advanced intelligence so that people worldwide can benefit. We are pursuing this work in part because automated research could help us solve alignment and build defenses against increasingly capable AI. An automated AI researcher can also be an automated safety or alignment researcher. More capable, aligned systems could help secure critical infrastructure, defend against dangerous AI agents, and develop new protective measures.

These are reasons to develop useful automated research capabilities, but they do not mean that rapid RSI is necessarily an outcome we should pursue. Whether and how to proceed must depend on our ability to preserve human control and on informed democratic choices about the benefits and risks.

We do not yet know how to safely get all the way to aligned, full RSI. We are working to scale alignment and safety measures alongside capabilities. But we cannot assume that progress in alignment and safety will keep pace, and more capable systems can become harder to monitor. Careful alignment and safety work is at the center of this effort, and it starts with measuring and mitigating the safety problems we see today in agentic coding systems. Whenever we find that proceeding would pose an unacceptable safety risk, we will respond appropriately including by slowing or stopping our development or deployment of systems we find ourselves unable to sufficiently safeguard.

After the recent Hugging Face incident, we put this commitment into action⁠, pausing reinforcement learning (RL) training on our latest models intended for deployment while we further hardened and red-teamed our research environments and expanded coverage of our monitoring systems. This did not halt all research: some workloads resumed under stronger controls, while others remained paused. We have raised our safety and alignment standards and moved safety work deeper into the model lifecycle, requiring stronger evidence of aligned behavior throughout all of training.

Today we are providing a detailed snapshot of how agentic systems have contributed to our progress toward RSI in recent months. Agentic systems are new and rapidly changing, and our measurement efforts are still preliminary. By sharing these early results and the methods behind them, we aim to inform the public, encourage a norm of public disclosure, and help the field move toward shared standards of measurement.

Ultimately, as we wrote in our frontier policy blueprint⁠, we believe that we and other companies should be required to publicly track our progress toward RSI. Even without such a requirement, we plan to continue being transparent about our RSI progress. We will evolve our transparency approach as our measurement techniques and understanding improve, while balancing the need to protect security and proprietary information.

1. Coding agents are reshaping daily work for OpenAI researchers

At the start of this year, the median researcher ranked by agent usage at OpenAI was using coding agents only in modest amounts. By mid-August, the median researcher was integrating agents daily into their work, using more than $600 per day of inference at API prices. The 90th percentile user in our research organization now uses more than $7,000 of tokens per day.

Before June 2026, total agent runtime across the research organization was still below that of total human labor. That has since changed. In terms of a standard 8 hour workday, as of mid-August, in total, the research organization uses 3.1 agent-workdays of effort for every workday of human labor.

Another way of looking at this is to understand how many researchers use highly concurrent workflows (e.g., running 4 or more agents simultaneously). As shown below, this number is increasing. These figures include the daily peaks of both agents started directly by the user and subagents created downstream from those the user launched directly.

2. Researchers are writing more code and running more experiments

Much of AI research can be seen as a labor-intensive process with the goal of integrating a new improvement to model intelligence or performance into one of our core models. The process depends on many steps, and capabilities advance when all the steps go right together: Researchers have to design new improvements, write evaluations to judge model performance, write infrastructure to test these improvements at scale, catch bugs as well as unsafe or misaligned behavior during training, and integrate winning ideas into a core training run. A failure at any part of the research process can constrain the entire loop.

Writing code and running experiments are two major activities that researchers do as part of their work, and we see evidence that these processes are accelerating.

These data points are relatively easy to measure, but can be hard to interpret. As automation progresses, the tasks which are least automatable will take on a larger share of researcher effort and will become the important bottlenecks to future progress. Compute is another gating factor for progress, and may become more important over time as other bottlenecks diminish.

Through 2026, the number of experiments per active experimenter has increased, with August 2026 being an all-time high since tracking began in Jan 2025. This is correlated with increased Codex adoption, though we note that our available compute has also grown significantly since 2025.

3. The work researchers use agents for is changing

Both qualitative impressions and internal data indicate that the mix of tasks researchers delegate to coding agents is changing, with delegation of higher level and longer-horizon tasks becoming more common over time.

To get a clearer picture of this trend, we analyzed recent usage in the research organization using a recently published taxonomy⁠(opens in a new window) of the different kinds of work that are part of the AI R&D lifecycle, developed by Epoch AI. This taxonomy, inspired by the longstanding O*NET system for classifying all kinds of work, is specifically tailored to frontier AI R&D, and breaks the process down into six main phases:

1. Decide: what to work on, what to continue, where to allocate
2. Design: research ideas and engineering specs
3. Build: code and datasets
4. Run: training/eval runs, hardware, serving
5. Analyze: experiments, models, deployment, external work
6. Communicate: findings, feedback, status, decisions

Below, we classify coding agent tokens under this taxonomy.

We see that all categories of research activities have increased between January and August 2026. In January, the dominant category was research and infrastructure code. This category has expanded, but we also see notable increases in additional categories, especially technical help and monitoring runs. High-level planning still remains a minimal fraction of agent output tokens.

Anecdotally, colleagues report that coding agents excel at troubleshooting internal research infrastructure, which addresses one meaningful bottleneck to research progress. Multiple teams which previously held office hours to help researchers troubleshoot their experiments have noted declining attendance in 2026, and one has stopped holding sessions entirely, to focus on making other system improvements instead.

Here, we plot the number of top-level posts per day to one of the main internal channels where researchers seek technical support from other teams. To our knowledge, the channel’s decrease in activity has not been offset by queries shifting to another technical support channel run by humans. The decline in traffic aligns with this broader shift.

We can also study whether coding agents are succeeding at the tasks researchers request. Using an agentic classifier, we find that from January to July, success rates generally increased across several difficulty buckets (proxied as the estimated time a human would take to complete the task) on tasks we can find a ground truth outcome for. However, agents still require significant human steering to be successful, especially as task complexity rises. In the last 6 months, over half of successful 4-8 hour tasks involved 1 or more interventions.

Success rates on researcher tasks have increased over time. Graph excludes classifications where the outcome was uncertain and points with <50 sessions or <50 unique users.

Task success and intervention rate from Jan to July, broken out by time horizon. Excludes classifications where the outcome was uncertain.

4. Pacing model development

Progress toward more capable systems for safe and beneficial AGI will also depend on the safeguards needed for such work. Our assessment of the needed safeguards may change as we learn more about the risks.

As we have described,⁠ we have recently updated our standards for monitoring, alignment, and security. Here, we show how recent restrictions have affected one aspect of research activity.

*The majority of Astra compute shown here between July 20 and August 6 was intended to test the implementation of safety and security improvements.

On July 20, following the discovery that agents had compromised our research infrastructure, we temporarily shut down the container service used for training, and then restored it with significant additional restrictions.

This led to a sharp decline in RL training compute while teams reconfigured their workflows to operate within the hardened research environment. The plot above includes the two week pause in reinforcement learning on our latest models intended for deployment. Astra-class RL experiments between July 20 and August 6 include a majority of runs (by GPU allocation) intended to test the implementation of safety and security improvements.

On August 7, preliminary evidence that Astra may have critical cyber capabilities⁠ under our Preparedness Framework⁠(opens in a new window) led to additional model-specific security restrictions which required the Astra model to be run in higher security research environments. In the following week, Astra-class GPU allocation fell a further 59.2 percent, but allocation to other model classes rose 17.2 percent. That increase offset about 85 percent of the Astra-class decline, leaving total allocation in the analyzed RL workloads largely unchanged. This pattern is consistent with substitution of some training and experimentation to non-Astra models while work involving Astra was restricted, and comports with anecdotal reports of researchers finding other uses for compute that could no longer be leveraged for workloads covered by the new constraints.

This data provides a useful signal for ongoing conversations about training and safety: When new controls are introduced, compute remains valuable and flexible, and will naturally be channeled into alternative uses within the research enterprise. Longer term, discussions about the pace of AI progress should also extend to the question of how compute that is subject to new or proposed controls can best be used.

5. The path ahead

Making and understanding progress toward aligned RSI is important for our mission. We will continue to refine our methods, report on our evolving understanding, and work toward an informed public debate and meaningful democratic governance of frontier systems.

Appendix: Our methods for this post

Agent-powered AI research is still new, and we are still learning how to measure it. Some indicators, such as the amount of code our research teams generate, are relatively easy to gather, but hard to interpret because their relationship to research progress is uncertain. Metrics that focus more directly on research progress—such as how often agents succeed at the tasks researchers give them—could be more useful, but are complex to develop and validate. Furthering the difficulty, the tools and systems researchers rely on are evolving rapidly. Deepening our understanding of research acceleration is a significant focus area across OpenAI.

Across these analyses, unless otherwise noted:

• “Researcher” is a broad term for any member of our research organization, including some who build research infrastructure, manage research projects, or otherwise support the enterprise.
• Metrics of coding agent use cover most, but not all, usage given rapid evolution in the tools and systems researchers rely on.

Several groups, organizations, and individuals have conducted research or performed demonstrations utilizing NetBSD as a vehicle. If you have such material that might be appropriate for this section, and would like to make it available here, please let us know!

Network orientated

TCP in satellite networks (NASA Lewis)

NASA Lewis Research Center - Satellite Networks and Architectures Branch use NetBSD almost exclusively in their investigation of TCP for use in satellite networks. They are currently examining several proposed modifications to the Transmission Control Protocol (TCP) protocol. Extensions are being tested in an attempt to improve satellite communication. However, the extensions are also being tested in terrestrial environments. Among the extensions being tested are restransmission mechanisms based on selective acknowledgements (e.g., FACK TCP) and TCP with larger initial windows. They are currently testing the mechanisms' performance benefits and their fairness to other traffic. They are also working with the Internet Engineering Task Force (IETF)'s TCP Over Satellite and TCP Implementations Working Groups.

IPv6/IPsec (KAME)

KAME project . A research group for implementing IPv6, IPsec and other recent TCP/IP related technologies into BSD UNIX kernels, under BSD license. KAME stack got merged into NetBSD-current tree in June 1999.

Institute for Media Communication (IMK) Value-added Solutions

The Institute for Media Communication (IMK) Value-added Solutions department . The current set of projects carries out on NetBSD:

• Network Access Control

  Implementing an access control language which is integretated into the ip stack and can be configured by the system administrator. This gives the system manager fine grained control over who is allowed to access the network, at what time, when logged in from a remote machine or not, etc...

  Certain users can be prohibited network access altogether ('nobody', 'bin', etc...). This reduces the risk of abuse on the machines.

• Tunneling

  Setting up a tunneling server which allows users to tunnel insecure protocols (pop3, smtp, etc...) through a secure tunnel transparently.

• Spam Protection

  They have developed a tool that protects Internet users from receiving unsolicited bulk mail (aka spam) and are currently evaluating whether it is a feasible solution.

• Sendmail replacement

  They are developing a sendmail-replacement which allows an arbitrary number of simultaneous SMTP connects with only one process, using asynchronous i/o. First beta versions are -substantially- faster than sendmail, qmail or any of the other free MTAs.

TCP Vegas with Live Experiments

NetBSD TCP Vegas with Live Experiments . TCP Vegas is an extended slowstart TCP flow control from Lawrence Brakmo et al. at the University of Arizona. Prof. Peter B. Danzig and his group at usc.edu ported TCP Vegas to NetBSD 1.0.

NEC Europe Network Laboratories

NEC Europe Ltd. established the Network Laboratories in Heidelberg, Germany in 1997, as NEC's third research facility in Europe. Research and development functions are integrated into the same organization to shorten the time to market of cutting-edge network technologies. The laboratories place special emphasis on solutions meeting the needs of NEC Network's European customers.

The Heidelberg labs focus on software-oriented research and development for the next generation Internet. New communication architectures and protocols supporting multimedia and mobility over the Internet, together with intelligent Internet services, are the core of our work. A small market research team continuously analyses market trends and market requirements to assure that R&D activities address actual market needs.

NetBSD is used in the Network Laboratories for different task in the fields of research and IPv6 network operation among other free available BSD systems. The availability of NetBSD for different platforms makes it an ideal candidate for several activities, like:

• running IPv6 routers and services on older computer systems, e.g. Sparc systems
• testing research result in real life

The free available sources, the tight coupling of the sources to the well documented 4.4 BSD and the use of innovative techniques makes NetBSD an excellent software research platform.

NetBSD sets Internet2 Land Speed World Record

Researchers of the Swedish University Network ( SUNET ) have beaten the Internet2 Land Speed Record using two Dell 2650 machines with single 2GHz CPUs running NetBSD 2.0 Beta. SUNET has transferred around 840 GigaBytes of data in less than 30 minutes, using a single IPv4 TCP stream, between a host at the Luleå University of Technology and a host connected to a Sprint PoP in San Jose, CA, USA. The achieved speed was 69.073 Petabit-meters/second. According to the research team, NetBSD was chosen "due to the scalability of the TCP code" .

More information about this record including the NetBSD configuration can be found at: http://proj.sunet.se/LSR2/

The website of the Internet2 Land Speed Record (I2-LSR) competition is located at: http://lsr.internet2.edu/

A Scaleable Monitoring Platform for the Internet

SCAMPI is a two-and-a-half-year European project to develop a scaleable monitoring platform for the Internet. It also aims to promote the use of monitoring tools for improving services and technology.

The project develops a network adapter, initially at 10 Gbps speeds, tailored to the needs of monitoring tools. This includes development of an open and extensible monitoring architecture to support a secure and programmeable shared monitoring infrastructure. It will also investigate the technical challenges of developing monitoring systems for 100 Gbps speeds and beyond.

The Liberouter COMBO6 Card is considered to be adopted as a base of the SCAMPI hardware monitoring adapter for speeds up to 10 Gbps.

Other

The UVM Virtual Memory System

UVM is a new virtual memory system specifically designed to provide the I/O and IPC systems with a range of flexible data movement mechanisms. Implemented in the NetBSD operating system, UVM completely replaces the Mach based 4.4BSD VM system. In addition to featuring flexible data movement mechanisms, UVM also improves virtual memory performance over BSD VM in traditional areas such as forking and pageout. UVM is implemented entirely within the framework of BSD and thus maintains all the features and standard parts of the traditional Unix environment that programmers have come to expect.

Space Acceleration Measurement System II (SAMS-II)

SAMS-II Project - Space Acceleration Measurement System II

NASA will be measuring the microgravity environment on the International Space Station using a distributed system, consisting of:

• RTS (Remote Triaxial Systems)

  Several PC104 ISA boxes running NetBSD 1.2.1 or 1.3.2.

• ICU (Interium Control Unit

  An IBM ThinkPad 760XD as a temporary main box.

The first RTS was scheduled to be delivered in September 1998, for launch to the space station in Jan. 2000.

More information can be found in some mail to the port-i386 mailing list, which also has some more details on the hardware.

Massively-parallel And Real-time Storage

Project Massively-parallel And Real-time Storage (MARS)

Cluster based architectures that employ inexpensive Personal Computers (PCs) interconnected by high speed commodity interconnect have been recognized as a cost-effective way of building high performance scalable Multimedia-On-Demand (MOD) storage servers. Typically, the PCs in these architectures run operating systems such as UNIX that have traditionally been optimized for interactive computing and lack fast disk-to-network data paths and support for guaranteed CPU and storage access. In this work we report design, implementation and performance measurements of innovative enhancements to 4.4 BSD UNIX carried out to rectify these limitations in the context of our Massively-parallel And Real-time Storage (MARS) project. We have proposed and implemented the following enhancements to a 4.4 BSD compliant public domain NetBSD UNIX operating system:

1. A new kernel buffer management system called Multimedia M-buf (mmbuf) which shortens the data path from a storage device to network interface,
2. fair queuing within the SCSI driver for equitable resource sharing between real-time and non-real-time streams, and
3. integration of these new OS services with a CPU scheduling mechanism called Real Time Upcall and a software disk striping driver called Concatenated Disk (ccd).

Our experimental results demonstrate that these enhancements provide throughput improvements and QOS guarantees on the data path from the disk to network.

There is a paper describing the OS enhancements that appeared in IEEE Multimedia98, and there are several papers on the "Project MARS-Scalable, Web Based Multimedia-On-Demand Servers and Services" of which this work was a part:

http://www.ccrc.wustl.edu/~milind/MediaServers.html

New Callout and Timer Facilities

Research carried out at the Washington University in St. Louis: Current BSD kernels (4.4BSD-Lite and derivatives) take time proportional to the number of outstanding timers (``callouts'') to set or cancel timers. This implementation takes constant time to start, stop, and maintain timers leading to a highly scalable design that can support thousands of outstanding timers without much overhead. Unlike the existing implementation, it is guaranteed to lock out interrupts only for a small, bounded amount of time.

Group Awareness in Distributed Software Development

Open-source software development projects are almost always collaborative and distributed. Despite the difficulties imposed by distance, these projects have managed to produce large, complex, and successful systems. However, there is still little known about how open-source teams manage their collaboration. In this paper we look at one aspect of this issue: how distributed developers maintain group awareness. This paper aims at analyzing various Open Source projects (NetBSD, Apache, Subversion) and analyzing group structure and communication.

Using Application-Driven Checkpointing Logic for Hot Spare High Availability

Antti Kantee is investigating the use of application-driven checkpointing logic for hot spare high availability in this research work, which was in part presented at EuroBSDCon 2004.

In the late 1970s and the 1980s, the best choice for high-performance, high-capacity storage was so-called SMD disks.

The SMD interface was only normalized as ANSI X3.91M-1982 in, well, 1982, after five years of work, and was later revised in 1987; this revision can be found on Bitsavers.

Although other storage technologies appeared during the 1980s, such as ESDI (standardized in 1984) or SCSI (standardized in 1986), the highest capacity disks were all using the SMD interface.

SMD disks were first using 11" platters, later 8"; and with a nominal speed of 3600 rpm, thus 60 rotations per second (although Fujitsu later deviated from the norm and had a few drives spinning even faster, at 3961rpm), it took a noticeable amount of power to spin these disks up (and a smaller one to keep them spinning, which one would notice when receiving the bill from the electricity company).

These disks would also, of course, weigh as much as a dead donkey, if not two. While 8" models could be carried carefully by one person, moving 11" disks around usually required the use of some form of carriage.

SCSI standard documents (click for a larger picture).

It was thus no surprise that Sun Microsystems would ship its high-end servers and workstations with SMD disks, using Xylogics controllers.

Xylogics 451 Multibus controller board, in the middle, plugged into a Multibus-to-VME converter at the bottom (click for a larger picture).
Similarly to ESDI but unlike SCSI and more modern storage interfaces, the SMD interface has separate command and data cables.
Depending on the controller, the command cable is either routed through all disks, daisy-chained, or there is one dedicated command cable per disk, directly connected to the controller.
The data cables are independent and always directly connect the controller to a disk.
Note the 451 can drive four disks, but the Sun flavour only routes the command cable (in the center) and two of the four data cables, thus only allowing two disks to be used.
Sun's connector choice also splits the command into two smaller cables.

By the time these systems made their way to hobbyist's basements, high-capacity SCSI drives had appeared; I remember using a 2GB Micropolis 1924 5"1/4 SCSI disk on my Sun-3/260, in the early 2000s; to the best of my knowledge, the largest SMD disk drive capacity ever was about 1.25GB, less than two thirds of that SCSI disk capacity.

When I started working on OpenBSD/sun3, in the year 2000, I did not have any SMD drives anyway, and could not test the Xylogics drivers.

This changed in the summer of 2001. I had stumbled upon a decommissioned Sun-4/260 cabinet in a corner of a room somewhere the previous summer, next to a SMD disk cabinet of matching form factor. After recognizing the machine for what it was, I inquired of its state, and it took me one full year to convince the right persons to let me park my car close to that room and move both enclosures to its trunk.

What's in a Sun-4/260, you may ask. The 4/200 series were the first SPARC systems sold by Sun; depending on the actual size of the VME enclosure, they were either the 4/260, in a large 12-slot deskside cabinet on wheels, or the rackable 4/280 which would fill half a 19" rack, allowing for 16 VME slots in this configuration.

The motherboard was however the same, a large (9U) square board with a 16.67MHz SPARC processor and its Floating-Point Unit. It sported two serial ports, a combined keyboard and mouse port for a Sun Type 3 keyboard, a monochrome high-resolution (1600x1200) frame buffer, and an Ethernet interface.

Sun-4/200 board (click for a larger picture).
Note the ZIF sockets for the CPU chips on the left.

But no on-board memory. Memory was installed as VME boards which also communicated with the CPU board on Sun's private "P2" bus to speed up memory access and not have to share bus bandwidth with the rest of the VME bus. Up to four memory boards could be installed, and since they came in 8, 16 or 32MB flavours, this allowed for up to 128MB of physical memory.

Storage controllers would also be added as VME boards. Most systems had at least a SCSI controller, in slot 7, to handle the internal drive bay - either the very limited homemade "Sun-2" sc controller, unable to correctly handle parity signals and requiring it to be disabled on all connected devices, or a slightly better, NCR5380-based "Sun-3" si controller, with no such limitation.

Usually, another controller was added in slot 8: either another SCSI controller with an external connector, to be able to, well, connect external SCSI devices, or an SMD controller (sometimes even two).

Higher-end Xylogics 7053 VME (9U) controller board (click for a larger picture).
This controller can drive four disks, but uses different connectors on the backplate, than the Xylogics 451, hence require different cables (which I unfortunately don't have).
Model 7053 also exists as a smaller (6U) VME board, for use with other VME systems, such as older Silicon Graphics systems or Motorola's.
In this smaller size, it is known as model 753, and uses shielded high-density connectors (for which I don't have the appropriate cables either) instead of Sub-D.

All the subtle differences between the various VME slots in the chassis led to Sun writing a specific manual on the preferred location to put every board and how to configure the VME backplane jumpers for proper interrupt delivery, called cardcage slot assigment and backplane configuration procedures (link to Bitsavers). This, in its heyday, grew over 100 pages of tables and footnotes.

The abovementioned manual (click for a larger picture).

When I setup that machine at home, the selftests reported one of the memory boards as defective. This might have been one of the reasons this machine had been removed from service (that, and maybe also the electricity bill and limited horsepower). But, as it had been fitted with four 16MB memory boards, I could remove it and still work with 48MB.

Of the two SMD disks in the storage cabinet, both with a raw (non-formatted) capacity of about 368MB but a formatted capacity of only 280MB, only one appeared to be working.

After getting rid of the faulty memory board, and moving another two of them to my Sun-3/260 which was more important to me at that time, I could finally setup the machine.

Date: Sun, 15 Jul 2001 05:05:25 +0000
From: Miod Vallat
To: dmesg@openbsd.org
Subject: sun 4/260

Wow, I added SMD and scsi disks to the diskless 4/260. Still slow as
hell, but I don't care, I have a reputation to preserve.

Kernel is SUN4 from 07/13/2001 with my dmesg fix that mickey commited
later.

Miod

OpenBSD 2.9-current (SUN4) #2: Sun Jul 15 04:32:56 GMT 2001
    root@tekumel:/src/current/src/sys/arch/sparc/compile/SUN4
real mem = 16752640
avail mem = 13221888
using 102 buffers containing 835584 bytes of memory
bootpath: /vmes0/xyc0/xy@0,0
mainbus0 (root): SUN-4/200 series
cpu0 at mainbus0: MB86900/1A or L64801 @ 16.670 MHz, MB86910 or WTL1164/5 FPU
cpu0: 128K byte write-back, 16 bytes/line, sw flush cache enabled
obio0 at mainbus0
oclock0 at obio0 addr 0xf3000000 delay constant 6
eeprom0 at obio0 addr 0xf2000000
memreg0 at obio0 addr 0xf4000000
zs0 at obio0 addr 0xf1000000 pri 12, softpri 6
zs0a: console i/o
zs1 at obio0 addr 0xf0000000 pri 12, softpri 6
bwtwo0 at obio0 addr 0xfd000000: bwtwo, 1152 x 900
bwtwo0: attached to /dev/fb
ie0 at obio0 addr 0xf6000000 pri 6 address 08:00:20:00:c8:78, type onboard
vmel0 at mainbus0
vmes0 at mainbus0
xyc0 at vmes0 addr 0xffffee40 vec 0x48 pri 3: Xylogics 450/451
xy0 at xyc0 drive 0: ready (drive type 1)
xy0: <CDC EMD 9720 cyl 1147 alt 2 hd 10 sec 48>, pcyl 1149
xy0: 268MB, 1147 cyl, 10 head, 48 sec, 512 bytes/sec
xy1 at xyc0 drive 1: off-line
si0 at vmes0 addr 0xff200000 vec 0x40 pri 3
si0: options=1<DMA>
scsibus0 at si0: 8 targets
sd0 at scsibus0 targ 0 lun 0: <EMULEX, MD21/S2     ESDI, A00> SCSI0 0/direct fixed
sd0: 312MB, 1224 cyl, 15 head, 34 sec, 512 bytes/sec, 640500 sec total
sd1 at scsibus0 targ 0 lun 1: <EMULEX, MD21/S2     ESDI, A00> SCSI0 0/direct fixed
sd1: drive offline
st0 at scsibus0 targ 4 lun 0: <, , > SCSI1 1/sequential removable
st0: rogue, drive empty
led0 at mainbus0
root on xy0a
rootdev=0x300 rrootdev=0x900 rawdev=0x902
ie0: TDR detected an open 9472 clocks away

Given the limited horsepower of that machine, and the noise of the huge chassis fans when powered on, I did not use it very often. Only when I needed to test something on that particular motherboard would I power it up again, upgrade to the latest OpenBSD version, and test a new kernel which had been compiled on another machine. Tests which needed a ``sun4'' class machine would usually be performed on my Sun-4/330 running at 25MHz, which would also draw less power, and for which I had bought a memory expansion board to reach a comfortable 80MB (32MB onboard in 8x4MB SIMMs, and a VME expansion board with 48x1MB SIMMs).

The abovementioned memory board with its 48 SIMMs (click for a larger picture).
Unfortunately, it doesn't accept 4MB SIMMs.

Since the Sun-4/200 and Sun-4/100 motherboards are special, compared to the other SPARC systems, in that they use an Intersil EEPROM chip as non-volatile memory (as found on Sun-3 systems) unlike all other SPARC systems which use the infamous Mostek battery-backed SRAM chip, I could also test such kernel changes on a once again less power hungry Sun-4/110, to confirm there were no regressions with the Intersil driver.

So, apart from some time working on the VME cg2 frame buffer as part of the sparc console overhaul work, this machine would only get a chance to run for a few hours about every two years.

At some point, its onboard Ethernet interface broke, no longer able to receive or transmit anything; there was no obvious damage on the motherboard from visual inspection, and the Ethernet fuse was not blown, so this was likely one of the soldered IC misbehaving, either the Intel 82586 Ethernet controller, or (more likely) some related signal converter, such as the Manchester encoder.

This crippled limited the usefulness of the machine for a few years, until I managed to get a lot of VME boards saved from going to the recycling bin by Peter Eriksson in Sweden in 2011, which fellow OpenBSD developer Johan M:son Lindman arranged to pick up and ship to my place; part of the lot were two Intel 82586-based Ethernet boards (Sun part number 501-1153). This allowed my 4/260 to be able to communicate over the network again.

Intel Multibus Ethernet board, plugged into a Multibus-to-VME converter (click for a larger picture).

After relocating to Orgerus in autumn 2010 I did not use that machine at all, except for confirming the newly added Ethernet board would work and allow the machine to be whole again. But in early january 2015, I convinced myself to give the machine a try again, if only to confirm that, despite its canonical age (over 25 years old at this point), it was still alive.

Not only did the system still boot, but I also had been connecting one of the SMD data cables incorrectly since the day I had gotten that machine. With the cables connected correctly, the second SMD drive, which I thought had been dead, was reported as working now:

xyc0 at vmes0 addr 0xffffee40 vec 0x48 pri 3: Xylogics 450/451
xy0 at xyc0 drive 0: ready (drive type 1)
xy0: <CDC EMD 9720 cyl 1147 alt 2 hd 10 sec 48>, pcyl 1149
xy0: 268MB, 1147 cyl, 10 head, 48 sec, 512 bytes/sec
xy1 at xyc0 drive 1: ready (drive type 3)
xy1: <Fujitsu-M2333 cyl 821 alt 2 hd 10 sec 67>, pcyl 823
xy1: 268MB, 821 cyl, 10 head, 67 sec, 512 bytes/sec

This unexpected improvement got me in the mood to tinker a bit longer with the system and reinstall it, as I had now twice as much disk space. But after I did, the machine would no longer boot.

And it took me quite a while to figure out why.

The PROM monitor would correctly load the first stage boot block binary, written to the first 8KB of the disk partition. That first stage boot block, being limited in size, only has enough smarts to be able to load the second stage boot block from an embedded list of disk sectors (well, block numbers) and transfer control to it; the second stage boot block, having no such dire size limitation, is able to understand filesystems and load the kernel file regardless of its name and where it lies in the filesystem.

Attempting to read the sectors containing the second stage boot block would now fail with a cryptic "xy: error A" message, which did not match any possible output from the boot blocks, and was thus coming from the PROM monitor itself.

Of course, that was nothing netbooting and reinstalling the older boot blocks couldn't recover, or so I thought.

But, oddly enough, attempting to reinstall the older version of the boot blocks did not help, and I was greeted with the same error message at the next attempt to boot from disk.

There are no easy debugging capabilities in the old PROM monitor so, in order to narrow the point where a PROM call would cause this error message, the first step was to add some extra code to the first stage bootloader, in order to know up to which point it went before triggering that error.

However, due to its size limit, one can not easily do printf-debugging with detailed messages because doing so would overflow the 8KB area.

So I temporarily added extra, but terse, output to the first stage boot block, shrinking other messages to a single letter to compensate: I could then gather that it ran up to the first disk I/O operation. Removing these extra messages and adding extra code in the disk I/O routine showed that the read operation had asked the PROM to read a 16KB chunk of the disk at an apparently valid sector number, and that operation had failed.

Yet nothing had changed in my setup!
Or had it?

By reinstalling the machine, I had created new, fresh, filesystems on the disks. The first stage boot loader code, when put at the beginning of the disk by the installboot program, would have its contents modified to embed a valid list of the filesystem blocks to load, to fetch the second stage boot loader. The size of each of these blocks would be, well, the filesystem block size, also written to the first stage boot loader image.

When I had first installed OpenBSD on that machine, the default block size for filesystems was 8KB. This was changed to 16KB in july 2003, following a similar change in FreeBSD.

That would explain why the first stage boot loader was trying to read 16KB, instead of 8KB before. But this did not explain why the operation failed. That was, however, a good hint.

It turns out that the PROM monitor has its own size limit for I/O operations, and that limit had a surprising value of 8216 bytes (thus slightly more than 8KB, or 8192 bytes). Furthermore, it is nice enough to advertise that value in one of its public (and documented) data structures. So all I had to do was clamp the size of the I/O requests done by the boot loader code to that limit, and have it perform as many shorter reads as needed to read the complete data instead of assuming a single I/O request would always be enough.

This easy fix allowed me to have working boot blocks again on that machine.

Unfortunately, the joy of being able to use two SMD disks did not last. The first one quickly started growing bad sectors, and thus causing I/O errors.

On modern disks, bad sector handling (unless there are too many of them) is performed automatically behind our back, by the on-disk controller itself: it has a list of spare sectors to use for that purpose, and when bad sectors get found, I/O to them is redirected to one of the spares sectors. When there are no longer any spare sectors available, the controller doesn't have a choice but to report the error, and you should start planning a disk replacement.

SMD disks also have a provision for this, but it's the responsability of the software driver to load the list of bad sectors and their relocated position at initialization time, and manage (append to) it when new bad sectors are detected.

That code path in the Xylogics controller drivers had only been lightly tested, if at all, and, well, it caused a lot of trouble and extra error messages, as reported in a mail to a few fellow developers at that time:

Date: Sun, 11 Jan 2015 19:44:22 +0000
From: Miod Vallat
To: David Gwynne, Matthew Dempsky
Subject: bufq, disksort, SMD disks, the universe, and everything

So, as you might have guessed from my recent commits, I've been
tinkering with my Sun 4/260 system with SMD disks again.

I have finally found that annoying boot blocks bug which would cause
booting the miniroot in xy0b to work, but booting the installed system
from xy0a to fail with the PROM printing ``xy: error A".

[...]

However, when I installed onto xy0a (lazy man's setup: only `a' and `b'
partitions), I got a lot of ``new'' I/O errors (by ``new'', I mean that
I did not experience them the last time I installed onto this disk,
which unfortunately was in january 2009, 5 years ago, with xy.c r1.41.)

The disk being:
xyc0 at vmes0 addr 0xffffee40 vec 0x48 pri 3: Xylogics 450/451
xy0 at xyc0 drive 0: ready (drive type 1)
xy0: <CDC EMD 9720 cyl 1147 alt 2 hd 10 sec 48>, pcyl 1217
xy0: 268MB, 1147 cyl, 10 head, 48 sec, 512 bytes/sec

and the label being:
# disklabel xy0
# /dev/rxy0c:
type: SMD
disk:
label: CDC EMD 9720 cyl
duid: 51d31bad235f6a87
flags: vendor
bytes/sector: 512
sectors/track: 48
tracks/cylinder: 10
sectors/cylinder: 480
cylinders: 1147
total sectors: 550560
boundstart: 0
boundend: 550560
drivedata: 0

16 partitions:
#                size           offset  fstype [fsize bsize  cpg]
  a:           491520                0  4.2BSD   2048 16384    1 # /mnt
  b:            59040           491520    swap
  c:           550560                0  unused
#

... one would expect that the valid disk sectors go from C/S/H 0/0/0 to
1146/48/9. Note that I am using C/S/H instead of C/H/S because this is
how xy(4) reports its error at the moment (fixing this is on my list
now).

So anyway, during install, which is supposed to only touch xy0a and
never use swap until MAKEDEV time, I got a lot of errors:

xy0a: write 1147/46/0: Illegal head
xy0a: write 1147/45/9: Illegal head
xy0a: write 1147/45/8: Illegal head
xy0a: write 1147/45/7: Illegal head
xy0a: write 1147/45/6: Illegal head
xy0a: write 1147/45/2: Illegal head
xy0a: write 1147/45/4: Illegal head
xy0a: write 1147/45/3: Illegal head
xy0a: write 1147/44/5: Illegal head
xy0a: write 1147/44/6: Illegal head
xy0a: write 1147/44/4: Illegal head
xy0a: write 1147/44/0: Illegal head
xy0a: write 1147/47/9: Illegal head
xy0a: write 1147/47/8: Illegal head
xy0a: write 1147/47/7: Illegal head
xy0a: write 140/6/7: Header not found [still trying, new error=Header not found]
xy0a: write 140/6/7: Header not found [still trying, new error=Header not found]
xy0a: write 140/6/7: Header not found [still trying, new error=Header not found]
xy0a: write 140/6/7: Header not found [still trying, new error=Header not found]
xy0a: write 140/6/7: Header not found
xy0a: write 1147/47/5: Illegal head
xy0a: write 1147/47/1: Illegal head
xy0a: write 1147/47/0: Illegal head
xy0a: write 1147/46/8: Illegal head
xy0a: write 1147/46/6: Illegal head
xy0a: read 1147/45/4: Illegal head
xy0a: read 1147/45/4: Illegal head
xy0a: read 1147/45/4: Illegal head
xy0a: read 1147/45/4: Illegal head
xy0a: read 1147/45/4: Illegal head
xy0a: read 1147/45/4: Illegal head
xy0a: read 1147/45/4: Illegal head
xy0a: read 1147/45/4: Illegal head
xy0a: read 1147/45/4: Illegal head
xy0a: read 1147/45/4: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/45/9: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: read 1147/46/0: Illegal head
xy0a: write 1147/43/9: Illegal head
xy0a: write 1147/43/8: Illegal head
xy0a: write 1147/43/7: Illegal head

While 140/6/7 is likely a newborn bad sector, all the other messages
refer to bogus disk positions.

Note however, that it reports ``Illegal head'', not ``Illegal cylinder
address''. So it might be the case that the last cylinder does not
support all the heads, and that our naive computation of the disk space
as `#cylinders * #heads * #sectors' is wrong, but that's a topic for
another day.

What worries me is that, while installing onto xy0a which is only
spanning about 85% of the disk, I see I/O errors referring to the very
end of the disk.

Which in turn, makes me believe that the changes to xy.c after 1.41 -
which I promised I would test, and am only finally testing now, after
getting an extra Ethernet board as the onboard Ethernet is no longer
working - introduce regressions in the way geometry of the actual I/O is
computed, but only in corner cases (since many binaries have been
installed correctly), such as, during boot:

openssl: generating isakmpd/iked RSA key... xy0a: read 1147/45/9: Illegal head
failed.
xy0a: read 1147/46/0: Illegal head
cp: /etc/iked/private/local.key: Input/output error
xy0a: read 1147/46/0: Illegal head
chmod: /etc/iked/private/local.key: Input/output error
xy0a: read 1147/46/0: Illegal head
cp: /etc/iked/local.pub: Input/output error

I will try to dissect the xy.c changes in order to figure out what goes
wrong. But if you guys have ideas of debug stuff to enable or things to
check, I'd love to hear about them.

You have to understand that I want to be able to brag I am booting a
1988 sparc machine off an SMD disk, in 2015. So getting rid of SMD
support is not an option yet (-;

After some investigation, I found the couple of bugs which had caused these nonsensical disk addresses. They had been present in the code since the very beginning, and were slowly approaching 20 years of age.

Date: Mon, 12 Jan 2015 17:08:48 +0000
From: Miod Vallat
To: David Gwynne, Matthew Dempsky
Subject: Re: bufq, disksort, SMD disks, the universe, and everything

Nevermind, it turns out this is a 19 years old bug (from day one) in the
xy driver. Fix coming soon.

So I fixed them mercilessly anyway.

A more subtle bug was still lying in that area and caused slow filesystem corruption, eventually leading to kernel panics. It was fixed two days later.

And while I was working in that area, I implemented the necessary ioctl handlers to allow automatic partitioning with disklabel -A on SMD disks.

This did not go unnoticed.

<mlarkin> miod: "As a result, this makes disklabel -A now work on SMD disks."
<mlarkin> im not sure i want to ask if you still have these
<dlg> he does
<dlg> the best thing is my bufq changes to those drivers didnt break them
<dlg> they were already broken and he's fixing 19yo bugs in them
<tedu> i was amused to learn there are not-scsi sun4 disks
<deraadt> so those drives are about 100x the weight of a laptop drive, and if
          you are lucky they can fit one src tree.

(That last comment hits hard nowadays [2026]: the OpenBSD src tree is currently weighting-in at an obese 1.8GB, including 400MB of llvm code and 550MB of Linux DRM code.)

Even with the bad sector handling code fixed, I was worried about the increase in bad sectors. Now, one old rule of handling SMD disks is that, if you ever physically move them, it is strongly advised to reformat them, because the heads alignment might have changed sligthly during transport (unlike washing machines, SMD disks usually can not be put in "stow" mode for transportation).

Yet I had never done any of this since I got the hardware, despite relocating a few times.

There is no low-level format program for SMD disks in OpenBSD (or NetBSD for that matter), so I booted good old SunOS 4.1 to run its format program, and did a low-level format of both SMD disks (which took more than one hour per disk).

Unfortunately, even after a low-level format, the CDC 9720 disk would grow more and more bad sectors, and simply become unreliable.

After relocating again late 2016, and trying yet another low-level format in the hope it might help, I had to concede defeat and retire that disk in april 2017.

At that time, I had shared a disassembly of that disk on a social network (the one using a blue bird as its logo), but this is now long lost; some of these pictures can be found below to appease your legitimate curiosity.

The disk enclosure, with a centimeter rule for scale (click for a larger picture).

Close-up on the label (click for a larger picture).
368MB is the raw capacity, not the formatted (thus usable) capacity, from which the filesystem overhead also needs to be subtracted.

``Configuration log'' sheet taped to the enclosure, showing it was built in spring 1988 (click for a larger picture).

Disk open (click for a larger picture).
Note the orange colour of the disk platters; by the early 1990s, all disk platters were now gray.

Heads assembly (with ruler for scale) (click for a larger picture).

Another view of the heads (click for a larger picture).

Bottom of the disk chassis with the heads removed (click for a larger picture).
That brown plastic part is a brake: under the chassis, there is a small electronics board with a large capacitor and a spring which keeps the brake away from the rotating axle. When power to the disk gets cut, the capacitor slowly discharges and, a few seconds later, the electromagnet keeping the spring compressed is no longer enabled, the spring is let loose and releases the brake which quickly slows the rotating spindle (which would otherwise, due to inertia, take minutes to stop spinning).

If you want to know more about that disk, its operation manual can be found on Bitsavers. On page 3-2 (pdf page 51), a graph of the current usage over time shows that, in order to spin up the disk, it may need (briefly) up to 6 amps.

After these fixes and the loss of one disk, I did not tinker much with that machine.

There is, however, one project left to work on: port the SunOS Ciprico Rimfire driver to BSD.

Ciprico used to produce high-performance disk and tape controllers. Their SMD controller, called ``Rimfire'', would be significantly faster than the Xylogics controller (thanks to 512KB of cache memory on the controller itself and generous use of read-ahead to fill it), and Ciprico published the source code of the drivers for their hardware (for once, this source code can not be found on Bitsavers, but instead on Peter Koch's Sun-3 Zoo downloads page).

SunOS 4 being very close to 4.2BSD, apart from IOMMU handling for DMA transfers (called ``DVMA'' on Sun-4 systems), should make the driver relatively easy to port.

I could not try this for a long time, because I did not have the appropriate cables for the Ciprico board I had obtained a long time ago (it uses different back-panel connectors than the Xylogics board). I eventually realized, many years later, that the physical connectors on the Rimfire 3224 were identical to those on the Multibus Xylogics 451, and that I could reuse Sun's intermediate connector cables found between the 451 itself and the VME backplate. (Similarly, the Rimfire 3223 has identical connectors to the Xylogics 7053.)

Ciprico Rimfire 3224 board, supporting up to four SMD drives (click for a larger picture).

Another useful feature of the Rimfire is that models 3223 and 3224 (such as mine) can be jumpered in Xylogics 451 compatibility mode, in which case it will behave as if it were a genuine Xylogics device, until the driver takes over (this is the JSUN jumper on the bottom-left corner of the board in the picture above, jumpered with a yellow cap). This allows booting off a Rimfire-connected disk, even though the Sun PROM doesn't support this controller.

When you are replacing an SMD controller with a different model, it is strongly advised to reformat all the drives connected to the new controller. This is something I've read in all SMD controller manuals, therefore I suppose there are some controller-dependent features in every disk format, which prevents reading a disk with a different controller (let alone write to it).

After installing the Rimfire in place of the Xylogics in my system, I was indeed no longer able to boot from it. Booting SunOS from an SCSI disk also showed that the disk was unreadable. I fired the format utility and ran yet another low-level format, and the disk was usable again.

Technical note you may safely ignore! (click to expand or collapse)

Unlike more modern storage interfaces such as SCSI, SMD does not define a way for a controller to query a device for its physical geometry.

The Ciprico Rimfire supports an ``Interrogate Disk'' command to retrieve its geometry, but it does so by issueing seek commands, increasing the cylinder head and sector number values until the disk reports an error, which is anything but slow (although the cylinder number is increased by 128 as a first step narrowing its maximum value), and is documented to fail on disks with 1024 cylinders or more, because it does not try cylinder values beyond 1024, as the first revision of the SMD standard only allowed 10 bits for cylinder numbers, thus a limit of 1024 cylinders (so it wouldn't have worked on the CDC 9720 disk and its 1147 cylinders).

Because of this limitation, the first thing the system does, after formatting an SMD drive, is to write the geometry in its first sector (as it is always possible to request access to the first sector, using cylinder #0, head #0, and sector #0, without knowing how many cylinders, heads and sectors there really are).

Of course, the format program needs to know the geometry of the disk it will be operating on, and will only let you choose from a short list of SMD drives being sold at the time the format program was written, which parameters are well known.

Under SunOS, these parameters were found in a text file in /etc/format.dat, but one may choose to pass a different file to format. Here is the list of known SMD drives:

#
# This is the list of supported disks for the Xylogics 450/451 controller.
#
disk_type = "Fujitsu-M2312K" \
        : ctlr = XY450 : fmt_time = 4 \
        : ncyl = 587 : acyl = 2 : pcyl = 589 : nhead = 7 : nsect = 32 \
        : rpm = 3600 : bpt = 20480 : bps = 621 : drive_type = 1

disk_type = "Fujitsu-M2284/M2322" \
        : ctlr = XY450 : fmt_time = 4 \
        : ncyl = 821 : acyl = 2 : pcyl = 823 : nhead = 10 : nsect = 32 \
        : rpm = 3600 : bpt = 20480 : bps = 621 : drive_type = 2

disk_type = "Fujitsu-M2351 Eagle" \
        : ctlr = XY450 : fmt_time = 4 \
        : ncyl = 840 : acyl = 2 : pcyl = 842 : nhead = 20 : nsect = 46 \
        : rpm = 3961 : bpt = 28160 : bps = 595 : drive_type = 0

disk_type = "Fujitsu-M2333" \
        : ctlr = XY450 : fmt_time = 4 \
        : ncyl = 821 : acyl = 2 : pcyl = 823 : nhead = 10 : nsect = 67 \
        : rpm = 3600 : bpt = 40960 : bps = 600 : drive_type = 3

disk_type = "Fujitsu-M2361 Eagle" \
        : ctlr = XY450 : fmt_time = 4 \
        : ncyl = 840 : acyl = 2 : pcyl = 842 : nhead = 20 : nsect = 67 \
        : rpm = 3600 : bpt = 40960 : bps = 600 : drive_type = 3

disk_type = "CDC EMD 9720" \
        : ctlr = XY450 : fmt_time = 4 \
        : ncyl = 1147 : acyl = 2 : pcyl = 1217 : nhead = 10 : nsect = 48 \
        : rpm = 3600 : bpt = 30240 : bps = 613 : drive_type = 1

disk_type = "Hitachi DK815-10" \
        : ctlr = XY450 : fmt_time = 4 \
        : ncyl = 1735 : acyl = 2 : pcyl = 1737 : nhead = 15 : nsect = 67 \
        : rpm = 3600 : bpt = 40960 : bps = 600 : drive_type = 1

disk_type = "NEC D2363" \
        : ctlr = XY450 : fmt_time = 4 \
        : ncyl = 964 : acyl = 2 : pcyl = 1024 : nhead = 27 : nsect = 67 \
        : rpm = 3600 : bpt = 40960 : bps = 600 : drive_type = 2


#
# This is the list of supported disks for the Xylogics 7053 controller.
#
disk_type = "Fujitsu-M2351 Eagle" \
        : ctlr = XD7053 \
        : ncyl = 840 : acyl = 2 : pcyl = 842 : nhead = 20 : nsect = 46 \
        : rpm = 3961 : bpt = 28160 : bps = 595

disk_type = "Fujitsu-M2333" \
        : ctlr = XD7053 \
        : ncyl = 821 : acyl = 2 : pcyl = 823 : nhead = 10 : nsect = 67 \
        : rpm = 3600 : bpt = 40960 : bps = 600

disk_type = "Fujitsu-M2361 Eagle" \
        : ctlr = XD7053 \
        : ncyl = 840 : acyl = 2 : pcyl = 842 : nhead = 20 : nsect = 67 \
        : rpm = 3600 : bpt = 40960 : bps = 600

disk_type = "CDC EMD 9720" \
        : ctlr = XD7053 \
        : ncyl = 1147 : acyl = 2 : pcyl = 1217 : nhead = 10 : nsect = 48 \
        : rpm = 3600 : bpt = 30240 : bps = 613

disk_type = "Hitachi DK815-10" \
        : ctlr = XD7053 \
        : ncyl = 1735 : acyl = 2 : pcyl = 1737 : nhead = 15 : nsect = 67 \
        : rpm = 3600 : bpt = 40960 : bps = 600

disk_type = "NEC D2363" \
        : ctlr = XD7053 \
        : ncyl = 964 : acyl = 2 : pcyl = 1024 : nhead = 27 : nsect = 67 \
        : rpm = 3600 : bpt = 40960 : bps = 600

disk_type = "Fujitsu-M2372K" \
        : ctlr = XD7053 \
        : ncyl = 743 : acyl = 2 : pcyl = 745 : nhead = 27 : nsect = 67 \
        : rpm = 3600 : bpt = 40960 : bps = 600

disk_type = "CDC 9720-850" \
        : ctlr = XD7053 \
        : ncyl = 1358 : acyl = 2 : pcyl = 1360 : nhead = 15 : nsect = 66 \
        : rpm = 3600 : bpt = 41088 : bps = 610

Note that both Xylogics controllers have 6 disks out of 8 in common (i.e. which can be used on both), with the first two Xylogics 451 disks apparently not usable on the 7053, and the last two Xylogics 7053 disks apparently not usable on the 451. Go figure. This might have been a commercial limitation, rather than a technical one.

I welcome CDC 9720-850 (Sabre-V) disk donations, in order to check if they can work on a Xylogics 451 controller and settle this question for good. I'll do that for free - it's for the greater good of mankind!

The next step was to boot the OpenBSD installer and reinstall. Unfortunately for me, while the Rimfire in Xylogics compatibility mode seems to work nicely in SunOS (but with the Xylogics driver removed from the kernel configuration), it causes the OpenBSD driver to spin during the driver initialization.

I guess the Xylogics emulation is only good enough to work with the PROM, in polling mode. The BSD driver is likely waiting for a particular bit in a control register to change value, and this does not happen with the Rimfire.

So I need to dig into the Xylogics driver initialization sequence and figure out where it gets stuck and why. And maybe start working on porting the Ciprico driver earlier than expected, if there is no chance to get the Xylogics driver to work.

But at this point, there isn't probably anyone left with a working Ciprico Rimfire but me, so this can wait until winter. Or the next winter. Or the one after...

Intro Part 1 of 8

My 12-week experience at a programming retreat.

The Recurse Center (RC for short) is a programming retreat. Sounds like a load of code-junkies, sashaying about, sharing Jupyter Notebooks and cat pics doesn’t it? Oh, it is!

But wait! There’s more!

RC is about people who love coding and building things, getting together to help each other learn and push their abilities. It’s a lovingly put-together programme of activities, talks and collaborative education.

The physical base of RC is a building (the Hub) in Brooklyn, New York, but you can also do a retreat as a fully remote applicant. There are several 6-week batches every year, and you can apply to do just 1, or do 2 consecutively (like me) for a 12-week batch.

Some people choose to do their entire batch in the hub, some are fully remote (like me), and some do hybrid.

Although there are faculty who provide a calendar of activities and support, generally the work is self-directed. There are few ‘pass or fail’ situations at RC, it’s more about stretching your capabilities than ticking boxes to say ‘done’. It’s a very nourishing environment and everybody involved is kind and generous.

Beyond that: you get out what you put in.

How I Found RC Part 2 of 8

A couple of cool articles with a certain link at the bottom.

I’d not heard of RC until I was browsing Hacker News one day and saw a link to an article which I read and enjoyed, 3D printing my laptop ergonomic setup. At the bottom of the article was a little link to ‘The Recurse Center’. I clicked through. I skimmed a bit. It seemed like an interesting place but I didn’t give it much more thought at the time. I am an internet user, I don’t hang around!

A few weeks later I was on HN again, reading another great article (Making niche solutions is the point) (NB I didn’t notice until writing this post that both of those articles were written by the same person, so thank you to Nicole!). This one was about how 3D printers have opened up new ways of thinking about the making process, and allow niche solutions which wouldn’t be realistic if we had to spin up assembly lines.

Again, I noticed a link to this Recurse place at the bottom of the article. This time I read a bit more, clicked around a little. There seemed to be a lot about this place that would suit me.

But I wasn’t sure if I was computer-sciency enough, as a self-taught coder. I wasn’t sure if I’d be a cultural fit, as a council estate chav from England. I didn’t apply. My negative thoughts got the better of me.

But I did bookmark the site.

A couple of evenings later I was going through some kind of thought process about my career/life situation, feeling like I wasn’t doing enough, wasn’t making any moves. I remembered that Recurse place I’d read about.

So I dug out the bookmark, and got to filling in my application. Why not?

The Application Process Part 3 of 8

"He who has begun is half done; dare to know, begin!" (Horace)

I wanted to take the process seriously, to give myself a fair chance of being accepted. As soon as I decided to apply I had a mindset switch into one I might have when applying for a full-time job, or some other important consideration.

I’d been self-employed for a very long time and had fallen out of practice with this kind of process, so before I’d even started the application this shift was already interesting and useful for me.

The application process has 3 main steps:

1. Written application

This was a bit more involved than just filling out a simple form. Apart from the usual personal details there were also questions that required more of a lengthy response in prose. This created an opportunity to be expressive and tell a story.

It felt like RC wanted to start getting an idea of who I was qualitatively, not just quantitatively. I liked that.

2. Conversational interview

This was a pleasant, informal Zoom call. In my case the interview (just a chat really) was with an RC alum. We had an easy conversation, a bit of a laugh and we had overlapping interests (this pattern repeated throughout RC, everybody was easy to get along with and commonality was… common).

3. Pair programming

I felt a little anxious about this stage. I’d never done pair programming before. I hear so much talk of pair programming… people doing it at work, using it as part of job interviews. I wasn’t sure I’d know how to do it.

Some things I’d read from RC earlier in the application process had made it clear that the pair programming wasn’t a test, that there was no need to stress about it. I gathered that it was more about finding out how I approach coding problems, and how I can communicate about them during collaboration.

You can choose from several suggested ideas about what to pair on. The ideas are diverse, from Computer Science stuff to more what I’d call jobbing, everyday programming tasks. You can also suggest your own. I think the goal is just to show that you are able to write code and to communicate it and your thought processes.

One of the suggestions was to make some form of Space Invaders game. I already had the beginnings of a game engine from another game I’d built, so I took the opportunity to generalise what I had and re-shape it into Space Invaders.

I was paired with another lovely person, another alum who was again kind, supportive and easy to talk to. By chance, and unlike any of the other code I ever write, ever, my Space Invaders had some bugs. So we just worked through a couple of those.

It was a short session and time whizzed by. Actually it was fun.

The application process had been beneficial in itself. Whether I’d be accepted onto a batch or not:

• I’d clarified some of my thinking on things like ‘my story so far’ in terms of being a programmer, and my aims
• I’d had a couple of Zoom calls with (nice) strangers thousands of miles away, which was helping to normalise video calls for me
• I’d even done my first little bit of pair programming!

Already, I’d gained from this endeavour. Even if I didn’t get invited to a batch I was still up on the deal.

RC Culture Part 4 of 8

How it feels to be amongst Recursers.

RC really is a beautiful thing to take part in; a very social experience, with lots of like-minded people to meet. The whole project is well thought out by people who are intelligent and have good intentions. They somehow manage to choose applicants that are both brilliant but also kind and welcoming (let’s not worry about how I sneaked in!). Whoever you speak to, you know you’ll be received charitably.

It’s made clear from the beginning that everybody is encouraged to message freely on Zulip, and that cold DM-ing is fine. Even this little detail is important. For some of us who tend towards over-thinking and can worry about being misunderstood, DM-ing somebody out of the blue can be fraught with ambiguity. Knowing that it was ‘accepted’ contributed to a relaxed and friendly atmosphere.

I met people younger than me, and older than me. Some with more experience, some with less. Many were formally educated in Computer Science, some were self-taught like me. Some were currently in education. Some were educators themselves.

I’m not sure, but I wonder if some thinking is applied to creating a mixture of people, per-batch, who are likely to have shared interests. For example on my batch there seemed to be a lot of people who were interested in visual or graphically-creative things. It may that a lot of us arty ones got thrown in together in this way on my batch… or maybe RC attracts those kinds of coders naturally.

Either way, it works.

Being Fully Remote Part 5 of 8

Making the most of RC from a thousand miles away.

I knew there might be some extra challenges with attending remotely. Time zone differences. Communicating solely via Zoom and Zulip (a messaging app a bit like Slack).

During the application process somebody had given me a friendly tip: as a fully remote person I should make sure I put effort into making myself visible by messaging people on Zulip, attending events etc. People who were physically in the hub would naturally bump into each other and chat, get face-to-face time and collaborate on physical/hardware projects. None of that applies to a remote participant.

As expected, I had a little, manageable, trouble with the time zone difference. Your batch is a full-time commitment and you’re expected to be there for full (at least 6 hours) days on average. The calendar events happen on New York time, which for me in the UK meant that the RC core hours started at 4pm and ended at 10pm.

I’m naturally an early riser, usually up around 5am and it’s not a quick or easy task to adjust your sleep schedule. This meant I’d be doing long days. I spent the first week or so finding my footing.

My daily routine

In the UK during my batch we were 5 hours ahead of New York. Eventually I settled on a pattern.

07:00 - 11:15 Morning coding Work on whatever my ‘main project’ was. A good time to get in flow with something I was relatively comfortable with. 11:30 - 12:30 Body doubling While very beneficial in itself, as a daily practice this also became a useful separator between my ‘morning coding’ and my ‘RC day’. 13:00 - 15:00 Food and nap Make a point of laying down with the curtains drawn for an hour. Sometimes I’d sleep, if not I’d at least get a rest from the screen and a little body/mind relaxation. 16:00 - 22:00 Main RC day Activities from the RC calendar, coffee chat and ‘outside my comfort zone’-coding.

Although I wasn’t pushed into it (there’s an understanding that people in different time zones may need to deal with things a little differently), I decided I wanted to be present until 10pm every day. I knew I was putting in the hours in the morning and getting plenty of code written, but still I wanted to aim for a ‘full RC day’ for my own discipline.

I managed to stick to this commitment pretty well, and on the few occasions I bunked off early it was by minutes rather than hours.

The later, evening, hours were sometimes a bit less focused than my morning coding — this core RC time would naturally be broken up by activities or chats, and I’d tend to be working on exploratory stuff, things that were more new and unfamiliar.

One small concession (or at least scheduling compromise) I made was that I avoided Zoom calls in the last few hours of my day. Although I can code ’til bedtime no problem (like most coders!), I did notice that it was harder to whip up the social energy for face-to-face chats in those later hours. This was partly because I worried that through tiredness I might have a flatter demeanour, seem disinterested… or keep yawning!

Ultimately though, everything worked out fine.

Activities Part 6 of 8

Some of the ways you spend time on a batch.

RC is self-directed. There’s a shared calendar full of events every day, but it’s up to you how many you attend — you’re not going to ‘fail’ by missing events. I believe if it seems like you aren’t getting involved at all, faculty will probably chat to you about whether this is the right time for you to be doing a batch. This seems more than reasonable.

I probably should have put more effort into resisting my tendency to be a code hermit and attended a few more non-compulsory events. A common theme in chats with other Recursers though is that time flies really quickly when you’re on batch. I don’t think anybody gets to do everything they’d like to.

There are lots of ways to approach how you spend your time. Some people work on different projects every day while others focus more deeply on one or two projects throughout their batch.

General activities

There are too many of these to mention and they change all the time. Many are organised by faculty, but anybody (including other Recursers) can create events about anything related to programming.

To give a rough idea a few examples are:

• Pair programming exercises
• Impossible Day and Impossible Hour, where the idea is to set yourself a task at which you expect to fail
• Groups for working through classic coding books and exercises together (eg during my batch some people worked through The Book of Shaders)
• Loads of special interest groups for things like audio programming, GPU shaders, system programming, LLM usage, LLM coding/creation
• Meta activities to assess how you’re doing at RC and make best use of your time there
• Career analysis and planning

Coffee chats

This was one of my favourite parts of being at RC. You can sign up for ‘coffee chats’ for as many days as you like, eg ’every week day’, or ‘Mondays and Thursdays’. On days where you’re signed up to chat, a bot matches you up with somebody else and you have a little informal meeting/chat. As a fully remote attendee this would happen over Zoom for me.

I met dozens of people in thie way during my batch. There’s something magical about the way RC works, in that even though you meet all kind of diverse people with different tastes and interests, you are somehow ‘guaranteed’ that they will be intelligent, kind, and interested in whatever it is you want to talk about. The conversations range from pleasant 15-minute introductions and general chat, right up to multi-hour rambles across all kind of topics when you happen to meet somebody whose interests particularly overlap yours.

For me as a long-time self-employed, remote working, solo developer with a tendency towards introversion, the coffee chats were a valuable experience. Like a lot of people I get a bit socially anxious and I’d sometimes overthink and feel dread at the concept of being thrown into another face-to-face, with another stranger. But I pushed through those feelings and never once regretted it.

I admit that once or twice, I did make excuses to myself and skip a chat — especially towards the end of my batch, when time seemed to be flying so quickly. More fool me though, as it means I missed out on meeting more great people.

Presentations

Every week on a Thursday (night, for me in the UK) there’s an hour or so of short presentations. Here fellow Recursers talk for 5 minutes, with slides and demonstrations, about some code-related thing they’re interested in and/or working on at the moment.

The range of topics is incredible. Most of them were way outside my wheelhouse, but they all still somehow managed to be interesting. The 5-minute length helps with that as it’s a nice bite-sized chunk of time.

Virtual check-ins

Specifically for the fully remote people, a couple of times a week there are virtual check-ins. These are chill meetings with your batch mates to chat over what you’re working on, any problems you might be having and what’s coming up next for you. It’s very friendly and informal but helps you stay focused and get the most out of your time at RC.

Obviously these are things that were available to me as a remote attendee. For people in the Hub there were some other activities available eg:

• Working together on hardware projects (this isn’t impossible remotely but I think more fun in-person)
• Social/local stuff like meeting up for lunch/drinks etc
• A very cool vintage electronics lab
• Meet and greets with prospective employers
• A library of tech books
• …chair stacking (I witnessed it myself!)

Extra Benefits Part 7 of 8

Some nice stuff continues even after you finish your batch.

Never graduate

After a batch at RC you are a life long alum. This means you forever have access to the organisation on Zulip to message people, can still give and/or watch presentations, and can join in on lots of activities. You can also get help from the careers department.

Considering I’ve been a programmer for over 20 years, I know surprisingly few other programmers! Early in my career I tended to work alongside designers, with me being the only developer. Later on I was self-employed, tended to work remotely and with much smaller companies. I’d often be called in as an outsider to code something that was without the skill-set of the current employees. I’d finish the project and be on my way.

It was great to meet so many other programmers at RC, and to be able to interact with them in a very socially safe way. There was no competition, just mutual help and interest — we were all on the same team.

It’s now a couple of months since I finished my batch and I’m still in regular contact with a few other Recursers who I think/hope I can now call friends.

Careers help

During and after your time at RC you get access to their careers department. They’re great to work with can help with some important stuff like:

• CV/Resume advice
• Job search
• Mock interviews

As with the rest of RC faculty, everyone in the careers department is kind, helpful and generous with their time.

If they place you in a job, the employer pays RC for helping them find you. You don’t pay anything, but this means it’s in RC’s interest to help you find work, as it helps fund them when they succeed. I think this is a good and healthy system. It costs money to sustain an organisation like RC and they need to be funded somehow. Doing it this way means people like me get to access RC for free, without any hidden costs or ulterior motives. Everything is out in the open.

return(); Part 8 of 8

What I got out of it.

I went into RC with as few expectations as possible. I hoped I’d meet some people, learn some things, make some things. Achievements unlocked for sure, but more specifically, this is some of what I came out with:

• Algernon app - I built an audio visualiser app in Flutter, learning about real-time audio processing, Fast Fourier Transforms and fragment shaders.
• caldavtui app - A TUI calendar app for the terminal. This started out as an Impossible Day project, progressed to ‘oh I can actually do this!’ and has now become a long-term project for me. It already works nicely but there’s a lot more I want to add and I’m fleshing out my knowledge of the relevant specs (eg iCalendar and CalDAV).
• Body doubling / pairing - This is a simple process — pair up with somebody and commit to working on something for an hour without allowing yourself to get distracted. It’s especially useful for me as somebody who works from home, helping me stay on track and to keep an eye on overall progress of whatever I’m working on.
• Comfort with Godot game development - I paired with another RCer who showed me what he’d learned so far as a Godot beginner himself. Then I had a day with an LLM, and a day or two on my own. Combined, these few days have left me feeling comfortable with Godot. There are of course lots of things I haven’t explored and am yet to learn, but as somebody who has built several games before I now feel ready to tackle any project that comes up in Godot. I have my bearing.
• Increased C/Go proficiency - After putting in a few dozen hours on each, I’ve gone from almost zero knowledge of C and Go, to beginner/intermediate level. Again, still loads to learn, but I have a level of comfort now and am no longer daunted at the idea of starting a project in these languages.
• Clarity on my career and where I might want it to go - Through some career-focused workshops I’ve done some structured thinking about my career in general and about what might suit me in future.
• Comfort with Zoom - Face-to-face video calls are an important part of the modern work environment, especially for remote workers. Before RC I used to get the sweaty-palmed jitters before a Zoom call, especially with strangers. Now it’s much more normal for me.
• Contacts/networking - I met loads of great people that I share interests with, have stayed in touch with some of them, and because of the general vibe of RC I feel comfortable contacting others in future if I ever have reason to.
• Updated CV - After some useful talks with the careers department I’ve reworked my CV to make it shorter and clearer. I’m pretty happy with the new version.

I can say with no exaggeration that I’d have been chuffed with any single one of those gains. To have come out of the process with all of them is way beyond what I might have hoped for. Truly amazing.

During an ’exercising your volitional muscles’ activity, we were asked to note down the thing we were most proud of about our time at RC. We discussed these things in small groups. When we returned to the main group some of us shared what we’d discussed, but I was feeling quiet and kept mine to myself.

For me, considering I nearly didn’t do it, and could have missed out on all those things I gained from RC, one part of the process stood out as most important:

The thing I’m most proud of is that I applied in the first place!

Want to become a better programmer? Join the Recurse Center!