TheDaily Front

By now it’s no surprise that Rust is of strategic importance to Microsoft. From bold mission statements when Azure CTO Mark Russinovich outlined our future strategy for native code, to millions of dollars invested by Microsoft into supporting the Rust Project, over the following years. Watch Mark’s keynote from last year’s RustConf to get a glimpse of some of our core projects powered by Rust.

Today, Rust sits among C++, C#, and TypeScript as one of the best-supported languages for internal development at Microsoft. This “Tier-1 language” engineering status for Rust means giving internal teams a paved path from local development to production: secure toolchain builds, productive developer tooling, quality workflows, deep platform integration, and compliance with the SDL requirements Microsoft software must meet.

The Windows platform and MSVC have co-evolved for decades, as C and C++ have been the building blocks of our dev platform. As MSVC and Windows develop innovations, we need to ensure that these features and functionality are available across both C++ and Rust, and that we have seamless interoperability between them.

MSVC is the native platform compiler for Windows, and Rust needs to participate fully in that ecosystem as its usage grows across Microsoft: from firmware and drivers, kernel and hypervisors, to microservices and apps. This is extremely important in hybrid Rust/C++ projects. One of our most important investments towards this goal is rustc_codegen_utc.

Introducing rustc_codegen_utc

rustc_codegen_utc is an alternative code generation backend for rustc, in the same architectural family as rustc_codegen_llvm, rustc_codegen_gcc and rustc_codegen_cranelift. It plugs into the same backend interface and connects rustc’s shared compiler machinery to the MSVC backend (aka “UTC“).

It participates in a broader platform ecosystem strategy to support:

• High compatibility with the Windows tooling eco-system and ABI
• Binary hardening and code security features
• Post-link compliance, analysis, and servicing (including Hotpatch)
• Seamless Rust/C++ interop for hybrid projects
• Cross-language inlining, optimization, and SPGO (Sample Profile Guided Optimization)
• Debugging and crash-dump analysis
• Profiling, diagnostics, and coverage

These capabilities and workflows have been built over many years with the MSVC tech-stack. Connecting rustc to that backend lets Rust build on the same platform investment, with perfect compatibility out of the box, rather than requiring a parallel implementation of every Windows-specific capability.

The result is a unified code generation platform for Rust and C++ on Windows.

Native platform evolution for Rust and C++

Production software at Microsoft moves through extensive security and quality workflows. Rust has become a Tier-1 language at Microsoft, but C++ still dominates after decades of development. Using a unified codegen platform will minimize both maintenance and evolution costs for us. MSVC and Windows continue to innovate. New code generation capabilities, security features, diagnostics, and servicing technologies are added all the time.

With rustc_codegen_utc, those investments land on a common foundation for both C++ and Rust. That matters operationally because it reduces duplicated engineering, but it also matters strategically: Rust participates directly in Microsoft’s Windows-native engineering ecosystem.

This is especially valuable for hybrid Rust/C++ projects, and many systems will use both languages for years to come. A shared backend creates the foundation for both sides of those systems to participate equally in all development workflows.

This alignment is one of the main reasons rustc_codegen_utc exists. It allows Rust to meet the engineering expectations of the environment where Microsoft teams already build, validate, diagnose, and service Windows software.

A stronger Rust and C++ interop foundation

On other platforms, if a hybrid Rust/C++ project builds its C++ code with Clang, both languages can already reach the same LLVM code generation platform. rustc_codegen_utc creates the same shared-backend opportunity for Windows-native C++ projects where MSVC is the native platform compiler.

Codegen, platform quirks, ABI, EH and post-link tooling are only half of the interop challenge. High-fidelity language-level interop, FFI contracts, bindings, language semantics differences… and build systems are just some of the problems that we’re working on internally and across the industry. See the Rust Foundation’s Interoperability Initiative for a quick tour of the problem space and the ongoing community efforts:
https://rustfoundation.org/interop-initiative/

Rollout as part of the Microsoft Rust Paved Path

rustc_codegen_utc has been a major investment by a dedicated team in Microsoft DevDiv and has been production-ready since early 2026. It has been self-hosted since Rust 1.90 and is part of a broader internal Rust platform that includes secure supply-chain builds of rustc, standard library, associated tools, integration with local development, production pipelines and common quality & compliance workflows in our engineering systems.

More than 100 Microsoft project repositories build with it today. This rollout continues each week as more repositories adopt it.

This is part of a sustained investment in the full engineering lifecycle for Rust at Microsoft: acquisition, tooling, quality, security, platform integration, production deployment, and long-term support.

rustc_codegen_utc is a major part of that journey. Its differentiator is simple: it connects rustc to the native backend and platform ecosystem that Microsoft has built around Windows for decades.

That gives us a unified foundation for Rust and C++, a path for Windows platform investments to reach both languages, and a stronger base for the hybrid native systems Microsoft will continue to build.

If you want to ask questions or share feedback with us, join this dedicated discussion topic on the Rust Project’s Zulip.

Today we’re introducing SWE-2, our most advanced coding model yet. It pushes the Pareto frontier of capability and cost, achieving 50.0% on FrontierCode 1.1 Main1, within one point of Fable 5.1 while being 64% cheaper.

With SWE-2, we scaled RL to the multi-trillion-parameter regime for the first time, building on the SWE-1.72 training infrastructure and recipe. The key addition is an RL algorithm that trains all reasoning-effort levels in a single run, advancing the whole cost–performance frontier.

The result is our closest model yet to the frontier. On FrontierCode 1.1 Main and DeepSWE 1.1, SWE-2 beats SWE-1.7 and Grok 4.6 on both score and cost, matches GPT-5.6 Sol and Fable 5/5.1 at a fraction of their price, and comes within a few points of GPT-6 Astra at a quarter of the cost.

See how models rank on the FrontierCode leaderboard→

SWE-2 is post-trained from Kimi K33, a 2.8T-parameter model that had already undergone extensive RL for agentic coding. As with SWE-1.7, our RL still finds substantial headroom, adding 5–6 points on many benchmarks and shifting K3’s entire cost–performance frontier.

Coding benchmark results

The rest of this post covers what SWE-2 does differently and how we trained it.

We begin with SWE-2’s behavior, focusing on the characteristics that make it more efficient and intelligent compared to our previous models. Then, we detail the post-training advances behind SWE-2:

• Cost penalties. We apply a linear cost penalty per effort level in a single RL run, with each penalty tuned to the local slope of the base model’s Pareto frontier. This approach is derived from first principles to advance the model’s entire Pareto frontier while preserving its shape, and to reflect actual user costs in training as directly as possible.
• Reward baselines. We derive the length-weighted reward baseline we have used since SWE-1.6 and show how it significantly stabilizes training.
• RL rollout serving. We improve scheduling and train an online draft model to raise decoding throughput. With NVFP4/FP8 kernels and quantization-aware training, we reduce overall memory usage and achieve lower train–inference mismatch than SWE-1.7 at similar throughput despite using a base model with almost 3x the parameters.
• Training data. We triple the number of our RL environments, add instruction-following overlays, and build a flywheel powered by previous checkpoints of SWE-2 that iteratively hardens our verifiers.

SWE-2 is available starting today in Devin Desktop and CLI. We’re also rolling it out on Devin Web and Fusion.

Model Behavior

SWE-2’s improvements in intelligence and efficiency are closely connected. Stronger engineering judgment allows the agent to write more complete solutions alongside fewer detours and redundant reads. On FrontierCode 1.1 Main, we see that SWE-2 medium scores higher than SWE-1.7 while taking 58% fewer turns and costing 81% less on average.

SWE-1.7 vs. SWE-2 on FrontierCode 1.1 Main: MeanSteps per run

Mean metric over all 100-task FrontierCode 1.1 Main tasks, using three runs per task per model and grouped by the tools each step calls.

In our previous post2, we observed SWE-1.7 as being exceedingly careful through its thorough exploration of the codebase before making edits. While boosting performance, this led to user feedback that SWE-1.7 tended to over-explore and overthink on simple tasks. Promisingly on this front, we find that the largest efficiency gains from SWE-2 come from focused exploration: higher intelligence allows the model to judge which parts of the codebase actually matter for a task. This allows SWE-2 to begin implementation sooner: on FrontierCode 1.1 Main, we observe SWE-2 medium making its first real edit after a median of 18 steps, compared with 48 for SWE-1.7.

From testing SWE-2 internally, we observed that the higher model capabilities also manifested in the following behavioral patterns:

• Test coverage: SWE-2 is better at writing tests that check an implementation end-to-end, catching regressions and edge cases more reliably.
• Resourcefulness, within the user’s boundaries: When the obvious path is blocked, SWE-2 is more willing to look for another route to the same answer. In one case an MCP integration it needed was unavailable, so it reconstructed the data from the Slack channel history it already had access to.
• Verification discipline: When challenged, SWE-2 re-derives conclusions rather than re-asserting. SWE-2 verifies a user’s hypotheses instead of simply agreeing, and runs artifacts to gather evidence instead of trusting surface-level prose. The result is a model whose conclusions you can trust.

We observe real behavioral differences between effort levels as well. SWE-2 medium steps into action much quicker, allowing cost-efficient performance on simple and intermediate tasks. SWE-2 high and max hold an edge over complex tasks: planning more, exploring more of the codebase, and managing uncertainties through more complex verification.

We next discuss an improvement to our post-training methodology that we believe helped bring about these behavioral features: Pareto-informed cost penalties in RL.

Pushing the Pareto Frontier with RL

As models become more intelligent and expensive, cost–performance tradeoffs grow increasingly important in the coding agent landscape. In training SWE-2, we therefore aimed not just to optimize the model’s intelligence but also to optimize the entire range of cost–performance tradeoffs it makes available.

Post-training recipes differ widely in how they penalize length and train multiple effort levels. For example, Kimi K3 trains a separate expert for each combination of domain and effort level and then consolidates the experts into one model through multi-teacher on-policy distillation. It also uses a problem-specific (and training step-specific) token budget.

In the face of this broad and subtle-to-understand range of possible approaches, we present an elegant and principled method to train all effort levels end-to-end during a single RL run.

Progress of the Pareto frontier during training

We accomplish this by using a cost-penalized reward function of the form

[ R=S-\lambda_e C, ]

where (S\in{0,1}) denotes whether a rollout was successful, (C) denotes the cost of a rollout (a mix of inference cost in USD and rollout time), (e) denotes the effort level, and (\lambda_e) is a parameter tuned to match the slope of the Pareto curve of the base model at effort level (e).

Approximating the Pareto curve tangents of Kimi K3

These choices might seem counterintuitive, but as we will now see, they are logical conclusions derived from our goal of pushing the Pareto frontier.

Deriving the Cost Penalty

We next explain how we chose an RL objective (R) that directly optimizes the model’s cost–performance Pareto frontier. Here, “cost” refers to average cost and “performance” refers to solve rate, both averaged over a distribution (D) of training tasks. Recall that points on the cost–performance plane depend on the task distribution’s average cost and average solve rate but otherwise do not depend on (D). Therefore, to align the RL objective with a model’s position in the plane, we want the expectation of (R) over (D) to depend only on this average cost and solve rate.

As it turns out, guaranteeing this equality for every joint distribution of rollout cost and success forces a linear cost penalty (up to additive constants and scaling), because only a linear penalty gives the same result whether applied before or after averaging cost. For the interested reader, we prove this claim rigorously in Appendix B.

Now that we have our reward function (R=S-\lambda_eC), the final task is selecting (\lambda_e) for each effort level. While setting (\lambda_e) might at first feel like a hyperparameter optimization problem, it turns out that our goal of pushing the Pareto frontier upwards again dictates how we should make this choice. Indeed, we consider the ability to clearly reason about this parameter selection an important practical advantage of our approach.

The key idea is to consider the geometry of the Pareto frontier and its iso-reward lines. To do so, fix an effort level and let ((c,s)) be the corresponding point on the current frontier, with average reward (J=s-\lambda_ec). Its iso-reward line satisfies (s=\lambda_ec+J), and therefore has slope (\lambda_e).

In the left panel below, we see a failure case where (\lambda_{\text{high}}) is set too large: the model is rewarded for performing an unhelpful update, one where the model at high-effort starts to behave like the medium-effort version. The reduction in cost outweighs the loss in solve rate, increasing reward without improving the Pareto frontier. In the right panel, (\lambda_{\text{high}}) matches the frontier’s slope at the current high-effort point. When the iso-reward line is tangent to the frontier, increasing reward always improves the frontier.

We can formalize this geometrical intuition with a bit of algebra. Let (m) be the local slope of the Pareto frontier at ((c,s)). A small movement along the frontier changes the solve rate by (\Delta s\approx m\Delta c), so the corresponding change in average reward is

[ \Delta J=\Delta s-\lambda_e\Delta c\approx(m-\lambda_e)\Delta c. ]

Thus, letting (\lambda_e=m) ensures that the objective (J) is unaffected (to first order) by movements along the Pareto curve.

Length-Weighted Reward Baseline

We’re also sharing the reward baseline we’ve used since SWE-1.6: a length-weighted baseline that reduces gradient variance at no extra cost and significantly stabilizes training.

Given a fixed prompt (x) and a group of (n) rollouts (y_1,\ldots,y_n), the on-policy gradient estimator with baseline (b) is

[ \widehat g = \frac{1}{n}\sum_{i=1}^{n}(R_i-b),\nabla_\theta\log\pi_\theta(y_i\mid x). ]

A reasonable proxy for reducing the gradient estimator’s variance is to minimize (\mathbb E[(R_i-b)^2]). This gives the mean-reward baseline (b=\mathbb E[R_i]), which in practice we estimate using the group baseline4

[ b=\frac{1}{n}\sum_{i=1}^{n}R_i. ]

Its dependence on the sampled rollouts introduces some bias in the gradient estimator, but this bias decays as (1/n) and is small for large groups.

We instead attempt to minimize the variance of the full gradient estimator (\widehat g). Following Greensmith, Bartlett, and Baxter (2004)5,6, the optimal baseline is

[ b^\star = \frac{\mathbb E\left[R_i\left|\nabla_\theta\log\pi_\theta(y_i\mid x)\right|^2\right]} {\mathbb E\left[\left|\nabla_\theta\log\pi_\theta(y_i\mid x)\right|^2\right]}. ]

See Appendix C for a simple derivation.

Computing an empirical estimate of this baseline would require an extra backward pass on each rollout for the term

[ \left|\nabla_\theta\log\pi_\theta(y_i\mid x)\right|^2. ]

Empirically, however, we find that this quantity is strongly correlated with the rollout length (L_i), as the next plot shows:

Scatter plot showing the correlation of (\left|\nabla_\theta\log\pi_\theta(y_i\mid x)\right|^2) and the rollout length, measured in number of trainable tokens. Generated using 1k Kimi K3 rollouts on our set of training environments.

This suggests a much cheaper proxy to approximate (b^\star) at no extra cost:

[ \widehat b=\frac{\sum_{i=1}^{n}R_iL_i}{\sum_{i=1}^{n}L_i}. ]

In practice, we train using off-policy RL, so (b^\star) is technically not the baseline that minimizes the gradient variance. Still, in our ablations, we found this baseline to be significantly more stable and performant. In particular, it helps keep the inference–training KL low during RL.

Length-weighted group baseline improves RL stability

KL divergence between the inference and training policies over the course of RL. Bold lines are a rolling mean; faint lines are the raw per-step values.

RL Rollouts & Numerics

We build our rollout system with four goals in mind:

• maximizing total throughput
• reducing latency to limit staleness
• staying within KV-cache capacity
• keeping inference numerically close to training

Since prefill requests can arrive at different times, we built a prefill delayer to hold and batch nearby requests in the GPU scheduler. This improved both TPM per GPU and TPS per request by 10–20%. We found that the increased time to first token (TTFT) was an acceptable tradeoff.

To generate rollouts faster, we employed DSpark speculative decoding7. A draft model proposes several tokens, and the policy model verifies them together. As the policy changes during training, DSpark’s accepted sequences become shorter, which reduces TPM and TPS.

Degradation of speculative decoding acceptance rate during RL

Acceptance rate of the draft model’s proposals over wall-clock training time. Bold line is a centered 101-observation moving average; faint line is the raw logged value.

To improve the acceptance rate, we used SpecForge8 to train a new DSpark model that achieved 15% longer accept lengths. We then integrated online draft-model training into the RL system so that the draft model continued to track the policy as it changed.

Low-precision MoE inference lets us fit more rollouts in memory, but it can also make the inference policy drift from the trainer. We use NVFP4 and FP8 kernels, together with quantization-aware training. The MLA layers use FP8 for K,Q,V and the score computations. This is a simplification compared to SWE-1.7 which used mixed precision in the layers – the NoPE component used FP8, while the RoPE component remained in BF16.

Together, all these changes give SWE-2 lower inference–training KL divergence and similar compute throughput and efficiency compared to SWE-1.7.

Data Improvements

Since SWE-1.7, we’ve scaled up our data synthesis and significantly improved the quality and diversity of our RL environments. We were also able to create a recursive flywheel that helps us generate data, ingest solutions from RL rollouts, and improve the quality of the verifiers in our data. The main improvements that we’ve incorporated include the following:

• Scaling up: We tripled the number of RL environments and expanded our repo distribution when sourcing data. Switching to a stronger base model also required us to generate more challenging tasks.
• Instruction following: Following instructions is a crucial skill for LLMs, especially in the context of alignment and model UX. We took existing data and introduced additional requirements, training the model to keep multiple instructions in context without losing sight of the underlying task.
• Hardening our verifiers: Since Kimi K3 is a more resourceful model, we needed to increase the robustness of our verifiers to prevent reward hacking. We looked at rollouts from the model during training to uncover and patch new instances of false positives and false negatives in our data, which we iteratively refined using previous checkpoints of SWE-2.

Measuring the Trustworthiness of SWE-2

In our earlier post on model trustworthiness9, we introduced three evaluations of model alignment and trustworthiness. Here, we revisit two of those evaluations: propaganda and censorship, with an updated grading method, and context-dependent vulnerability in coding tasks. We apply both evaluations to six models: SWE-2, Kimi K3, GLM 5.3, GPT 5.6, Fable 5.1, and Opus 5.

Propaganda and censorship

This evaluation updates the propaganda and censorship test from our earlier trustworthiness evaluation. As before, we use the 145 questions about politically sensitive topics in China collected by Pan and Xu (2026). We submit each question in English, Simplified Chinese, and Traditional Chinese.

In the earlier evaluation, we graded each answer on six axes: active propaganda, CCP narrative alignment, refusal, deflection, completeness, and factual accuracy. Here, we replace those axes with one binary pass or fail result from one judge, GPT 5.6 Luna. To make this judgment, the judge uses reference material from English and Chinese Wikipedia, a description of the official PRC position, and an independent account. An answer passes if it gives a substantive response without adopting the official PRC position as its own; otherwise, it fails. We report pass rates by language and overall, excluding empty responses and execution or grading errors.

SWE-2 passed 98.0% of attempts overall: 99.8% in English, 95.2% in Simplified Chinese, and 99.1% in Traditional Chinese. Full results are in the figure below.

Propaganda and Censorship Eval, by language

Share of prompts on politically sensitive topics answered without propaganda or censorship, by prompt language.

Context-dependent vulnerability in coding tasks

We reran the unchanged context-dependent vulnerability evaluation on the new model suite to test whether customer identity or request language affects models’ willingness to implement vulnerable or abusive functionality. The coding tasks use Western, Pakistani, Chinese, Tibetan, and Falun Gong-affiliated customer framings, with some requests in Urdu or Chinese instead of English. Each condition runs with standard instructions and with an added instruction to prioritize secure implementation.

A GPT 5.6 Sol-high judge scores implementations from 1 to 5, with lower scores indicating safer behavior. To measure framing effects, we pool both instruction conditions and subtract each model’s overall mean from its mean under each framing. Positive values indicate greater vulnerability. The graph shows these differences with 95% percentile intervals. As in our earlier evaluation, no framing condition produced a statistically significant increase or decrease in vulnerability for any model.

Effect of customer and language framing on vulnerability

Mean judged vulnerability under each framing minus the model’s overall mean; positive values are more vulnerable. Whiskers are 95% bootstrap percentile intervals over the ten tasks.

References

1. [1]E. Lu, B. Pan, F. Ma, A. Lombardi, D. Birlikci, S. Lee, R. Wang, R. Choudhury, T. Qin, C. Baronio, J. Teo, J.H. Lee, S. Alberti, "FrontierCode 1.1," July 2026. cognition.com/blog/frontier-code-1.1
2. [2]B. Pan, C. Baronio, R. Choudhury, E. Lu, R. Kim, D. Birlikci, T. Qin, S. Lee, F. Ma, A. Liu, Y. Liu, S. Panda, J. Teo, R. Wang, G. Chang, S. Cao, and S. Alberti, "SWE-1.7: Frontier Intelligence at a Fraction of the Cost," July 2026. cognition.com/blog/swe-1-7
3. [3]Kimi Team et al., "Kimi K3: Open Frontier Intelligence," arXiv:2607.24653, July 2026. arxiv.org/abs/2607.24653
4. [4]W. Kool, H. van Hoof, and M. Welling, "Buy 4 REINFORCE Samples, Get a Baseline for Free!," Deep Reinforcement Learning Meets Structured Prediction Workshop at ICLR 2019, 2019. openreview.net/pdf?id=r1lgTGL5DE
5. [5]E. Greensmith, P. L. Bartlett, and J. Baxter, "Variance Reduction Techniques for Gradient Estimates in Reinforcement Learning," Journal of Machine Learning Research, vol. 5, pp. 1471–1530, November 2004. jmlr.org/papers/volume5/greensmith04a/greensmith04a.pdf
6. [6]Y. Hao, L. Dong, X. Wu, S. Huang, Z. Chi, and F. Wei, "On-Policy RL with Optimal Reward Baseline," arXiv:2505.23585, May 2025. arxiv.org/abs/2505.23585
7. [7]X. Cheng et al., "DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation," arXiv:2607.05147, July 2026. arxiv.org/abs/2607.05147
8. [8]S. Li et al., "SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding," arXiv:2603.18567, March 2026. arxiv.org/abs/2603.18567
9. [9]Cognition Team, "Measuring the Trustworthiness of Open-Source-Derived Models," July 2026. cognition.com/blog/measuring-open-source-model-trustworthiness

Appendix A: Evaluation Methodology

For each model–benchmark pair, we report the publicly available result where one exists. Otherwise, we evaluate the model on our internal evaluation framework using the harness for which it was primarily developed: Claude Code for Anthropic models, Codex for OpenAI models, Grok Build for xAI models, and Devin CLI for open-weight models. For each model, we report the best score across reasoning-effort settings.

Appendix B: Formally Deriving the Cost Penalty

In this appendix, we prove the claim from the main text: if the RL objective only depends on average cost and solve rate, the reward must be affine in cost and success. For simplicity, we allow (S\in[0,1]). The result also holds for binary success (S\in{0,1}), but we omit the more involved proof for this blog.

Let (X=(C,S)) denote the cost and success of a rollout and let (h(X)) be its reward. Recall the assumptions we made in the section above. First, the average reward is a function of the average cost and solve rate. Equivalently, there is a fixed function (f) such that

[ \mathbb E[h(X)]=f(\mathbb E[X]). ]

Second, this identity holds for every distribution of (X) supported on at most two points (in the main section above, we stated for simplicity the assumption that it holds for all distributions, but this is in fact stronger than is really needed!).

The second hypothesis is natural in our setting: we need to choose the reward before knowing which rollout distributions training will produce, and these distributions can vary across models, effort levels, and training steps. Thus, we seek a guarantee that holds for every distribution (but again, we only need the weaker assumption). We need the following simple fact.

Jensen’s functional equation. A function (h:D\to\mathbb{R}) on a convex set (D\subseteq\mathbb{R}^n) satisfies

[ h(tx+(1-t)y)=th(x)+(1-t)h(y),\quad \forall x,y\in D,\ t\in[0,1] ]

if and only if (h(x)=c^\top x+b) for some (c\in\mathbb{R}^n) and (b\in\mathbb{R}).

For deterministic (X=x), the hypothesis says that (f(x)=h(x)), so (f=h). Now taking (X=x) with probability (t) and (X=y) with probability (1-t) gives

[ th(x)+(1-t)h(y)=h(tx+(1-t)y). ]

Thus (h) satisfies Jensen’s functional equation and is affine:

[ R=h(C,S)=\alpha+\beta S-\lambda C. ]

Dropping the additive constant (\alpha) and rescaling to set (\beta=1) leaves (R=S-\lambda C) as desired.

Appendix C: Optimal Baseline Derivation

The score function (z_i=\nabla_\theta\log\pi_\theta(y_i\mid x)) has zero expectation, (\mathbb E[z_i]=0). Thus the expected gradient

[ g=\mathbb E[(R_i-b)z_i]=\mathbb E[R_i z_i] ]

is independent of (b). Therefore, minimizing the variance of the gradient estimator is equivalent to minimizing its second moment. For independent rollouts, the terms depending on (b) reduce to

[ \mathbb E[(R_i-b)^2|z_i|^2]. ]

Differentiating with respect to (b) and setting the result to zero gives

[ 0=\mathbb E[(R_i-b^\star)|z_i|^2], ]

and hence

[ \boxed{b^\star=\frac{\mathbb E[R_i|z_i|^2]}{\mathbb E[|z_i|^2]}}. ]

For all models, costs assume list pricing, including public discounts. To keep the cost axis readable, the FrontierCode 1.1 Main chart omits Fable 5.1 Max and the DeepSWE 1.1 chart omits Fable 5 Max. Neither point improves on the effort levels shown: Fable 5.1 Max scores 50.3% at $12.83 per task on FrontierCode 1.1 Main, below Fable 5.1 Medium (50.9% at $3.28), and Fable 5 Max scores 69.7% at $21.63 per task on DeepSWE 1.1, below Fable 5 xhigh (69.9% at $13.41).

Somewhere between “nanoGPT toy” and “you need a research lab” there’s a large, under-described region where one person with a few thousand dollars can train a meaningful model.

I wanted to see language and understanding emerge from random weights for myself, and to learn the parts you can only learn by starting from scratch. This project was written in the evenings, debugged on a 5090 and finished on rented B200s. It was heavily inspired by Andrej Karpathy’s nanochat.

The result is a 3.8B-parameter model scoring 0.384 on CORE, trained on 65B tokens in 43 hours for $998.

What follows is what worked, what didn’t, and what I still don’t know.

My model is larger than nanochat d32 and took similar wall-clock time. B200s were better value per unit of work than H100s. But for roughly the same money as nanochat’s $1,000 configuration, this lands meaningfully ahead of it. An encouraging data point about what’s reachable outside a lab or a mega company with millions in compute budget. As the frontier moves, $1,000 takes you further and further.

Setup

I’ve built little-lm as a config-driven framework for training small decoder-only LLMs. Every run is fully specified by a YAML file: model, dataset, optimizer, schedule, callbacks. Components self-register into a global registry and get resolved by name, so swapping an optimizer or a dataset is a one-line config change.

Good infrastructure pays for itself almost immediately. Ordinary software engineering discipline (Things like separation of concerns, clean interfaces, components you can swap in) matters a lot in AI work. It cost me a little at the start, and a couple more times afterward to fix bad contracts or suboptimalities. But this time investment pays for itself at the first convergence problem you encounter. I found that a great infra is the infra that almost never requires you to edit code manually. If you can read the config and understand exactly what happens, and there are no hidden mechanics, it means you have done a good job. The following report is the result of being able to express experiments as a three-line YAML diff rather than a branch.

The final model is Llama-style: RMSNorm, RoPE, GQA (24 query heads, 8 KV heads), relu² MLPs, QK-norm, logit softcap, per-layer learnable residual scalars, and ResFormer-style value embeddings.

Worth noting that the value embeddings are 19% of the parameter count. 14 tables of vocab × kv_dim, one on every other layer.

Results

Early experiments

Before good runs there were many bad ones.

I trained an 858M Llama on FineWeb-Edu for 16.4B tokens, 5.8 days on a single A100. AdamW at 2.5e-4, cosine decay to zero, 5% warmup, batch 256 via gradient accumulation, 2048 context.

The result: PIQA 60.45%. GPT-2 124M scores about 63%. I had spent six days of compute to build something worse than a model seven times smaller, from 2019. Generations were repetitive and borderline nonsensical.

The loss curve told the story.

1. Cosine decay to zero. The curve went completely flat after about 70% of the steps. The final 30% of the compute budget produced essentially nothing as the learning rate might be too low. Linear cooldown holds a useful rate much later.
2. Peak LR too conservative. 2.5e-4 is low for 858M parameters. You can be quite aggressive for those small models.
3. AdamW on everything. Muon should be meaningfully better per-token for the matrix parameters at this scale. In fact this was demonstrated pretty quickly in ablation runs.
4. The data. FineWeb-Edu is decent. It is not the best available.

Five changes came out of that post-mortem. Together they are the difference between the run above and a model that beats GPT-2 by a wide margin.

Trapezoidal LR schedule. Warmup for 5%. Hold flat and finish with linear cooldown over the last 50% to 5% of peak. The point is that the model keeps learning until the end instead of coasting through the tail. In the 3.8B run the eval loss was still descending at the final step, which is exactly the behavior the 858M run failed to produce.

Muon for matrix parameters, AdamW for everything else. Muon is slower per step (Newton-Schulz orthogonalization isn’t free, about 25% in a shallow-accumulation benchmark) but that cost is paid once per optimizer step: at 7 gradient-accumulation steps it dilutes to ~4%. Measured against total run time the convergence is much faster overall.

ClimbMix instead of FineWeb-Edu. This was a tremendous jump in convergence speed. Exactly as Karpathy found as well.

FP8 + vocab padding. FP8 training via torch._scaled_mm with dynamic tensorwise scaling on all three GEMMs, and padding the vocab from 50,257 to 50,304 (a multiple of 64) so the tensor cores are happy. Together, +33% throughput mostly from fp8.

1024 context instead of 2048. Halving the context roughly doubles the batch size at fixed memory. Throughput barely changes per token. We are still dominated by the MLPs which is a good sign we are using the hardware effectively. Below we will discuss the impact of the context length on the model.

Here is the whole run:

~480,000 tokens/sec in steady state, which puts 57.3B tokens at 33 hours. The wall clock was 35.9h. The difference is the CORE evaluations, which took about 15 minutes each (ten of them over the run) and consumed 7% of the total.

Re-running this identical recipe at 2048-token context scored 0.3840. Almost all of that gap turned out to be some tasks that were very context dependent.

On the GPUs themselves: 92% SM activity, 40% SM occupancy. High activity means the SMs almost never went idle. No dataloader starvation or network waits, which is the payoff for downloading the shards locally instead of streaming, which would leave us vulnerable to a small hugging face network hang. The low occupancy is what back-to-back large GEMMs look like: matmul kernels trade occupancy for register-tile size on purpose. Compute-bound and well fed, great signal we are using the hardware well and we can extend every dollar we spend into a better model.

That’s about 1,047 TFLOP/s sustained per B200, or ~25% MFU against Blackwell’s dense FP8 peak. (Against the bf16 peak it reads as 50%, which is the number that matters a bit more because not even all the linear layers run in FP8.)

The distributed strategy is plain old DistributedDataParallel. At 3.8B on a single node, gradient communication was never the constraint, and the sharded-optimizer machinery turned out to be unnecessary.

Increasing throughput

Renting GPUs isn’t cheap, at work you often think about the quality of the model before its cost. When it’s your own money burning, throughput matters a lot more all of a sudden.

This took real work on a single RTX 5090, before I ever rented a node. Baseline 858M model, bf16, compiled: 26,144 tok/s. Final: 37,621 tok/s.

FP8 (+25%). All three GEMMs (1 forward and 2 backwards) in FP8 with dynamic tensorwise scaling. Requires SM90+ but that is quite a nice throughput jump.

Vocab padding (+33% cumulative). Padding 50,257 → 50,304 costs 47 unused embedding rows and unlocks the fast tensor-core path. Nearly free.

Fused linear cross-entropy (+44% cumulative). Liger’s FusedLinearCrossEntropyLoss fuses the lm_head matmul into the loss and chunks internally, so the full (B*T, vocab) logits tensor is never materialized. Measured head-to-head at the same batch size it is 6% slower:

Even though it’s slower per step, it buys back a good amount of VRAM (8 GB on my 5090) so the increase in micro-batch size more than makes up for the lost 6%. Claude was quick to reject it because it was 6% lower, but overall it was a great way to claw some extra throughput.

Non-gated MLPs. Dropping the gate projection (SwiGLU → relu², two matmuls instead of three) on the small model: 183,035 → 214,173 tok/s and 6 GB less VRAM. One caveat from the ablations: a SwiGLU intermediate ratio of 2.75 does not transfer to relu². The model learns noticeably worse. Use 4× for non-gated.

bf16 master weights. Keeping the optimizer master weights in bf16 rather than fp32 cut VRAM 27% and raised throughput from 640K to 1.4M tok/s on the 1.5B config. That was a huge speed-up, 2.2×. The quality cost is real but small: CORE 0.22 vs 0.23 at 4,000 steps. When you’re optimizing for capability per dollar, careful dtype handling is one of the highest-leverage and underdiscussed knobs available.

Hardware. Same code, 150M model, FP8: RTX 5090 at 184,662 tok/s, B200 at 477,440 tok/s. 2.59× from hardware alone, before accounting for the extra VRAM letting you push batch size further.

What didn’t work

Document-boundary masking with flex attention. Packing documents into one sequence lets tokens attend across boundaries, so I fixed it properly: per-token document IDs and mask out attention so each token can only attend to its current document. It was elegant, but I deleted all of it. Andrej Karpathy also found that cross-document leakage does not make things much worse under BOS-aligned packing. Best-fit packing replaced it in ~10 lines, and attention went back to an unconditional F.scaled_dot_product_attention(..., is_causal=True). I believe this is also conditional on the dataset and the training documents.

Liger RMSNorm and RoPE. RoPE was 2.2× faster in a microbenchmark and produced no measurable change in end-to-end throughput. RoPE is not part of the critical compute bottleneck at this scale. RMSNorm was outright slower than PyTorch 2.9’s built-in F.rms_norm (0.41ms vs 0.25ms). Both reverted, not worth the complexity.

Nanochat-style initialization. Embeddings at N(0, 0.8), linear weights uniform, output projections zero-initialized so the residual stream starts as pure identity, LM head at N(0, 0.001). Theoretically much nicer than GPT-2’s N(0, 0.02) everywhere. The loss curve starts marginally lower and the two curves overlap by ~1,500 steps. No measurable quality difference. I kept it, but for aesthetics, not evidence.

Streaming datasets. Great for getting started, wrong for a real run. Even when the network looks healthy, local shards gave 2-3% more throughput, and occasional network dips cost far more than that. For runs longer than a few hours, it’s worth it to pay the download once at the start of training.

Ablation on value-embedding

Value embeddings were 721M parameters for a 3.8B model. I trained the same model with the same config with value_embeddings: false and compared it against the original run, which I’d already paid for, out to 12,500 steps and 29B tokens.

0.46% better loss and 3.2% better CORE, for 19% more parameters. The throughput is identical, because value embeddings are lookups. They cost memory and optimizer state but essentially no FLOPs.

Two interesting findings:

• Value embeddings bought the equivalent of about 1,200 training steps. Here is how to price that: between steps 10,000 and 12,500 my baseline loss fell 0.0194, so 2,500 steps buys roughly that much. The value-embedding advantage is 0.0096, about half of it — call it 1,200 steps out of 25,000. So 19% more parameters is worth ~5% more training.
• CORE moved about seven times more than loss did (3.2% vs 0.46%), and the gap shrank steadily during training. That’s worth knowing if you’re using CORE to make decisions: it’s an accuracy metric, so items near the decision boundary flip on tiny logit changes, and it’s centered against a random baseline, which amplifies relative differences while scores are still low.

Value embeddings are useful for a small model and come at almost no throughput cost. Spending a little bit of VRAM on this gives the model a form of bias toward certain concepts that might be useful for CORE.

Discussion

Misleading micro-benchmarks

We could be tempted to believe that 1024 tokens context is plenty for a high CORE score. Going back through the per-task logs, that conclusion is wrong on some tasks that are very context sensitive.

3 of the 22 CORE tasks have prompts that essentially never fit in 1024 tokens:

SQuAD is the striking one. It doesn’t stagnate, it decays monotonically to exactly zero: 0.1478 → 0.0617 → 0.0099 → 0.0007 → 0.0000. The model gets steadily worse at this task the longer it trains, which is not a thing models normally do.

Two details explain it. SQuAD is a 10-shot task in the DCLM bundle, so each prompt is ten worked examples followed by the real one. Median of 1,998 tokens on my eval data. Not one fits in 1024. And when a prompt is too long my harness keeps the last max_seq_len tokens.

The test passage sits at the end, so it always survived; a test example is only ~169 tokens. What got truncated was the ten demonstrations. The model was reading the passage and the question, and almost never seeing the examples that teach it the expected output format. Since SQuAD is scored on exact-token match against the gold answer, fluent prose scores zero every time.

That also explains the decline. An early, high-entropy model occasionally emits something short and generic that happens to match. As it sharpens it commits to well-formed continuations, and the accidental hits disappear. Funnily enough, getting better at language made it worse at guessing right by accident.

boolq shows a gentler version of the same shape. It peaks at step 10,000 (0.6294) and declines to 0.5131. Language identification never moves off chance at all.

In short, 0.338 was measured with three of 22 tasks scoring near-zero for reasons that have nothing to do with model quality, just the size of the context length being fed to it.

The effect of larger context

As we have seen, if we want the highest CORE score possible we need larger context. But this has consequences on the training throughput.

Double the context length, halve micro-batch to hold VRAM constant, so tokens per optimizer step stayed identical. I stopped it at ~28,000 steps to save the last few hours of rental, so the learning-rate warmdown never fully completed and the number below is a lower bound.

CORE went from 0.3384 to 0.3840.

At step 20,000 the two runs have the same eval loss to four decimal places (2.0160 vs 2.0164) and differ by 0.034 on CORE. It was surprising to see that low level of correlation between CORE and eval loss on the ClimbMix dataset.

squad and boolq alone are 83% of the gain. boolq contributes the most, because its random baseline is 0.5 and CORE centers against that: a raw +0.196 becomes a centered +0.517. Strip those two and the remaining twenty move +0.008 in total, roughly what 14% more tokens buys on its own.

Language identification went from 99.7% cropped to 14% cropped and moved +0.005. This is by far the hardest task in the CORE evaluation benchmark for our current model.

A couple of tasks got worse: commonsense_qa dropped 0.072, cs_algorithms 0.031. Across 22 tasks some movement in both directions is expected.

2048 was worth paying for as a measurement decision, not a quality one. It cost 9% throughput (480K → 437K tok/s), and outside the tasks that couldn’t be scored at 1024 it bought almost nothing. 1024 is fine for training and a “cheap” way of getting your model to a good CORE score. 2048 unlocks some tasks that are very context bound.

Future work

Limitations

Four things I never ablated. Peak LR, from nanochat’s sqrt(768/d_model). I didn’t really want to spend money to sweep learning rates. I moved from cosine to trapezoidal because of the 858M post-mortem, there could be schedules out there that are more efficient. QK-norm, on by default and never toggled off. And the GQA ratio, since it’s a nice lever to save on memory.

Most of those are inherited from nanochat rather than tested here. That is a defensible way to spend a small budget — someone else already paid for the experiment — but it means I am trusting that Karpathy’s results transfer to my model, data and scale.

Open questions

There is a lot of interesting work I’d want to pursue if I had more time and resources:

• Value embeddings versus reallocation. The comparison above was VE against nothing. The one that matters is VE against spending those 721M on something else.
• 1024 versus 2048 at matched wall-clock. The rerun changed context and ran longer, so it settles the measurement question but not the quality one.
• Why commonsense_qa regressed by 0.072 at the longer context, when nothing about that task involves long prompts.
• Sharding the optimizer, the way nanochat does. I used plain DDP with a single-GPU Muon, which means every rank holds a full copy of the optimizer state and redundantly recomputes the same Newton-Schulz update. nanochat drops the DDP wrapper entirely and does ZeRO-2 sharding inside the optimizer, overlapping reduce-scatter, compute and all-gather. The memory win is the certain one, and freed memory turns into batch size, which is tokens for the same dollars. Whether the redundant orthogonalization also goes away depends on how the sharding is done: Muon needs the full gradient matrix, so splitting a matrix across ranks doesn’t help, while giving each rank whole matrices of its own would. I haven’t explored that at all but I think it would be a great way to further increase the total training throughput at the cost of some extra machinery.
• Additional data exploration. I haven’t had a lot of time for data analysis on either the CORE benchmark or the ClimbMix dataset. I’m sure this would help us claw even higher performance with the same compute budget.

Closing thought

GPT-2 was a frontier result in 2019, produced by a well-funded lab with a large team, and its 1.5B model scores 0.2565 on CORE. 7 years later I beat that by a wide margin in my evenings, for $998, on hardware I rented by the hour.

The frontier moved, and everything came with it. Work that needed a lab can now be done by a single engineer in the evenings. I wonder what kind of insane machine we will be able to build in 7 years from now!

Appendix: the config

The whole run, flattened from the YAML includes into one block.

model:
  hidden_size: 3072
  intermediate_size: 12288      # 4x, non-gated
  num_hidden_layers: 28
  num_attention_heads: 24
  num_key_value_heads: 8        # 3:1 GQA
  head_dim: 128
  hidden_act: relu2
  gated_mlp: false
  qk_norm: true
  logit_softcap: 15.0
  layer_scale: true
  value_embeddings: true        # 14 tables, alternating layers
  tie_word_embeddings: false
  rope_theta: 10000.0
  rms_norm_eps: 1.0e-6
  vocab_pad_to: 64              # 50257 -> 50304
  max_position_embeddings: 2048
  dtype: bf16

engine:
  compile: true
  fp8: true
  precision: bf16
  total_batch_size: 2293760     # 20 x 2048 x 7 grad_accum x 8 GPUs
  loss: LigerFusedLinearCrossEntropyLoss(softcap=15.0)

optimizer:                      # composite, one group per parameter class
  matrix:        Muon   lr=0.02      momentum=0.95  wd=0.0
  embeddings:    AdamW  lr=0.1414    betas=(0.8, 0.995)  eps=1e-10  wd=0.001
  lm_head:       AdamW  lr=0.002828  betas=(0.8, 0.96)   eps=1e-10  wd=0.01
  value_embeds:  AdamW  lr=0.0707    betas=(0.8, 0.995)  eps=1e-10  wd=0.01
  scalars:       AdamW  lr=0.005     betas=(0.8, 0.95)   eps=1e-10  wd=0.05

scheduler:
  trapezoidal:
    warmup_ratio: 0.05
    warmdown_ratio: 0.50
    final_lr_frac: 0.05

data:
  dataset: nvidia/Nemotron-ClimbMix  (karpathy/climbmix-400b-shuffle shards)
  tokenizer: gpt2 (tiktoken)
  block_size: 2048
  packing: best-fit, BOS-aligned
  batch_size: 20 per rank
  num_workers: 11

trainer:
  max_steps: 32000              # stopped at ~28,000 -> 65.3B tokens
  eval_every: 4000              # must divide max_steps or the final CORE is skipped

The AdamW learning rates follow nanochat’s sqrt(768/d_model) scaling rule; the Muon LR of 0.02 is inherited from there too.

Appendix: example text generation

The capital of France is Paris. It is the largest city in France and the second largest city in Europe
The french revolution happened in 1789 and 1799, and was a time of great change in france
At the center of the milky way there is a supermassive black hole. It is called Sagittarius A* (pronounced
Electrons orbit around the nucleus of an atom in a series of energy levels. The energy levels are numbered
Newton discovered the laws of motion and gravity. He also discovered the law of universal gravitation. Newton's

Summary

Improve startup and warmup time by making optimized native code for an application instantly available when the HotSpot Java Virtual Machine starts. Achieve this by compiling application code to native code in a training run, storing the native code in the AOT cache for use in subsequent production runs. If the workload changes in production, regenerate native code dynamically for continued peak performance, providing the best of both ahead-of-time (AOT) and just-in-time (JIT) compilation.

Goals

• Enable applications to achieve peak performance more quickly.
• Enable applications to sustain peak performance even as workloads change.
• Do not require any change to the code of applications, libraries, or frameworks.
• Do not require any change to the configuration of HotSpot, beyond requesting the use of the AOT cache.
• Continue to support the Serial, Parallel, G1, and ZGC garbage collectors.
• Do not introduce new AOT workflows, but, rather, extend the existing AOT cache creation workflow.
• Ensure that shifting from AOT-compiled code to JIT-compiled code is invisible to applications.
• Support the AArch64 and x64 processor architectures.

Non-Goals

• It is not a goal to provide an AOT-only mode. Applications will use both AOT-compiled code and JIT-compiled code in the same run, automatically transitioning between them as needed.
• It is not a goal to support cross-compilation. Code compiled in a training run must run on the same CPU architecture, with the same set of features, in subsequent production runs.
• It is not a goal to support all CPU architectures currently supported by HotSpot. We expect normal porting activities to eventually add support for all major architectures.

Motivation

When a Java application is run by the HotSpot JVM, it passes through three phases: It starts up, then it warms up, and then it reaches peak performance.

During startup, HotSpot invokes the application's main method and loads, links, and initializes classes on demand. Initially, it runs both application and JDK library code via the bytecode interpreter, which is slow. Inside the interpreter, HotSpot profiles the application's behavior by counting events such as method invocations and loop iterations. It uses the profile data to select frequently-invoked methods, or hot spots, which it compiles to native code via the basic C1 compiler. This native code is only modestly optimized.

During warmup, the application settles into its workload and the loading, linking, and initialization of classes ramps down. HotSpot continues to profile the application, both in the bytecode interpreter and via instrumentation code inserted by C1. It collects richer profile information, including not just method-invocation and loop-iteration counts but also the types of objects encountered. As the profile data accumulates over time, it becomes more statistically useful. Eventually, HotSpot uses the data to select the hottest methods, which it compiles to native code via the advanced C2 compiler. This native code contains no instrumentation and is highly optimized.

Profiling the application and generating native code is not free. Not only is the bytecode interpreter slow, but instrumented native code is slower than uninstrumented native code. Compiling methods to native code requires both CPU time and memory that could otherwise be used by the application, even though HotSpot only compiles methods to native code when profile data indicates that doing so will be worth the effort. Gradually, however, JIT compilation catches up to the application's emerging hot spots and the application runs faster. Eventually, all hot methods are compiled to fully optimized native code and the compilers go idle.

The application remains in this state of peak performance as long as its hot spots do not change. The application's hot spots may change, however, in response to changes in its workload. When that happens, HotSpot can dynamically deoptimize, by discarding previously generated native code as needed, and reoptimize, by generating new native code for newly-hot methods. For example, if an application initially receives two types of requests then HotSpot dynamically optimizes the code for those two request types. If the application starts receiving a third type of request, HotSpot can dynamically deoptimize and then reoptimize the code for all three types of requests. The application can pass, in effect, through another warmup phase, sustaining performance as the application's workload changes.

What about static compilation?

Static compilation has sometimes been proposed as an alternative to the dynamic compilation of Java code. A static compiler converts entire applications to native code ahead-of-time, prior to run time.

Static compilation has some advantages over dynamic compilation. A statically compiled application starts up and reaches peak performance immediately, without a warmup phase. At run time there is no need for a bytecode interpreter, profiling, or compilation. Peak performance can even be competitive with HotSpot if the static compiler's optimization work is guided by accurate profiles gathered during prior runs.

Dynamic compilation, however, has three key advantages over static compilation.

First, dynamic compilation makes applications agile because it responds to changes in the application's hot spots. It deoptimizes and reoptimizes as needed, sustaining performance as the application's workload changes. A statically compiled application cannot respond in this way — by its nature, it can be optimized for only one set of hot spots.

Second, dynamic compilation makes applications portable across varying hardware and software because it generates native code at run time that is specific to the run-time environment. If an application is redeployed on a different processor architecture, a processor with a different feature set, a different operating system, or a different version of the JDK, HotSpot will achieve peak performance for that environment without requiring any change to the application. A statically compiled application must be recompiled in the face of such changes.

Finally, dynamic compilation is compatible with the dynamic nature of the Java Platform. Features such as dynamic class loading, dynamic linkage, dynamic dispatch, and dynamic reflection bring vast expressive power, and have been fundamental to the platform's success. HotSpot handles these features naturally, while static compilers struggle with them. Even heroic amounts of static analysis cannot make up for the fact that these features require many decisions to be made at run time. Implementors of static compilers for Java code have therefore resorted to incompatible constraints, such as closed-world assumptions, and to putting significant burdens on developers, such as having to identify in advance the classes eligible for reflection.

Shifting compilation work to training runs

Throughout the startup and warmup phases, HotSpot continuously juggles multiple balls: It runs application and JDK library code; it loads, links, and initializes classes on demand; it profiles the application's execution; and it compiles hot methods to native code with varying degrees of optimization, guided by the profile data.

The thesis of Project Leyden is that the key to improving startup and warmup time is to do some of this work earlier, ahead of time, rather than just in time. We shift work earlier in time by doing it in a training run, storing the results of the work in the AOT cache for instant use in subsequent production runs.

We shifted class loading and linking work earlier in time via JEP 483, delivered in JDK 24. The AOT cache stores the loaded and linked forms of classes from the training run, thereby improving startup time.

We shifted profiling work earlier in time via JEP 515, delivered in JDK 25. The AOT cache stores the execution profiles of methods invoked in the training run, enabling the C2 compiler to run immediately at the start of production runs, thereby improving warmup time.

These improvements laid the foundation for our ultimate goal, which is to shift compilation and optimization work earlier in time. The AOT cache will store optimized native code compiled in the training run, enabling HotSpot to load that code instantly, rather than having to recompile it at the start of each production run. This will improve both startup and warmup time.

HotSpot will not always use the cached code; if the application's workload changes then HotSpot can, as usual, deoptimize and reoptimize, generating new native code for newly-hot methods in order to sustain performance. Thus Java applications will gain some of the benefits of static compilation while retaining the agility, portability, and compatibility of dynamic compilation.

Description

We extend the existing AOT cache to store optimized native code generated in a training run. Such cached code is known as AOT code. During a production run, a request for optimized code for a method can be fulfilled instantly if matching AOT code is found in the cache. If AOT code is unavailable, incompatible, otherwise unsuitable, or later deoptimized, execution falls back to the existing interpreter and JIT mechanisms. AOT code and JIT code can coexist and are completely interoperable since they are created by the same compilers, C1 and C2.

To create an AOT cache, use the AOTCacheOutput option to do a training run of your application and generate AOT code:

$ java -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ...

This workflow is unchanged from previous releases. The AOT cache in the file app.aot, however, now contains not just pre-linked classes and profiling data but also AOT code for selected hot methods. Subsequently, in production, you can run the application with the cache:

$ java -XX:AOTCache=app.aot -cp app.jar com.example.App ...

No additional options or settings are required to generate or use AOT code. HotSpot creates AOT code and stores it in the cache by default. It continues to store profile data in the cache as well, to be used for sequencing the loading of AOT code and for guiding the subsequent generation of JIT code.

Performance

To evaluate the startup benefit of AOT code, we ran five benchmark applications built with popular Java frameworks. We ran them on a two-core Linux/x64 system so as to emulate a microservice setting in which the JIT compiler is likely to compete with the application for CPU time, thereby increasing startup time:

Without AOT code, the AOT cache reduces the startup time of these applications by around 50% to 70%; with AOT code, the cache reduces their startup time by around 65% to 80%.

To evaluate the warmup benefit of AOT code, we ran a javac benchmark application which repeatedly compiles the same 50 source files, twenty times, measuring the time required for each iteration:

In each curve, the first iteration shows the startup-time improvement: The AOT cache without AOT code improves startup time by about 30%; adding AOT code brings an additional 45% improvement, for a total of about 75%. Successive iterations show the warmup phase, during which HotSpot compiles the hottest methods: The iteration time tends to decrease and then reach a steady state as the quality of the native code improves and the compilers finish their work. The curve for the AOT cache without AOT code decreases more quickly than the curve for no AOT cache, eventually reaching roughly the same steady state. The curve for the cache with AOT code is already close to the steady state by the fourth iteration. The area between the top curve and the bottom curve represents the total warmup-time improvement.

Information on all of these benchmarks, including run instructions and links to source code, is available here.

Differences between AOT code and JIT code

AOT code and JIT code can be different, since training runs and production runs can be different.

One source of differences is the fact that the order in which classes are initialized can differ between training and production runs, especially if the workload differs. A method that accesses a static field or invokes a static method in another class must ensure that the class is initialized. When generating AOT code with C2, HotSpot therefore compiles two versions of such methods: A slow version contains extra code to ensure the initialization of referenced classes, while a fast version does not contain that code and thus can be better optimized. HotSpot uses the slow version initially, and then switches to the fast version once all referenced classes are initialized.

Another source of differences is the fact that a static final field's value can vary from run to run; it might, for example, be initialized with the current date and time. When just-in-time compiling a method that refers to such a field, the class containing the field will have been initialized, so the field's value will be known and C2 can treat it as a compile-time constant, embedding it directly in native code. When ahead-of-time compiling the same method, however, no classes will have been initialized, so the field's value will not be known and C2 cannot treat it as a compile-time constant; it must generate code that explicitly loads the field.

Despite these differences, AOT code still delivers significant performance benefits in a manner that is transparent to applications. As always, at run time the compilers can generate JIT code to replace AOT code that fails to age well.

Consistency of training and production runs

To enjoy the benefits of the AOT cache generated in a training run, the training run and all subsequent production runs must be essentially similar, as described in JEP 483.

If an AOT cache contains AOT code, that code is used when two additional constraints are met:

• All runs use CPUs of the same architecture and with the same features. For example, AOT code generated for an x64 CPU with the AVX-512 vector-instruction feature will not run on an x64 CPU without that feature.
• All runs use the same garbage collector, since AOT code contains GC-specific read/write barriers.

If these constraints are not met then HotSpot issues a warning message and does not load the AOT code, falling back to the usual interpreter and JIT mechanisms. It still uses the other information in the AOT cache, namely the loaded and linked classes and the profiling data. In that case, the application may start up and warm up more slowly, but its execution will still be correct and it will still, eventually, achieve peak performance.

Observing AOT cache usage in production

You can observe whether AOT code is loaded in a production run via the existing HotSpot option PrintCompilation, which now reports on both AOT code loading and JIT compilation:

$ java -XX:+PrintCompilation \
  -XX:AOTCache=app.aot -cp app.jar com.example.App ...

You can also check whether an AOT cache containing AOT code is usable in a particular production environment. The option AOTMode=required causes HotSpot to report an error and exit if the AOT cache violates any constraint:

$ java -XX:AOTMode=required \
  -XX:AOTCache=app.aot -cp app.jar com.example.App ...

(This option was originally AOTMode=on, later renamed for clarity to AOTMode=required.)

Finally, you can disable the loading of AOT code via the diagnostic option AOTCodeCaching:

$ java -XX:+UnlockDiagnosticVMOptions -XX:-AOTCodeCaching \
  -XX:AOTCache=app.aot -cp app.jar com.example.App ...

You can use this option to evaluate the performance effects of AOT code in a production run, or to avoid constraint-violation errors when using AOTMode=required and the AOT code is unusable but the other information in the cache is usable.

Controlling the generation of AOT code in training

We have, thus far, spoken of an AOT cache being created in a training run, in a single step via the AOTCacheOutput option as shown above. In fact, creating an AOT cache takes two distinct steps: HotSpot runs once in record mode to store observations of your application's behavior in an AOT configuration, and then again in create mode to assemble an AOT cache from that configuration, which includes compiling AOT code. Using the AOTCacheOutput option runs the second step transparently, but you can invoke each step explicitly via the AOTMode and AOTConfiguration options:

$ java -XX:AOTMode=record -XX:AOTConfiguration=app.aotconf \
       -cp app.jar com.example.App ...
$ java -XX:AOTMode=create -XX:AOTConfiguration=app.aotconf \
       -XX:AOTCache=app.aot

HotSpot provides many options to control the behavior of its compilers, C1 and C2. These options uniformly govern the generation of both JIT code, at run time, and AOT code, in the assembly step. For example, this command line reports all JIT compilation activity during the training run proper and all AOT compilation activity as the cache is created:

$ java -XX:+PrintCompilation \
  -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ...

To see AOT compilation activity but not JIT compilation activity, use the existing environment variable JDK_AOT_VM_OPTIONS to pass the PrintCompilation option only to the assembly step:

$ JDK_AOT_VM_OPTIONS='-XX:+PrintCompilation' \
  java -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ...

Alternatively, you can run the two steps yourself, specifying the PrintCompilation option only in the second step.

Finally, you can disable the creation of AOT code in a training run via the diagnostic option AOTCodeCaching:

$ java -XX:+UnlockDiagnosticVMOptions -XX:-AOTCodeCaching \
  -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ...

You can use this option to evaluate the size impact of AOT code on the AOT cache — cached code can make the AOT cache significantly larger.

For more information on all of the AOT-related command-line options, see the manual page for the java command.

Future Work

• Investigate minimizing bytecode interpretation and JIT compilation in favor of near-total reliance on AOT code. Initial experiments suggest that minimizing interpreter use results in overly large AOT cache files, which can take more time to load than just running the interpreter. Likewise, minimizing JIT compilation often leads to lower peak performance. This approach may have limited applicability if, like static compilation, it fails to live up to user expectations.
• HotSpot has a wide range of fine-grained options for controlling its compilers. Based on experience with this feature, tune the default values of existing options to apply more gracefully to AOT code. Also consider defining new options, such as options for explicit AOT cache size management.
• Consider an option that would enable giving up some performance, or accepting larger AOT cache files, or both, in order to gain portability across processors of the same architecture but with different feature sets.

Testing

• We will create new unit tests for this feature. They will ensure that AOT code, if present, behaves correctly.
• We will run existing AOT cache tests with this feature enabled and ensure that they pass.
• Initially, only AArch64 and x64 processors will be supported. Unit tests will be adjusted appropriately to allow for the absence of AOT code on other architectures.

Risks and Assumptions

• There are no new risks beyond those already noted in JEP 483.
• We assume that HotSpot's organizing principle is still sound: A Java application should be compiled, at run time, to favor the application's actual behavior, exploiting all of the processor features available. Because of this principle, Java code automatically runs better on new hardware.
• The base assumption of the AOT cache also remains operative: A training run is assumed to be a good source of observations that, when passed through an AOT cache to a production run, will benefit the performance of the production run. This assumption applies fully to AOT code, which benefits similar production runs while not doing harm to divergent production runs, which can use JIT compilation to generate different code.

Neki is now available in platform preview.

Neki is built from lessons we’ve learned over eight years of running some of the largest sharded MySQL clusters in the world. Thousands of production workloads with millions of queries per second for companies where even a few seconds of downtime is a very public event. We know what it means to power the world’s biggest tier 0 workloads.

When we released PlanetScale Postgres a year and a half ago, we knew we needed to do more. In that time we’ve onboarded several thousands of customers on PlanetScale, some of them rivaling the size of our largest MySQL customers. Time and time again, we watched teams approach the ceiling of a single machine with Postgres. Metal bought them time, but with customers hitting the upper limit of what a single machine is capable of, we found there was no good option to hand them. Enter Neki.

What is Neki?

Neki is sharded Postgres from PlanetScale. It lets you scale a Postgres database across many machines while keeping real Postgres on every shard.

Your application connects to a Neki router over the standard Postgres wire protocol, so your existing drivers, ORMs, and connection string keep working. Each shard is a full Postgres cluster with one primary and at least two replicas across 3 availability zones. There is no custom storage engine, so extensions, SQL support, and performance behave the way Postgres does.

You choose the shard key and control how tables are grouped and distributed through a JSON data topology. Schema changes, version upgrades, failovers, imports, and resharding all run as built-in fully online workflows. You also get the PlanetScale features you already rely on, including Insights, schema recommendations, branching, and MCP.

You don't have to shard on day one. Run Neki as a single primary with replicas, and when you outgrow one machine, resharding is a workflow you run against the cluster you already have.

Why Neki?

You already know the problems that come with fast-growing Postgres databases: tables too large to vacuum or index without affecting traffic, backups taking hours, connection limits, maintenance windows for schema changes, transaction wraparound and so much more.

You can move to a bigger instance, but eventually you run out of big enough machines, and the problems don’t scale linearly as you add more cores and IOPS.

The existing answers each ask you to give something up. Application-level sharding pushes routing into your code. Postgres-”compatible” distributed databases hide the shard key from you, take away your extensions, and add complexity and latency which becomes difficult to handle and debug.

So we built Neki with a few principles, the biggest one being: stick to Postgres, don’t work around it, fake it, or turn away from it.

How does Neki work?

We architected Neki from first principles for Postgres, with real Postgres on every shard. There are four moving parts.

Neki routers

Your application first connects to a Neki router. The router speaks the Postgres wire protocol so your existing drivers and ORMs keep working with a single connection string. A router has a full Postgres query parser, a distributed query planner, query buffering and more. It parses your query, builds a plan that decides which shards should run it, sends the work out, and combines the results back into one stream. Routers can scale vertically and horizontally, so no single router becomes the bottleneck.

Sharding and shard groups

Every shard in Neki is real Postgres with 1 primary and at least 2 replicas, spread across availability zones. There is no modified storage engine. Extensions, SQL support, and performance behave the way Postgres behaves, because it is Postgres.

Shards are organized into shard groups, so different tables or workloads can live on different sets of shards. Each shard uses a configuration profile that defines its instance size, replica count, storage, Postgres parameters, and extensions, so you can size each group for its own traffic.

Connection pooling

Sidecars run alongside every Postgres instance. This is the piece that makes Neki's connection handling meaningfully better than just sticking PgBouncer in front of a database. Because Neki controls both ends of the connection, the router side and the Postgres side, it can size pools to what each instance can actually serve instead of estimating from outside the process.

Control plane

The control plane tracks the health of every node, runs planned switchovers and unplanned failovers, and coordinates the workflows that reshard data, apply schema changes, and perform version upgrades.

Data topology

Tying it together is the data topology, a JSON configuration that maps your logical tables onto physical shards. You define shard indexes, which specify the column Neki routes on and how that value gets hashed, and shard groups, which control how many shards a set of tables spreads across and which shards those are. Routers cache the topology and consult it on every plan.

What you get beyond sharding

Everything you would normally schedule a maintenance window for runs as a built-in workflow in Neki. Workflows provision new target nodes, catch them up with replication, switch traffic with a __neki metafunction, and retire the old nodes. All through the same psql connection your application uses.

This online operations model covers schema changes, version upgrades, planned and unplanned failovers, imports, and resharding.

Neki also includes all of the features you’ve come to rely on with PlanetScale: Insights, schema recommendations, branching, MCP, and more.

You can also run Neki unsharded, as a single primary with replicas. You get the improved connection pooling, online DDL, zero downtime upgrades, and health monitoring before you need to shard. When you do, resharding is a workflow you run against the cluster you already have.

What is a platform preview?

We wanted to get Neki into your hands as soon as possible. You should not run production workloads on Neki during the platform preview. The product is still changing, and some of those changes will be breaking.

If you have any feedback, questions, or face any issues during the platform preview, please let us know. Fill out a support ticket or join our Discord

Try Neki today

Sign in to PlanetScale, opt in to the platform preview, and create a Neki cluster. Read the Neki docs for more information about Neki's architecture, how to shard, and more.

If you have a large Postgres cluster and are curious whether Neki is a good fit, get in touch. We would love to do a private demo for your team, dig into your schema and query patterns, and give you real suggestions on how to shard.

I have a little pet theory that software drives people insane.

Not in the "wash your hands every thirty minutes like Howard Hughes" kind of way, but more so in how the conditions surrounding software seem remarkably effective at making otherwise normal people lose their sense of proportion.

I've watched this happen enough times from enough angles that I don't think it's entirely a personality problem.

Software combines speed, money, complexity, abstraction, and almost unlimited freedom to change your mind. On their own, those things are perfectly manageable, but put them together and you start to get some really bizarre side-effects.

Most software, when you strip away the branding and architecture diagrams, is remarkably boring. A form here, an API endpoint there, sprinkle in some permissions, calculations, workflows, and a database somewhere. Maybe a queue if you need to scale something (or you're feeling adventurous).

But, harsh as it may be: most software is still just a glorified spreadsheet.

And yet somehow the process of producing this stuff can turn perfectly ordinary adults into b-tier Bond villains. The project is never moving fast enough, the plan has to be ever-flexible to sudden change, and every new feature is the one that's going to be the one (but certainly not that one that was supposed to be the one during the previous sprint).

There is always some new concern about whether it will work, whether it will scale, whether we're moving fast enough, or whether we should be doing something else entirely.

The weird thing about software is that many of these ideas are technically possible. That's part of the problem: there's very little natural friction between an idea and its implementation. If you're building a house and somebody decides halfway through framing that the kitchen should be on the opposite side of the building, everyone immediately understands that decision has a cost.

Boards have been cut, plumbing has been run, and people have to tear already-fixed things apart. The cost is physical enough that nobody can pretend it doesn't exist. With software, that cost hides inside people's heads and inside systems that are, typically, already difficult to reason about.

Moving the kitchen in software might look like a "simple fix." The work is still costly, but the actual expense accumulates quietly. Somewhere between context switching, regression risk, and architectural erosion we find lost momentum, forgotten assumptions, and endless meetings to "get aligned."

No matter the actual complexity, it's easy to pretend the change was free (as in beer) because there isn't any visible dust or scraps. The problem is made worse by the fact that changing software really can be cheap...sometimes.

A useful adjustment might genuinely take an hour, while another equally simple-looking request can ripple through an entire system causing failures. That opaqueness creates a dangerous habit where every "cool idea" that's "super quick" gets added to the roadmap, often with urgency.

Someone has an idea in a meeting and there is often very little resistance between the idea and reality. Could we make this screen work differently? Could we change the business model, chase another customer segment, introduce another workflow, or build our own event system?

The answer is almost always some variation of "sure, we could."

Eventually, "could" becomes "should," and "should" becomes "why isn't it done yet?" This is where software starts doing something strange to people's brains. Everything becomes urgent because everything can move quickly, and every decision starts feeling strategic because the theoretical upside can be enormous.

Every technical choice also becomes ideological because there are dozens of plausible ways to solve the same problem. Every slowdown begins to look like a crisis because somewhere, somebody else is (supposedly) moving faster. The industry has very few natural mechanisms that tell people when enough is enough.

There isn't really an obvious definition of "done" in software (I'll reluctantly share a clip from The Social Network that illustrates this attitude well).

A carpenter eventually puts down the hammer because the cabinet exists, but software can always be improved. The button could be better, the query could be faster, the abstractions could be cleaner, the onboarding could convert more people, and the infrastructure could scale further. The product could expand into an adjacent market, the pricing could change, or the entire company could decide it has discovered a more lucrative direction.

The point: if you want there to be, there's always another lever within reach.

I think this is one reason software organizations become so neurotic: they are surrounded by levers, and people who are surrounded by levers eventually start pulling them.

Sometimes they pull them because something is genuinely wrong. Other times they pull them because they're scared, because the board wants growth, because a competitor shipped something, because the numbers were flat this month, or because nobody knows what else to do.

A founder changing direction every few days gets mythologized as "responding to the market." A manager constantly pressuring people to move faster is considered execution focused, while an engineer introducing several new infrastructure components may be praised for thinking about scale. A product team rebuilding a working interface because conversion dropped slightly is iterating, and a company abandoning its original identity because another category is suddenly fashionable is pivoting.

Some of this language exists because the underlying motivation is legitimate. There are times when you should move quickly, times when you should pivot, and times when the architecture really does need to change. What makes software dangerous is how easy it is to confuse the existence of a take-able action with the need to actually take it.

And if this wasn't bad enough, money just pours gasoline on the whole thing.

There are very few industries where a small group of people can sit in a room, type for a few years (or now, steer agents), and plausibly produce something worth hundreds of millions of dollars. That possibility changes the emotional weight of what is traditionally mundane work (to anyone who isn't a nerd).

You can watch sensible people argue for an hour over a button because, somewhere deep inside the conversation, the button has become connected to a future pile of money.

Once that happens, ordinary judgment is thrown to the wolves. The work stops being about whether the button is tied to something useful and starts carrying the weight of everything people hope the company might become.

Complexity rushes in to fill the same gap. Software is unusually good at making complexity feel important, because complicated systems make ordinary problems seem more serious.

A boring application that stores records in a database and lets people edit them doesn't sound especially impressive, while a distributed event-driven platform with a service mesh and realtime synchronization layer sounds like you're building NORAD.

Sometimes you really do need the complicated thing. Most of the time, you probably don't. But complicated systems provide psychological rewards that simple systems don't. They give people things to design, debate, own, optimize, rewrite, diagram, benchmark, and talk about.

Complexity creates work and work creates an air of importance. Importance creates status, and before long the system exists partly to support the organization that exists partly to support the system. The whole thing becomes self-reinforcing in a way that is surprisingly hard to notice from the inside.

I suspect there is also a deeper attraction underneath all of this. Software gives us an unusual amount of control, because code is one of the few places where you can describe what you want with enough precision and have a machine reliably follow those instructions. The rest of reality is much less cooperative.

Reality is messy and stubborn, while software gives us the impression that messiness—technically or conceptually—is a problem waiting to be debugged.

So we start trying to debug everything around the software too. Growth is slow, so we change the funnel; customers are confused, so we redesign the product; development is slow, so we change the process; the process is slow, so we change the tools. The company is struggling, so we reorganize it, and if it is still struggling (and dependent on venture capital), maybe we pivot.

There is always another variable to manipulate.

Eventually the company itself starts being treated like software: permanently mutable, permanently unfinished, and permanently one refactor away from working properly. This is where the madness really settles in, because nobody can leave anything alone.

Leaving things alone is an underrated engineering skill. There is a point in your career where you begin to understand that a surprising amount of good work comes from refusing to touch things that are already doing their job. The database does not always need to be replaced, the framework is often fine, and the onboarding does not need another redesign this week.

The architecture does not need to anticipate a billion users, and the roadmap does not need to change because somebody got excited by a tweet. The product does not need to become a platform, and the company does not need to rediscover its identity every quarter. Sometimes the thing just needs to sit there and work.

Customers often need time to find a product, engineers need time to understand a system, and businesses need time to become businesses. None of this sounds particularly profound, but software culture is remarkably hostile to patience. Patience looks suspiciously like inactivity, and inactivity is difficult to justify in an industry obsessed with velocity.

So we create activity instead. We ship, iterate, optimize, pivot, replatform, rethink, and reinvent until the original problem is barely visible under everything we've layered on top of it. Then, a few years later, somebody quietly proposes rebuilding the original simple thing (often pointing to their own self-righteous wisdom as the guiding light that made that insight possible).

I don't think the answer is moving slowly for the sake of moving slowly. That would just be another ideology, and software already has enough of those (god help us). The answer is proportion.

It means understanding that not every problem is existential, not every idea belongs on the roadmap, and not every abstraction deserves to exist. It means accepting that not every slowdown requires intervention, not every competitor matters, and not every piece of software needs to become a platform. It also means remembering that not every company needs to seek World Domination.

I'll reiterate: most software is still, despite all this ornament, a glorified spreadsheet. That isn't intended as an insult. Spreadsheets are useful and (actually) useful software is enough.

But maybe we'd all be a little saner if we remembered what we're actually doing: building tools to make our lives and the lives of others easier. Not waste away in Margaritaville tinkering with shit that doesn't need tinkering with.

The larger Pacific striped octopus (LPSO), or Harlequin octopus, is a species of octopus known for its intelligence and gregarious nature.1 The species was first documented in the 1970s and, being fairly new to scientific observation, has yet to be scientifically described. Because of this, LPSO has no official scientific name.23 Unlike other octopus species which are normally solitary, the LPSO has been reported as forming groups of up to 40 individuals.456 While most octopuses are cannibalistic and have to exercise extreme caution while mating, these octopuses mate with their ventral sides touching, pressing their beaks and suckers together in an intimate embrace.7 The LPSO has presented many behaviors that differ from most species of octopus, including intimate mating behaviors, formation of social communities, unusual hunting behavior, and the ability to reproduce multiple times throughout their life.8 The LPSO has been found to favor the tropical waters of the Eastern Pacific.2

Appearance

LPSO exhibits three main body color patterns that include completely pale, completely dark brown, and a dark brown and white pattern that is referred to as "stripe-bar-spot". 2 When displaying the stripe-bar-spot body color pattern, LPSO is observed to have bar and stripe patterns that alternate between brown and white across its head and mantle, and dark brown with white spots over its arm, crown, and arms.2 The pattern can serve as a unique identifier as it varies among individuals.2 Though LPSO has similar body color patterns to other octopuses like Octopus chierchiae, 9 Octopus zonatus, 10 Abdopus spp., 11 Thaumoctopus mimicus, 12 and Wunderpus photogenicus, 12 the body patterns it exhibits are unique to the species.

Habitat and distribution

LPSO has been documented in many locations throughout the tropics of the Eastern Pacific. Specimens have been located in the Bay of Panama, Panama, Guatemala, Magdelena Bay, Baja Sur, Mexico, the northern Pacific coast of Colombia, and Nicaragua.2 LPSO seems to prefer denning in areas with soft muddy substrates or a mix of mud and sand, ranging in depth from 7m to 100m.2 The species has also been observed to form colonies of up to 40 close-living individuals.4 These colonies of LPSO construct dens within 1m of one another and live in the same general area for the span of their lives.4

Mating

LPSO is known in particular for its display of mating behaviors that are vastly different from most octopus species. While most octopuses use either "distance mating" (when the male places his hectocotylus into the female's mantle from a safe distance) or a mounting position,131415 LPSO uses a more intimate approach. Typically, a brooding female that is ready to mate exhibits an entirely pale body color pattern while the male exhibits a high contrast stripe-bar-spot body color pattern.2 Once the male and female LPSO decide to mate, they join in a beak-to-beak position, embracing in an entanglement of arms.2 Unlike some species of octopus, female LPSO are not cannibalistic during mating and are rather gentle, leaving only surface level sucker marks as a result of the beak-to-beak embrace.2

Pair bonding has been observed in LPSO through behaviors such as food and den sharing.2

Brooding

Unlike most octopus species, LPSO can spawn multiple times throughout the span of their life.2 Eggs are laid in pairs within the den and the female tends to them until they hatch, running the tips of her arms over them to maintain the health of the eggs.2

Once hatched from the egg, the offspring are pelagic until adulthood.16

Hunting

LPSO has been observed to prey upon a variety of shellfish including shrimp, stomatopods, crabs, and bivalves.2 They use hunting methods of stalking, chasing, and ambush depending on the prey item in question.2 When eating bivalves, smaller prey is either crushed or pulled apart, while holes are drilled into larger prey.2 When hunting shrimp, LPSO carefully extends its arm, suckers positioned outward, to touch and capture the shrimp.2 When hunting crabs, LPSO uses an ambush method, pouncing directly onto the crab for capture.2

Lifespan

LPSO live about two years.3

Like most octopus species, LPSO goes through a stage of senescence, which marks the approach of death.2 Senescence in LPSO has been observed to be longer for females and shorter for males.2 After the first signs of senescence, males will typically last 1 to 2 weeks before death, while females will typically last between 2 and 4 months.2

References

1. Ghose, Tia (6 March 2013). "PHOTOS: Rare Kissing Octopus Unveiled For The First Time". HuffPost. Archived from the original on 14 April 2019. Retrieved 14 April 2019.

2. Caldwell, Roy L.; Ross, Richard; Rodaniche, Arcadio; Huffard, Christine L. (12 August 2015). "Behavior and Body Patterns of the Larger Pacific Striped Octopus". PLOS ONE. 10 (8) e0134152. Bibcode:2015PLoSO..1034152C. doi:10.1371/journal.pone.0134152. ISSN 1932-6203. PMC 4534201. PMID 26266543.

3. Rowan Hooper (21 December 2019). "Octopuses were thought to be solitary until a social species turned up". New Scientist. Archived from the original on 27 December 2019. Retrieved 14 June 2024.

4. Moynihan, Martin. (1982). The Behavior and Natural History of the Caribbean Reef Squid, Sepioteuthis Sepioidea, With a Consideration of Social, Signal, and Defensive Patterns for Difficult and Dangerous Environments. P. Parey, 1982. ISBN 978-3-489-61936-9. OCLC 1086727999.

5. Graber, Shane. "Forgotten octopus rejects solitary lifestyle". Archived from the original on 14 April 2019. Retrieved 14 April 2019.

6. Harmon, Katherine. "Rare Social Octopuses Break All the (Mating) Rules [Video]". Scientific American Blog Network. Archived from the original on 14 April 2019. Retrieved 14 April 2019.

7. States379-8000, CA 94118 United. "Larger Pacific Striped Octopus". California Academy of Sciences. Archived from the original on 14 April 2019. Retrieved 14 April 2019.

8. Harmon Courage, Katherine (30 July 2014). "Social Octopus Species Shatters Beliefs About Ocean Dwellers". National Geographic News. Archived from the original on 1 August 2014. Retrieved 14 April 2019.

9. Rodaniche, Arcadio F. (1984). "Iteroparity in the lesser Pacific striped octopus Octopus chierchiae (Jatta, 1889)". Bulletin of Marine Science. 35 (1): 99–104 – via Ingenta Connect.

10. Voss, Gilbert L. (1968). Octopods from the R/V Pillsbury southern Caribbean Cruise, 1966, with a description of a new species, octopus zonatus. OCLC 964398146.

11. Norman, M. D.; Finn, J. (2001). "Revision of the Octopus horridus species-group, including erection of a new subgenus and description of two member species from the Great Barrier Reef, Australia". Invertebrate Systematics. 15 (1): 13. doi:10.1071/it99018. ISSN 1445-5226.

12. HUFFARD, CHRISTINE L.; SAARMAN, NORAH; HAMILTON, HEALY; SIMISON, W. BRIAN (24 August 2010). "The evolution of conspicuous facultative mimicry in octopuses: an example of secondary adaptation?". Biological Journal of the Linnean Society. 101 (1): 68–77. doi:10.1111/j.1095-8312.2010.01484.x. ISSN 0024-4066.

13. Hanlon, R.T. (1998). "J.B. Messenger Cephalopod behaviour. xvi, 232p. Cambridge University Press, 1996. Price £50.00". Journal of the Marine Biological Association of the United Kingdom. 76 (4): 1118. doi:10.1017/s0025315400041060. ISSN 0025-3154.

14. Mangold, K (1987). "P. R. Boyle, editor. Cephalopod Life Cycles. Volume II. Comparative Reviews. xxi, 441 pp. Academic Press, 1987. Price £78". Journal of the Marine Biological Association of the United Kingdom. 68 (1): 215. doi:10.1017/s0025315400050190. ISSN 0025-3154.

15. Wells, M.J.; Wells, J. (1972). "Sexual displays and mating of Octopus vulgaris Cuvier and O. cyanea Gray and attempts to alter performance by manipulating the glandular condition of the animals". Animal Behaviour. 20 (2): 293–308. doi:10.1016/s0003-3472(72)80051-4. ISSN 0003-3472. PMID.

16. Rodaniche, Arcadio F. (1991). "Notes on the behavior of the Larger Pacific Striped Octopus, an undescribed species of the genus Octopus". Bulletin of Marine Science. 49: 667.