What SubQ tells us about the limits of the transformer

The architecture beneath an AI agent is not an implementation detail. It is the substrate of the mandate — the thing that determines what the agent can actually do, at what scale, and at what cost. SubQ, the sub-quadratic language model from Subquadratic launched on 6 May 2025, is the clearest signal yet that the transformer is no longer the only possible foundation.

Quadratic complexity as structural constraint

Transformer models scale with quadratic complexity relative to context length. Doubling the context quadruples the compute cost. In practice, this imposes a ceiling: Claude and Gemini operate at up to one million tokens. That is not a choice — it is a consequence of the architecture.

SubQ uses a sub-quadratic sparse-attention architecture (SSA) that breaks this relationship. The model claims 52x faster processing than FlashAttention at one million tokens and supports a context window of 12 million tokens. The company reports 150 tokens per second inference speed and 97 percent of transformer performance using 30 percent of the compute resources.

Numbers like these require caution. As of May 2026, no peer-reviewed papers on SubQ exist on arXiv. The claims rest on company announcements and YouTube videos without independent verification. That is insufficient for scientific conclusion — but it is sufficient to ask a structural question: what changes about agent design if the context window is no longer a scarce resource?

What a 12-million-token context window actually enables

Context length is not an abstract performance metric. It determines which tasks an agent can complete in a single operation, without chunking, summarising, or losing information along the way.

At 12 million tokens, an agent can read an entire codebase, a multi-year contract, or a continuous data stream from a production system — and reason across all of it simultaneously. SubQ is positioned explicitly for persistent conversations, real-time video analysis, and massive document processing.

Context is not memory. It is operational radius.

For industrial agents — the kind SkyeTec builds for European industry — this is consequential. An agent monitoring a power grid, processing sensor data from a plant, or maintaining a long decision history is not well served by an architectural ceiling that forces artificial truncation of the information flow.

The post-transformer landscape is wider than one model

SubQ is not alone. At NeurIPS 2024, work was presented distilling transformer performance into state space models (SSMs) such as Phi-Mamba — using only three billion tokens, less than one percent of typical training data. SSMs scale linearly with sequence length, not quadratically.

These two approaches — sparse attention and state space models — represent different solutions to the same problem: the quadratic cost of the transformer. They are not necessarily competitors. They may complement each other, or converge in hybrid architectures.

Anthropic and Google are reportedly investigating SubQ integration, with commercial deployment predicted within 18 months of the May 2025 announcement. These are early signals, not guarantees. But the fact that two of the three dominant laboratories are considering building on an external sub-quadratic architecture is structurally significant regardless of whether the timelines hold.

What this changes for agent design

The practical implication is not that one should switch architectures today. SubQ lacks independent verification. SSMs are promising but not mature for all use cases. The transformer is not obsolete.

The implication is that the architectural choice is now open in a way it was not three years ago. Anyone designing agents for long-horizon industrial operation — with requirements for persistence, cost control, and information integrity over extended time periods — must engage with that choice explicitly.

If SubQ's claims are verified, the cost calculus for long-context agents changes fundamentally. If SSM distillation matures, training costs change. In either case, the conclusion is the same: architecture is no longer a given premise. It is a design decision with operational consequences.