The State of AI

An Empirical 100 Trillion Token Study with OpenRouter

The past year in AI has been defined by rapid shifts in capability, adoption, and developer behavior. Breakthrough reasoning models, the acceleration of open-source innovation, and a surge in AI-native applications have reshaped how we build and interact with intelligent systems. Today, we are releasing a new empirical study that examines these changes through a particularly comprehensive lens: more than 100 trillion tokens of real-world LLM usage from OpenRouter.

OpenRouter now serves over 5+ million developers and routes traffic across 300+ models from more than 60 providers. The platform has grown from handling roughly 10 trillion tokens per year to more than 100 trillion as of mid-2025. Just last week, OpenRouter processed more than 1 trillion tokens every single day. For perspective, OpenAI’s entire API averaged about 8.6 trillion tokens per day in October. This scale gives OpenRouter a particularly comprehensive view into how developers use AI across industries, geographies, and model families.

Our goal with this study is simple: provide a large scale empirical picture of what people do with AI today, and what that tells us about the next chapter of the industry.

The Shift Toward Reasoning and Agency

Just a year ago, even the best language models were still single-pass, autoregressive predictors. They could follow instructions well and, with techniques like chain-of-thought, RAG, and tool use, handle more complex tasks. Anthropic’s Claude 3.5 Sonnet pushed tool integration and coding forward, Cohere’s Command R+ specialized in multi-step agentic workflows, and open-source projects like Reflection 70B experimented with supervised reasoning traces. Yet the underlying mechanism remained unchanged: a single forward pass that produced fluent text but lacked genuine iterative computation.

That changed on December 5, 2024, with OpenAI’s full release of o1, codenamed Strawberry, the first broadly deployed reasoning model to integrate structured, multi-stage inference directly into the architecture. Instead of generating answers in one shot, o1 ran internal deliberation loops, explored alternative paths, and refined its output before responding. In retrospect, it was the real inflection point: earlier systems extended what autoregressive predictors could express, but o1 was the first to introduce a model that could compute through deliberate, multi-step reasoning.

Our data shows this shift playing out in practice. The fastest-growing behavior on OpenRouter is what we call agentic inference. Developers are increasingly building workflows where the model acts in extended sequences rather than single prompts. A typical interaction no longer stops at a response. Instead, the model plans, retrieves context from tools or APIs, revises outputs, and iterates until the task is complete. Prompt lengths are increasing. Sessions have more turns. Specialized reasoning and tool-use models are gaining share.

The implication is profound. AI is moving from a static chat interface to an active participant in work. The competitive frontier is no longer only about accuracy or benchmarks. It is about orchestration, control, and a model’s ability to operate as a reliable agent. For founders, this represents a strategic opening. Products that embrace these workflows early will define the next generation of AI-native applications.

The Evolving Model Landscape

Because OpenRouter sits at the intersection of many models and real user workloads, its data reveals a broader picture of competition than benchmark leaderboards alone.

Several patterns stand out:

• Open-source models are rising quickly, particularly reasoning-forward models like DeepSeek R1 and Kimi K2, which are capturing share due to cost efficiency and flexibility.
• Creative and coding use cases remain the largest drivers of token volume, reinforcing the centrality of AI in software development and content generation.
• Retention patterns are increasingly influenced by breakthrough moments. When a new capability meaningfully changes what users can do, they switch models and do not switch back.

These insights are difficult to see from traditional benchmarks. They emerge only when examining large-scale, real-world interactions across models and workloads.

Why This Matters

The shift toward agentic behavior and multi-step workflows is not a theoretical prediction. It is already visible in production traffic. As the industry matures, the winners will be those who capitalize on this shift by building for reasoning, tool use, persistence, and long-horizon tasks.

For researchers, the dataset surfaces fresh questions. Why is roleplay so dominant across models? What patterns in tool-augmented usage hint at future architectures? What retention curves can predict the next breakout capability? Real-world data at this scale has been missing from the field. This study begins to shade some lights on these questions.

Read the Full Study

The State of AI: An Empirical 100 Trillion Token Study with OpenRouter provides a deeper analysis of these trends, including open vs closed source model usages, geographic patterns, category taxonomies, and cohort-based insights into long-term engagement.

You can access the full report here on OpenRouter. It is a comprehensive resource for anyone building, researching, or investing in AI. Understanding how 100 trillion tokens are used in the real world offers a data-driven guide to the future of the ecosystem.

This is just the beginning of a richer conversation grounded in real world usage. We invite you to dive in.

---

This newsletter is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. Furthermore, this content is not investment advice, nor is it intended for use by any investors or prospective investors in any a16z funds. This newsletter may link to other websites or contain other information obtained from third-party sources - a16z has not independently verified nor makes any representations about the current or enduring accuracy of such information. If this content includes third-party advertisements, a16z has not reviewed such advertisements and does not endorse any advertising content or related companies contained therein. Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z; visit https://a16z.com/investment-list/ for a full list of investments. Other important information can be found at a16z.com/disclosures. You’re receiving this newsletter since you opted in earlier; if you would like to opt out of future newsletters you may unsubscribe immediately.