AI Is Ready for Its Next Phase: Inference

AI Is Ready for Its Next Phase: Inference

AI's first phase was defined by frontier-model progress: larger training runs, better benchmarks, longer context windows, increasingly capable reasoning. That phase is not ending. Frontier labs will remain essential because they keep expanding what machines can do. But raw model capability is now abundant, and the premium on owning the single best model is declining. The cost of accessing GPT-3.5-level intelligence fell from about $20 per million tokens in November 2022 to $0.07 by October 2024. That's a reduction of more than 280 times. Leading open-weight models have moved close enough to proprietary models that they're already economically sufficient for many real workloads.

This changes the central product question. Companies will increasingly stop asking, "Which model is smartest?" and instead ask, "What is the cheapest model that reliably completes this step?" A classification task, document extraction workflow, support query, or coding suggestion may not need the most powerful frontier model. A production system may use a small specialized model for repetitive work, an open model for high-volume or private workloads, and a frontier model only when difficult reasoning justifies the higher price. The market will therefore organize around intelligence per dollar rather than benchmark leadership alone.

Kimi K3 captures both the promise and the limitation of this transition. Its emergence shows open-weight models can approach the frontier, weakening the assumption that advanced intelligence must come from two or three proprietary labs. But it also shows why "open" is not synonymous with "cheap." A model can have a low advertised token price and still be expensive per task if it generates too many tokens, requires more retries, uses tools poorly, or needs greater verification. The winning metric is not cost per token. It is cost per successfully completed, verified outcome: a correctly processed filing, a resolved customer-support case, or a code change accepted without repair.

That is why the next important layer will be the inference control plane. Its job will not be merely to rent GPUs or resell model APIs. It will decide which model handles each task, how context is assembled, and when retrieval or fine-tuning is necessary. It will also decide how requests are batched and cached, when outputs must be verified, and when work should be escalated to a stronger model or a human. The strongest platforms will route across proprietary, open, fine-tuned, and local models while optimizing for latency, privacy, reliability, and cost. This looks less like a cloud reseller and more like an operating system for intelligence.

For startups and enterprises, the moat will increasingly sit in the harness around the model. A general model does not understand a company's internal ontology, approval hierarchy, contracts, operating procedures, regulatory constraints, or error tolerance. That competence has to be built through proprietary context, tools, memory, permissions, workflow decomposition, evaluations, and human escalation. The model supplies general intelligence. The harness turns it into organizational capability. Companies that own a workflow deeply, and continually learn from real production failures, will be harder to replace than those whose only advantage is early access to a particular model.

The winners of the inference phase will therefore be broader than frontier labs alone. Frontier companies will continue to push the capability boundary. But inference clouds, model-routing platforms, local-AI systems, vertically integrated technology companies, and domain-specific applications will industrialize those capabilities and make them economical. Falling inference prices will not necessarily reduce the size of the market. They can unlock longer contexts, continuous agents, machine-to-machine workflows, and applications that were previously too costly to run.

History suggests that making a foundational layer cheaper does not shrink the technology market. It expands what can be built above it. Linux commoditized the operating system and helped enable cloud computing. Amazon transformed infrastructure it had developed for its retail operation into AWS. AWS generated approximately $129 billion in revenue and $45.6 billion in operating income in 2025. AI may follow the same pattern. Frontier labs will continue pushing the capability boundary. A much larger ecosystem will capture value by serving, routing, and embedding those capabilities into real work. The next defining AI companies may not train the largest model. They may be the ones that turn increasingly abundant intelligence into reliable, affordable, and ubiquitous products.