Main Theme
In this keynote titled “The Economics of the Multi-Model Era,” Philip Kiley, Head of Development at Base Ten, makes a compelling case that AI cost overruns are not a budgeting failure — they are a structural problem rooted in exponential demand growth, longer model reasoning chains, and the rise of agentic workloads. The central message: companies need to embrace a multi-model strategy — combining open and closed models — and take ownership of their inference infrastructure and post-training pipelines to regain control over cost, latency, and quality.
Key terms used throughout: open models, inference engineering, post-training, multi-model agents, efficient frontier, cost per successful task, reinforcement learning (RL), supervised fine-tuning (SFT), and agentic workloads.
Subthemes and Chapters
1. Why Everyone Is Over Budget
Kiley opens with a candid audience poll: almost no one is under budget on AI spend. His explanation is clear — it is not the audience’s fault. Human beings are naturally poor at reasoning about exponential growth, while budgets are reset incrementally each quarter. Three forces are driving AI costs upward at an exponential rate:
- Better models — more capable models attract more workloads and command higher prices.
- Longer thinking — reasoning models spend more output tokens working through a problem before responding. Output tokens are the most expensive token type.
- More agents — a single user action in an agentic workflow can trigger hundreds of inference requests, dozens of tool calls, and millions of tokens, making costs both compound and invisible to the end user.
2. The Open Model Adoption Journey
Kiley maps out a journey he has observed repeatedly over five years in the industry:
- Companies start on a closed frontier model to prove value and secure funding.
- At scale, they hit pain points around cost, latency, reliability, and governance.
- They begin to bring AI in-house, owning the model and infrastructure layers.
- They eventually reach the frontier: running post-training, building custom models, and creating moats around proprietary data.
This journey applies to AI-native products, AI-enhanced existing products, regulated industries (healthcare, financial services), and internal productivity and coding workflows.
3. Choosing Models: Quality, Cost, and Reliability
Model selection should never be based on generalized benchmarks alone. Kiley introduces the concept of cost per successful task as the right unit of measurement — two models may need very different numbers of tokens to answer the same question, making raw token pricing misleading.
- Quality is task-specific. As an example, Kimi K3 was the top-ranked model globally for front-end development but was not the top model for agentic workflows — meaning the right choice depends entirely on the task.
- Reliability is managed through gateways and routers that direct traffic based on size, modality, specialization, geography, and cost.
The industry is converging on this view. At OpenAI Dev Day (September 29, 2026), Sam Altman introduced the OpenAI Marketplace — which Base Ten is a part of — allowing enterprise teams to use open models like GLM 5.3 Flash and Kimi K3 natively within Codex and the Responses API. The multi-model future is no longer a niche position; it is an industry direction.
4. Inference Engineering to Reduce Cost
Once a company owns its inference infrastructure through dedicated deployments, it gains the ability to make deliberate trade-offs along the efficient frontier — trading off things it does not care about (such as some latency) for things it does (such as lower cost). Key techniques include:
- Expert parallelism — increases throughput for high-volume, latency-tolerant agentic workloads.
- Prefill/decode disaggregation — separating these two compute phases onto distinct workloads, which Base Ten reports can double token-per-minute throughput on the same hardware.
- Domain-specific quantization calibration — preserves quality on the dimensions that matter while significantly reducing compute cost.
- Caching — moving tokens from expensive output tokens to the cheapest type, cached input tokens, by leveraging high cache hit rates in looping agentic workflows.
5. Post-Training: Moving Beyond the Frontier
While inference engineering helps move along the efficient frontier, post-training moves the frontier itself — achieving higher quality, lower cost, and better speed simultaneously.
Two primary methods:
- Supervised Fine-Tuning (SFT) — a continuation of pre-training, effective for shaping model behavior.
- Reinforcement Learning (RL) — the model is run in its actual production environment, evaluated on real outcomes, and updated accordingly. Increasingly common for agentic workflows.
Case Study — Code Rabbit & Nvidia NemoTron 3.5 Lightning:
Code Rabbit, an AI code review company, needed a model to score pull requests (PRs) for complexity — routing simple PRs to lighter models and complex infrastructure changes to more powerful ones. Base Ten worked with them using a combination of SFT and RL to post-train NemoTron 3.5 Lightning for this routing decision task. The result: more than 50% cost reduction per review compared to their baseline GPT model, while also improving task-specific performance.
Kiley acknowledges a common objection — what happens when a new base model is released in three to six months? His answer: the ROI calculation typically justifies the investment even over a short window. More importantly, the most expensive part of post-training is not the compute — it is building the RL environment and datasets, both of which are fully reusable as base models evolve.
Inspiring Quotes
“It’s not your fault if you’re over budget for the year. It’s really hard for humans to reason about exponential growth.”
“Output tokens, which are used for reasoning, are the most expensive kind of tokens — and we’re scaling very, very hard on that axis.”
“One of the persistent myths in AI is that you can’t use it in regulated environments.”
“You don’t want to think about cost in terms of tokens per million anymore. You want to think in terms of cost per successful task.”
“If inference lets us move along an efficient frontier, post-training lets us move out of the frontier entirely.”
Key Action Points
- Shift your cost metric from token price to cost per successful task. Evaluate models on task-specific benchmarks relevant to your actual workloads, and measure what it truly costs to complete a task end-to-end — not just the per-token rate.
- Audit your agentic workloads for hidden token spend. Map out how many inference requests, tool calls, and tokens a single user action generates. Identify where caching, routing, and model tiering can reduce cost without sacrificing quality.
- Build a reusable post-training environment now. Even if base models change every few months, the RL environment, datasets, and research infrastructure you build today are reusable assets. The ROI on post-training compounds over time — start with one high-volume, well-defined task.
Philip Kiley’s book Inference Engineering, published in February 2026, covers these topics in greater depth and is available as a free PDF, EPUB, and audiobook download.