The fittest LLM is a moving taget

A major LLM provider recently warned that prices are going up. They didn’t say by how much, or when. If your product depends heavily on that model, you now have a cost you can’t forecast.

While pricing is the immediate problem, I think it points to a much broader one. We’re building products on top of an ecosystem that is still changing incredibly quickly, while often making architectural decisions as though the pieces underneath us are relatively stable. They aren’t.

This is why I’ve been thinking a lot about model portability.

0x00Supporting multiple models isn’t the same as being portable

Vendor harnesses are great. They’re turnkey, batteries included, and most can even work with models from different providers. That solves a lot of problems, but there’s an important difference between supporting another provider and being able to switch providers without changing how your product works.

Prices change. Models get deprecated. Terms change. Sometimes the change is obvious: the model you use disappears or becomes too expensive. Other times, it’s much harder to detect.

A provider upgrades a model and suddenly your product feels worse. Nothing crashes. There are no API errors. Your tests might still pass. The model simply behaves differently at the particular task your product depends on.

I’ve seen this happen. Engineers ended up writing emergency prompt instructions just to recover behavior they’d previously been getting out of the box. That experience stuck with me because it highlights something unusual about building on LLMs: with traditional infrastructure, migrations tend to fail loudly. Something breaks and you fix it. Model migrations can fail quietly. The application keeps running; it just gets worse.

0x01Portability isn’t only defensive

The obvious argument for portability is risk. You don’t want a pricing change, deprecation, or policy decision somewhere else to turn into an emergency for your team.

But in my opinion, the more interesting part is being able to ride the frontier as better and more efficient models appear. Maybe you want a frontier model doing the difficult planning, while a swarm of faster, cheaper models reviews the work and escalates interesting problems back to it. Expensive reasoning where it matters, cheap inference where volume matters.

Maybe that architecture works well today. Next month, someone might release a smaller model that is unusually good at one part of your workflow, and suddenly the optimal architecture looks completely different.

Nobody really knows yet. The space is moving too quickly, and there are too many combinations for anyone to run all the experiments. That uncertainty changes how I think about architecture. If we don’t know what the best model architecture will look like a year from now—or even six months from now—then optimizing exclusively for what works best today seems like the wrong goal.

The ability to experiment becomes part of the architecture itself.

0x02Switching should be boring

Ideally, trying a different model should feel closer to changing configuration than starting an integration project. That doesn’t mean models are interchangeable. They aren’t.

Different models respond differently to inputs. They have different tool-calling behavior, different failure modes, different latency profiles, and sometimes surprisingly different personalities. Switching models will always require experimentation and evaluation. The goal isn’t to pretend those differences don’t exist; it’s to make encountering them routine.

I’d rather discover in staging that a new model has changed the tone of my product than discover it after users start complaining. I’d rather have evaluations expose a regression than have engineers debugging something nobody initially recognizes as a regression. And I’d rather be able to test a promising new model this afternoon than put an integration project on the roadmap for next quarter.

0x03The question I care about

There’s a lot of attention right now on which model is best. It’s a reasonable question, but I suspect it’s the wrong one to optimize an architecture around because the answer is going to keep changing.

The question I find more useful is:

How expensive is it for me to ask which model is best again?

If the answer is “a few experiments and a configuration change,” you can keep adapting as the ecosystem changes. If the answer is “a migration project,” you’ve made a much bigger bet than simply choosing a model.

I’m building tooling around the assumption that models, providers, and architectures will keep changing—and that switching between them should be a normal part of building LLM products, not an emergency.

More on the other pieces soon.