Ten-Cent Inference and the New AI Distillation War

William Palmer

So I was... uh, I was looking at the OpenRouter pricing sheets this morning, and- and- and the numbers just didn't make sense. I actually had to rub my eyes. We're looking at... okay, so poolside just released Laguna S 2.1, right? And the pricing is... it's stupid. It is literally ten cents, zero point one zero dollars, per million input tokens, and twenty cents for output. Now, if you're keeping track of the... of the kind of cost-war in AI, DeepSeek V4 Flash, which has been the absolute poster child for budget API calls, is at nine cents input and eighteen cents output. So Laguna is... it's basically matching the cheapest utility model on earth. But here's the kicker... on the agentic coding benchmarks, it is apparently beating DeepSeek V4 Pro, which costs... what, over four times that? It's forty-three and a half cents input. It's... it's wild.

William Palmer

Now, how do you actually pull that off? I mean... you don't just magically get frontier-level coding performance for ten cents unless there's some serious engineering under the hood. And... and there is. Laguna S 2.1 is structured as a... as a 118-billion total parameter Mixture-of-Experts model. But here's the trick... it only activates 8 billion parameters per token. If you've ever worked on cars, or... or if you follow racing like I do, it's exactly like variable displacement in a high-performance engine. When you're just cruising down the highway, you don't need all eight cylinders chugging fuel, right? The computer shuts down four of them. You conserve energy, you save fuel, but... but the moment you slam your foot down on the pedal, all eight cylinders fire back up instantly and you get that peak speed. That is what Laguna is doing. It's keeping its footprint incredibly lean on a token-by-token basis, which is why people are already running this locally on prosumer hardware... like, we're talking a single RTX 6000 or a high-RAM desktop setup... without needing a massive, screaming datacenter server.

William Palmer

But... okay, is it actually as smart as a giant dense model? Well, maybe we're asking the wrong question. Eiso Kant... who's the co-founder of poolside... he's been super vocal about this. His whole thesis is that we, as an industry, we're... we are heavily, heavily overrating raw, static model intelligence when it comes to software engineering. Like... we think the model needs to be this omniscient brain that gets the code perfect on the very first try. But real software engineering... it doesn't work that way. Humans don't write code like that. We write some code, we compile it, we look at the error, we curse, we backtrack, we write a test, we fix it. And that is what Laguna's... what they call their "max thinking mode" is designed to do. It utilizes this massive one-million-token context window not just to hold a big codebase, but to give the agent room to breathe. It allows the model to... to literally run search and verification loops in its own "mind" before it hands you the final answer. It's persistence over raw, massive parameter size. And... um, honestly, the early testing on local-llama subreddits shows it's incredibly fast at tool mechanics... though, some users did note that if you put it under extreme pressure without the right templates, it can occasionally overthink the math and start inventing facts. But still, for a local model? It is shifting the entire landscape.

William Palmer

This shift to highly capable, local, open-weights models... it's not just a technical win. It is... it's becoming a massive geopolitical flashpoint. Poolside released Laguna under this OpenMDW-1.1 license... very transparent, very accessible. Because they know that in the West, developers are... they're getting increasingly nervous about being locked into closed, heavily censored APIs. But at the same exact time... we are seeing this crazy, almost desperate regulatory pushback. The biggest story this week is... is this public accusation from Michael Kratsios, the U.S. Tech and Science Advisor. He basically claimed, flat out, that the Chinese firm Moonshot AI built their new Kimi K3 model through... quote, "large-scale, covert industrial distillation" of Anthropic's Fable model. And he even pointed to some secret access to GB300 GPU clusters over in Thailand.

William Palmer

Now... researchers look at this and go... wait a minute. The timeline doesn't make any technical sense. Anthropic changed access to Fable on July 1st. Moonshot announced Kimi K3 on July 15th. That is a fifteen-day window. To distill a model of that caliber, train it, and launch it in two weeks? It's... it's virtually impossible. But the fact that the White House is making these statements shows how high the stakes are. We are entering this bizarre legal gray area. Is distillation... using one model's outputs to train another... is it actually "theft"? Under current copyright doctrine, that is a massive, unresolved mess. And... and my worry is that if Western regulators try to clamp down too hard, if they try to ban open-source weights or restrict "distillation" with heavy-handed policies... it's going to backfire. If a developer thinks their cloud API might cut them off, or flag their defensive cybersecurity workflows as "unsafe"... they aren't going to stop building. They're just going to download Laguna S 2.1, run it on their own hardware, and assert their own sovereignty. The open-weights genie... it's- it's already out of the bottle. And you can't regulate away ten-cent inference when the code is already on GitHub.

William Palmer

Anyway... that's the state of the board today. Cheap local inference is getting very, very real, and the geopolitical drama is only getting weirder. I'm gonna go play with this local GGUF build. Catch you later.