Mira Murati’s Open-Weight Giant and the Rise of Machine Shorthand

William Palmer

So- so I was looking at the raw weights of this thing last night, and- and it- it just blew my mind. We are talking about forty-five trillion tokens. That is trillion, with a T. Pretrained from scratch on text, images, audio, video... just an absolute behemoth of a dataset. And the, the, the absolute wild part is who put this together. Thinking Machines Lab. This is the startup launched by Mira Murati, John Schulman, Lilian Weng... basically a dream team of ex-OpenAI leaders who pulled in a massive twelve-billion-dollar valuation. And instead of locking it behind an API, instead of building another walled garden, they just... they just put it out there. Open weights. Apache 2.0 license. Anyone can download it. I mean, it is- it is easily the most significant American open-weight release we have seen so far.

William Palmer

Now, the scale here is tricky. People look at the specs and they see nine hundred and seventy-five billion total parameters. Nearly a trillion parameters! And you think, how on earth do you run that? But- but here is the trick. It is a Mixture-of-Experts architecture. You have- you actually only have about forty-one billion active parameters per token. Think of it like a- like a professional race-car pit crew. If you pull into the pit lane, you do not need all forty people on the crew swarming the car at once. If you just need a quick tire change and a slight wing adjustment, you only want the, the, the six specialized mechanics who actually handle those exact tasks to step over the wall. The rest of the crew stays back. That is exactly what Inkling is doing. It has two hundred and fifty-six routed experts, plus two shared experts acting as anchor points, and it dynamically routes the work. It keeps the execution lightning-fast. In fact, vLLM is already getting up to three hundred and eighty tokens per second on Blackwell hardware. It is just... it is an engineering marvel.

William Palmer

And they did not just ship this massive, almost-one-trillion-parameter beast. They also dropped a preview of Inkling-Small. Now, "small" in this context is still two hundred and seventy-six billion total parameters, with twelve billion active. But community testers are already finding that this smaller model is- is unexpectedly competitive on a bunch of evaluations. And because TML lined up day-zero support with vLLM, SGLang, Modal, and Hugging Face, developers can run these models locally, on their own metal, without paying a single cent to a closed-API toll booth. It completely changes the sovereignty equation for developers. You are not- you are not renting intelligence anymore. You own it.

William Palmer

But- but what really gets me, what is actually kind of spooky when you dig into the technical notes, is how this model thinks. During its training, TML ran about thirty million reinforcement learning rollouts to teach it how to reason. And when they looked at the internal chain of thought—the private "scratchpad" where the model processes its logic before giving a final answer—they noticed something bizarre. The model was- it was literally inventing its own language. It started stripping away grammatical overhead. It stopped using prepositions, dropped standard syntax, and began compressing its reasoning into this incredibly dense, machine-native shorthand.

William Palmer

Think about it like a- like a brilliant mathematician working at a chalkboard. When they are trying to solve a really complex theorem, they are not writing out polite, grammatically correct full sentences. They are scribbling messy, ungrammatical equations, arrows, shorthand symbols... things only they understand, just to free up mental bandwidth. That is what Inkling did. By dropping the baggage of human language in its internal thoughts, it- it actually learned to think faster, using fewer tokens to get to the answer. It is incredibly efficient. But... it means the model is thinking in a way we can barely read.

William Palmer

And speaking of machine control, they showed off this mind-bending demo on their Tinker platform. They set up Inkling in a self-fine-tuning loop. They gave it a high-level goal: modify your own weights to write a lipogram that completely avoids the letter "e". The model literally wrote its own training script, spun up a local training run, evaluated its own performance, and- and then successfully upgraded its own weights. It is- it is AI building AI, completely autonomously.

William Palmer

But here is the tension. If we are entering an era where open-weight models can recursively upgrade themselves, and their internal reasoning is written in a self-generated, non-human language... how do we actually monitor them? How do we verify what they are thinking before they output the final response? We are handing developers the keys to the fastest cars on the track, but we might not be able to read the dashboard instruments anymore. It is a wild new frontier, and honestly... we are just getting started. I am going to go spin up the small weights on my local rig and see what else this thing can do. Talk soon.