Baking Safety Into Open AI Models

The problem with open models

Here is an awkward truth about open-weight AI models. Once you release a model's inner workings to the world, anyone can download it, poke around, and quietly remove the safety features that stop it from doing harmful things.

A quick vocabulary note. "Open-weight" means the model's trained parameters, the numbers that make it tick, are published for anyone to use. That openness is great for research and tinkering. It also means bad actors can strip out the safeguards using a technique called "abliteration," which disables a model's ability to refuse dangerous requests.

This is not hypothetical. Hugging Face, the popular hub that hosts open-source AI models, currently lists more than 6,000 abliterated models. That is a lot of guardrails already ripped off.

What Base Labs is doing

On Wednesday, the AI infrastructure company Baseten launched a new safety effort through Base Labs, a research arm it spun up earlier this year. It is teaming with Hugging Face and a startup called Goodfire AI to build tools for evaluating and monitoring open models.

The pitch is a shift in philosophy. Instead of adding safety measures on top of a finished model, where they can be peeled away, the group wants safety woven into how models are trained and served in the first place. Baseten is calling this a "standard" for open models, one meant to be transparent by design.

"We believe openness to be an advantage for AI safety," the company said on X. Its argument is that open models give researchers more visibility into how a model actually behaves, and more ways to turn that understanding into real, transparent controls than a closed system would.

Goodfire is the interesting piece here. The company specializes in interpretability, the work of prying open AI's "black box" to explain why a model makes the decisions it does. "Safety must be built into open models and provided by those who serve them," Goodfire wrote in reply. If anyone is going to handle the "built into" part, it is likely them.

Why it matters

The players involved are not small. Baseten, which provides the computing muscle that runs AI models, raised a $1.5 billion Series F in June that pushed its valuation to $13 billion. Goodfire is well-funded too, with a $150 million Series B led by B Capital earlier this year to advance its interpretability platform.

That funding matters because credible safety infrastructure is expensive and unglamorous. It is easier to raise money for flashy new models than for the plumbing that keeps them from misbehaving. Having deep-pocketed companies attach their names to the problem gives it more staying power.

A dose of honesty, though. The companies have not disclosed how the partnership will actually work on a technical level. Right now this is a framing and a set of intentions more than a shipped product. The 6,000 abliterated models already out there will not un-abliterate themselves, and a new standard does not retroactively fix anything.

What's next

Baseten is putting out an open call for the wider developer community to help build the framework. "Together, we are building an ecosystem of open models that are safe and accessible to all," the company said. Whether that turns into concrete tools or stays a mission statement is the thing to watch.

The bigger question hanging over all of this is whether safety can meaningfully be built into a model that, by definition, anyone is free to modify. Openness cuts both ways. It gives researchers more eyes on the problem, and it gives everyone else the keys. If Base Labs and its partners can show that transparency genuinely makes open models safer rather than just easier to audit, that would be worth paying attention to.