Meta’s New AI Framework: Balancing Openness with Risk Mitigation

A Pledge to Make AGI Openly Available

Meta CEO Mark Zuckerberg has pledged to make artificial general intelligence (AGI), which can accomplish any task a human can, openly available one day. However, in a new policy document, Meta suggests that there are certain scenarios in which it may not release a highly capable AI system it developed internally.

Identifying High-Risk and Critical-Risk Systems

The document, which Meta is calling its Frontier AI Framework, identifies two types of AI systems the company considers too risky to release: “high risk” and “critical risk” systems. Both types of systems are capable of aiding in cybersecurity, chemical, and biological attacks, with the difference being that “critical-risk” systems could result in a “catastrophic outcome [that] cannot be mitigated in [a] proposed deployment context.”

Defining High-Risk and Critical-Risk Systems

As Meta defines them, both “high-risk” and “critical-risk” systems are capable of aiding in cybersecurity, chemical, and biological attacks. High-risk systems might make an attack easier to carry out but not as reliably or dependably as a critical risk system. The company provides examples of the types of attacks that are considered high-risk, including the “automated end-to-end compromise of a best-practice-protected corporate-scale environment” and the “proliferation of high-impact biological weapons.”

Evaluating System Risk

Somewhat surprising is that, according to the document, Meta classifies system risk not based on any one empirical test but informed by the input of internal and external researchers who are subject to review by “senior-level decision-makers.” Meta says that it doesn’t believe the science of evaluation is “sufficiently robust as to provide definitive quantitative metrics” for deciding a system’s riskiness.

Limiting Access and Implementing Mitigations

If Meta determines a system is high-risk, the company says it will limit access to the system internally and won’t release it until it implements mitigations to “reduce risk to moderate levels.” If, on the other hand, a system is deemed critical-risk, Meta says it will implement unspecified security protections to prevent the system from being exfiltrated and stop development until the system can be made less dangerous.

A Response to Criticism

Meta’s Frontier AI Framework, which the company says will evolve with the changing AI landscape, and which Meta earlier committed to publishing ahead of the France AI Action Summit this month, appears to be a response to criticism of the company’s “open” approach to system development. Meta has embraced a strategy of making its AI technology openly available — albeit not open source by the commonly understood definition — in contrast to companies like OpenAI that opt to gate their systems behind an API.

The Benefits and Risks of Open AI

For Meta, the open release approach has proven to be a blessing and a curse. The company’s family of AI models, called Llama, has racked up hundreds of millions of downloads. But Llama has also reportedly been used by at least one U.S. adversary to develop a defense chatbot.

A Contrasting Approach

In publishing its Frontier AI Framework, Meta may also be aiming to contrast its open AI strategy with Chinese AI firm DeepSeek’s. DeepSeek also makes its systems openly available. But the company’s AI has few safeguards and can be easily steered to generate toxic and harmful outputs.

A Balanced Approach

“[W]e believe that by considering both benefits and risks in making decisions about how to develop and deploy advanced AI,” Meta writes in the document, “it is possible to deliver that technology to society in a way that preserves the benefits of that technology to society while also maintaining an appropriate level of risk.”

Stay Up-to-Date with TechCrunch’s AI-Focused Newsletter

TechCrunch has an AI-focused newsletter! Sign up here to get it in your inbox every Wednesday.