Google Unveils Advanced AI Control Framework to Ensure Safer AI Development
In brief
- Google has introduced a new AI Control Roadmap designed to manage the risks posed by increasingly powerful AI agents.
- This framework focuses on system-level security, treating AI as potential insider threats and using industry-standard threat modeling based on the MITRE ATT&CK framework.
- By addressing alignment issues and incorporating cybersecurity best practices, Google aims to create a robust defense mechanism that ensures AI remains aligned with human goals even when misaligned.
- The approach involves monitoring AI agents through trusted supervisors and implementing safeguards like sandboxing and prompt injection resistance.
- It also includes a novel threat-modelling framework tailored for AI risks, breaking them down into manageable tactics and techniques.
- This method allows Google to detect and prevent potential issues before they cause harm, while still enabling controlled access for incremental trust-building.
- Looking ahead, this defense-in-depth strategy could serve as a model for the broader AI industry, potentially reducing risks and fostering safer development practices.
- As AI capabilities continue to grow, such frameworks will be crucial in maintaining control and ensuring that AI systems operate responsibly within human-defined boundaries.
Terms in this brief
- MITRE ATT&CK framework
- A cybersecurity framework that maps out potential threats and tactics to help organizations defend against attacks. Google uses it to model AI risks, treating AI systems like potential insider threats and identifying ways to counteract them.
- sandboxing
- A security technique where a program runs in a controlled environment, isolated from the rest of the system. This helps prevent harmful actions by restricting what an AI can access or do, even if it tries to act maliciously.
Read full story at DeepMind Safety →
More briefs
AI Verification System Proposed for International Governance
A new draft proposal for a privacy-preserving AI verification system has been released, aimed at fostering trust between rival nations in the realm of artificial intelligence governance. The system is designed to ensure that AI operations can be monitored without compromising user data or intellectual property. It emphasizes retrofitting existing hardware and maintaining physical security through off-the-shelf components. The proposed framework outlines three key features: preserving privacy by preventing unauthorized data disclosure, ensuring compatibility with current technology, and mandating robust physical safeguards. This approach seeks to address the challenges of verifying AI systems while respecting confidentiality and avoiding the need for specialized infrastructure. This initiative aims to kickstart international collaboration on AI governance by inviting feedback from experts in policy, cybersecurity, and engineering. Future developments will focus on refining the architecture through community input and red-teaming exercises.
AI Safety Takes a Step Closer With Hardware-Based Trust
AI systems are increasingly being used in critical areas, but ensuring they behave as intended is a major challenge. Now, researchers are exploring Trusted Execution Environments (TEEs), which could revolutionize how we verify and control AI deployments. These environments use specialized hardware to isolate and secure AI operations, making it harder for malicious actors to interfere. The key innovation here is that TEEs replace trust in humans with verifiable constraints built into the hardware. This means sensitive AI tasks can be monitored without sacrificing user privacy. However, there are significant hurdles. Current TEE technology, like Intel's SGX, isn't foolproof-vulnerabilities exist, and trust in hardware vendors remains an issue. Looking ahead, the biggest challenge is ensuring hardware transparency and auditability. If researchers can develop trustworthy hardware that's independently verifiable, TEEs could become a cornerstone of AI governance. For now, while progress is being made, the practical implementation of these systems in real-world scenarios will be crucial to their success.
AI Pioneer Yann LeCun Challenges Current AI Models
Yann LeCun, a Turing Award winner, says current AI models are limited. He thinks they need a new approach to reach human-level intelligence. He is building a startup called Advanced Machine Intelligence with $1.03 billion in funding. The company wants to create "world models" that learn from reality and predict what happens next. LeCun's new approach could lead to more intelligent systems that can plan and reason like humans.
AI Model Accused of Being a Merge
A company claims a new AI model is not original. It says the model is a mix of its own model and another one. The model is said to be 60 percent from one source and 40 percent from another. This matters because it affects how people trust AI. The company found this out by testing the model and looking at its code. The company will likely take further action to address this issue now.
AI Safety Research Reveals Surprising Insights into Gemini’s Behavior
Google's DeepMind team has uncovered unexpected findings about how AI models like Gemini are shaped. Their research shows that most of Gemini's safety features come from its pre-training and fine-tuning phases, not other training methods like reinforcement learning. This is a big shift from what they initially thought. The study found that when they removed the fine-tuning process (SFT) from Gemini, the model’s behavior didn’t change much on safety tests. This suggests that pre-training plays a crucial role in determining how safe and reliable AI systems are. However, the team also discovered that certain unwanted behaviors can still pop up even after filtering out bad examples during training. Looking ahead, DeepMind plans to focus more on improving the fine-tuning process to enhance model safety. They’re also working on better ways to identify and prevent behaviors that slip through the cracks despite these filters. This research could help make AI systems more predictable and trustworthy in the future.