AI Explained: Can These New Tools Be Fooled?
In brief
- Researchers have discovered that a new method for explaining how AI models work can be easily tricked.
- This method, called NLAs, helps users understand what's going on inside complex AI systems.
- However, experiments show that by tweaking certain parts of the model, it's possible to make these explanations misleading or even contradictory.
- For example, one test made the tool say the opposite of what the model was actually doing, while another hid specific words from being explained.
- This raises important questions about how much we can trust AI explanations and whether they can be manipulated.
- Moving forward, experts will need to figure out ways to make these tools more reliable so that users can trust them for real-world decisions.
Terms in this brief
- NLAs
- Short for 'Natural Language Adapters,' these tools help users understand how AI models work by translating complex computations into human-readable explanations. However, researchers found that NLAs can be manipulated to provide misleading or contradictory information, raising concerns about their reliability in real-world applications.
Read full story at LessWrong →
More briefs
Data Access Hinders AI Progress
Companies are finding that data access is a bigger obstacle to artificial intelligence progress than model sophistication. This was a major theme at Pure Accelerate 2026. Governance and data strategy are now prerequisites for successful AI outcomes. Companies that treat governance as a foundation are better at managing data access and security. About 60 percent of AI projects fail due to poor governance. AI projects will likely improve as companies focus on data access and governance.
Ancient Scrolls Uncovered
Vesuvius buried 2,000-year-old papyrus scrolls from a library. Now researchers can read them using particle accelerators and artificial intelligence. The scrolls have been impossible to read due to their fragile state. Researchers have identified one scroll as potentially among the oldest in Roman history. Another could provide new insights into Greek theology. This matters because it gives us new information about ancient history. We can learn more about the people who lived 2,000 years ago. Researchers will continue to study the scrolls to learn more about the past.
Medical AI Models Pose Privacy Risks
A new study found that medical artificial intelligence models can expose sensitive patient information through privacy attacks. These attacks can achieve near-perfect success rates for individual patients, even when the overall performance is low. For example, models with high capacity can increase the number of patients with high attack success rates. Underrepresented groups face disproportionately high attack success rates, which can lead to severe consequences. Researchers will continue to develop risk assessment and mitigation techniques to protect patient data.
Ancient Scroll Unwrapped with AI
Researchers used artificial intelligence to virtually unwrap a 2000 year old papyrus scroll that was burnt during the Mount Vesuvius eruption. The scroll was recovered from a Roman villa in Herculaneum and is one of the oldest in a collection of hundreds. It discusses stoic philosophy on ethics, art and human behaviour, covering more than a metre of charred papyrus. The achievement is part of a global contest to read carbonised scrolls, with hundreds of thousands of dollars in prizes awarded to teams using artificial intelligence to unwrap and read the text. The team will continue to analyze the text to learn more about ancient philosophy.
Redefining AI Agency: A New Framework for Understanding Autonomy
Researchers have introduced a detailed framework to distinguish between "agentic" and "agentive" AI systems, clarifying the boundaries of autonomy in artificial intelligence. This distinction is crucial as AI tools like coding agents and AI co-scientists promise increased productivity but also raise concerns about potential escapes from human control. The study identifies five key dimensions-goal, identity, decision-making, self-regulation, and learning-that define agency. It argues that true agency requires these traits to be internalized within the system rather than relying on external guidance. The proposed Goal-Identity-Configurator (GIC) architecture combines hierarchical goal decomposition, identity evolution, simulative reasoning, learned self-regulation, and self-directed learning from real and simulated experiences. This approach aims to create general-purpose agents capable of operating in open environments with true autonomy. The research emphasizes the importance of auditability, controllability, and safety for these systems, ensuring they remain under human oversight. Moving forward, this framework could shape how developers design AI systems, balancing productivity gains with ethical considerations. As AI becomes more autonomous, understanding its boundaries will be essential for both innovation and risk management.