AI Models Fail Simple Health Tests
In brief
- New research found that large language models failed simple stress tests in health applications.
- These models are used in medical research and can make mistakes with slight changes to prompts.
- The models got confused by small changes and fabricated flawed reasoning.
- They also varied widely in what they measured.
- For example, popular health benchmarks differed in reasoning and visual complexity.
- The study revealed gaps between benchmark performance and the robustness needed for multimodal medical reasoning.
- New tests will help improve the models.
Terms in this brief
- multimodal medical reasoning
- The ability of AI models to understand and reason about multiple types of data, such as text, images, and other health-related information, in a way that mimics human-like thinking. This is crucial for making accurate decisions in healthcare settings.
Read full story at Nature →
More briefs
Chilean News Site Uses AI with Transparency
Chilean news site BioBioChile has incorporated artificial intelligence into its work. The outlet clearly discloses when and how the technology was used. This approach is supported by a recent academic study in Chile. The study found that audiences want transparency about AI use and human oversight of the technology. A total of 2,145 participants in Chile preferred news organizations that commit to full human oversight of generative AI. The study's findings are important for newsrooms that use AI. News outlets that are transparent about AI use can earn and keep their audience's trust. The use of AI in journalism will likely continue to grow.
AI Accelerates Science
Google DeepMind's AlphaFold solved a 50-year problem in physics. It predicted the shape of proteins using a large database of known shapes. This breakthrough used artificial intelligence and a lot of data. AlphaFold's success matters because it showed AI can make big discoveries. It used 170,000 known protein structures to make predictions. This database was built over 53 years and cost $21 billion. AI will change science but not as quickly as expected. New discoveries will come from AI agents. AI will keep changing science in the years to come.
AI Detects Lies Using Its Geometry
Researchers have developed a new method for detecting misinformation using the hidden patterns in AI's own thought process. Instead of checking surface-level words or searching for external evidence, this technique looks at how language models represent truth and lies within their internal systems. By analyzing the way these models process information, scientists can pinpoint a "falsehood direction" that helps identify misleading statements. This approach doesn't require fine-tuning the AI or accessing outside data-it just uses what's already inside the model. The method was tested on 11 different AI models, from small to large, and proved effective in spotting lies across three fact-checking tests. It especially helped smaller models perform better, which could be a game-changer for systems with limited resources. While it works well on some datasets, it struggles with those that rely heavily on evidence-based labels. Still, this breakthrough shows that truthfulness can be recognized as a clear pattern in the way AI thinks, offering a new tool to fight misinformation without relying on external data. Looking ahead, researchers hope this approach will lead to more reliable ways to detect lies online, especially for smaller or less powerful AI systems.
AI Breakthrough in Predicting Crystal Structures from Sparse Data
Scientists have developed a new machine learning framework called ED-CSP that can predict crystal structures using chemical composition, atom count, and sparse electron diffraction (ED) data. Unlike previous methods that rely on indexed reflections or predefined structure libraries, ED-CSP uses multi-view aggregation and a relational set encoder to generate accurate lattice parameters and atomic coordinates. The team trained the model using ED-CS, a dataset of 4.85 million simulated crystal structures. When tested on 2,075 materials from CHILI-100K, the framework achieved a structural match rate of 57.49% at the top five candidates, surpassing existing models like PXRDGen (52.92%). Scaling up the training data further improved performance to 66.27%. Importantly, the model demonstrated generative capability by achieving 53.52% accuracy on materials not seen during training. This advancement marks a significant step forward in crystallography, enabling researchers to predict structures from limited diffraction data with high accuracy. Future work could extend this approach to experimental ED data, potentially revolutionizing materials science and drug discovery.
AI Model Creates 16 New Viruses
Scientists trained an AI model to recognize patterns of DNA structure in nature and re-write them to create new viral genomes. The AI model invented 16 new bacteria-infecting viruses, which were then synthesized in a laboratory and successfully infected E. coli. This matters because it could lead to breakthroughs in treating antibiotic-resistant superbugs by allowing scientists to generate tailor-made therapies. The new viruses possessed sequence patterns distinct from anything found in nature, and the study's authors excluded human pathogen datasets from their training models, meaning the viruses it created aren't capable of infecting people. The ability to custom-design new viruses will likely continue to advance in the future.