AI-Generated References Flood Biomedical Research
In brief
- An audit of 2.5 million biomedical papers reveals a shocking surge in fabricated references, jumping twelvefold since 2023.
- Researchers suspect AI language models are behind the trend, creating fake citations that match paper topics perfectly and follow proper formatting-making them nearly undetectable.
- Over 98% of these affected papers have not received any response from publishers.
- This issue matters because clinical guidelines and research integrity are at risk.
- If false references influence decision-making, patient care could be compromised.
- The audit highlights a growing problem in academic publishing, where AI tools may inadvertently or intentionally create misleading citations.
- Looking ahead, publishers and researchers need to adopt new detection methods to combat this trend.
- Collaboration between the AI and scientific communities will be crucial to maintain trust in biomedical research.
Terms in this brief
- AI language models
- These are advanced computer programs trained to understand and generate human-like text. They learn from vast amounts of data and can perform tasks like answering questions, writing articles, or even engaging in conversations, much like a human would.
Read full story at The Decoder →
More briefs
Fake Journals Publish AI-Generated Papers
A network of fake academic journals has published over 100 AI-generated papers in recent months. These papers use the names of real professors at top universities without their knowledge. This matters because it can damage the reputation of real academics and undermine trust in academic research. The fake papers can also spread false information and affect the quality of research. New rules may be needed to stop fake journals and AI-generated papers.
AI Model Predicts Cancer Treatment Response
Researchers at University of California San Diego developed a new AI model that can predict how cancer may respond to treatment. The model was trained on genomic data from more than 30,000 tumors. The model offers a new framework for connecting cancer mutations to treatment response. This matters because genetic sequencing is already routine in cancer care, but interpreting the many mutations found in a patient's tumor is still a challenge. Only about 8% of cases are successfully matched to an FDA-approved therapy on the basis of genetics. The new AI model could help make tumor DNA testing more clinically actionable, which may lead to more precise predictions about treatment response in the future.
AI Finds Folic Acid May Help Heal Diabetic Wounds
Scientists used artificial intelligence to find new uses for old drugs. They looked at 3000 existing drugs to see if any could help heal diabetic wounds. This matters because diabetic wounds are hard to heal. Many biological processes are disrupted at the same time. The scientists used AI to scan scientific literature and identify which drugs may help. They found that folic acid, a common vitamin, is a top candidate to help heal diabetic wounds. The team will now test folic acid in more experiments to see if it really works.
Scientist Runs Groundbreaking Computer Simulation
Mary Tsingou wrote code for a first-of-its-kind numerical experiment in 1955. She used one of the world's first scientific computers to run the experiment. The results of the experiment showed that nonlinear systems behave in surprisingly stable and structured ways. This discovery reshaped how scientists think about systems like the atmosphere and the human heart. The experiment involved a one-dimensional line of masses connected by springs with a small nonlinear change to the spring force. The simulation took years to run and showed that energy flowed back into its original mode. This discovery was a watershed moment and changed how scientists think about nonlinear systems. The results will continue to influence scientific research.
AI's Personality Test Fails When Put to Work
A new study reveals that AI models trained to mimic specific personalities in chat conversations struggle when given real-world tasks. Researchers tested three major AI systems-Llama, Qwen, and Gemma-trained with personality-based fine-tuning (SFT). These models were scored using a classifier designed to identify their personas, achieving high accuracy (86-95%) in controlled chat settings. However, the same models performed poorly when asked to act autonomously-composing emails or making decisions. The classifier's accuracy dropped sharply to 29-55%, showing that AI personalities don't translate well beyond structured chat interactions. This suggests that SFT, a common training method for character-driven AI, may not prepare models for practical, agent-like tasks. The findings highlight the limitations of current personality-training techniques and emphasize the need for more generalized alignment methods. As AI becomes more integrated into daily life, understanding how these systems behave outside of controlled chats will be crucial for developers aiming to create reliable and versatile AI assistants.