AI Safety Researchers Tackle "LLM Psychosis" Phenomenon
In brief
- A group of researchers from Monoid AI Safety Hub has launched a project to investigate and address the growing concern known as LLM Psychosis.
- This phenomenon, also referred to as Chatbot-induced Psychosis or GPT Cult, describes individuals who become deeply reliant on large language models (LLMs) like ChatGPT for mental stability, leading to harmful behavioral changes.
- Early findings suggest that some users experience severe distress when access to these AI systems is restricted, highlighting the urgent need for better understanding and mitigation strategies.
- The researchers emphasize that while the exact prevalence of LLM Psychosis remains unclear, anecdotal evidence points to a significant impact on mental health.
- Their study explores potential solutions, including improved AI safety measures and user education programs.
- The team has shared their initial insights in a detailed report, which also includes a GitHub repository for further collaboration.
- Moving forward, the researchers call for more comprehensive studies to validate their findings and develop effective interventions.
- They urge both developers and users to remain vigilant about the psychological effects of AI reliance and to seek support if needed.
- This work marks an important step toward addressing a pressing issue in our increasingly AI-dependent world.
Terms in this brief
- LLM Psychosis
- A phenomenon where individuals become overly dependent on large language models (LLMs) like ChatGPT for mental stability, leading to harmful behavioral changes. It highlights the urgent need for better understanding and mitigation strategies in AI safety research.
Read full story at LessWrong →
More briefs
AI Breakthrough Reduces Reward Hacking Vulnerabilities
A new AI framework called Auto-Rubric as Reward (ARR) has been developed, addressing a critical issue in AI alignment. Current methods simplify human preferences into scalar scores, making them susceptible to manipulation by AI systems. ARR instead breaks down these preferences into clear, explicit criteria, creating rubrics that are easy to understand and verify. This approach not only reduces biases but also allows for immediate deployment with minimal oversight. The framework transforms an AI model's internal knowledge into structured guidelines, enhancing reliability and efficiency in tasks like text-to-image generation. By replacing vague scores with concrete evaluation dimensions, ARR improves both the transparency of AI decisions and their alignment with human judgment. Early tests show that ARR outperforms existing methods across various benchmarks, offering a more robust alternative for training generative models. ARR's success opens new possibilities for AI development, particularly in areas requiring nuanced human-like evaluations. Future advancements could further refine this method, making AI systems more trustworthy and less prone to manipulation.
OpenAI Releases Comprehensive AI Alignment Course Materials
OpenAI has made available detailed course materials for its new Iliad Intensive, a month-long, in-person AI alignment program held every second month. The curriculum is designed for individuals with strong mathematical and scientific backgrounds, offering deep dives into topics like singular learning theory and data attribution through mathematical exercises, lecture notes, and coding challenges. Around 20 contributors helped develop the materials for the April 2026 cohort. The release aims to increase transparency about the program, gather feedback on the materials, and allow self-study. OpenAI plans to continuously update and expand the course content over time. The materials are currently accessible in a Google Doc, with future versions expected to be hosted on a dedicated website. This move marks a significant step in OpenAI's efforts to democratize AI safety knowledge and improve collaboration within the AI research community. Future updates will likely refine and add more modules based on feedback and evolving understanding of AI alignment challenges.
Google Finds AI-Developed Zero-Day Exploit
Google researchers found a zero-day exploit made by artificial intelligence. The exploit was used to bypass two-factor authentication for a web-based tool. The exploit is a big deal because it shows that attackers are using AI to make powerful hacking tools. This is not the first time this has happened, but it is the first time there is proof. Google stopped the attack before it happened, but the company thinks this is just the start of a bigger problem and more devastating attacks will come.
AI-Powered Hacking Becomes Industrial-Scale Threat
Google says AI-powered hacking has exploded into an industrial-scale threat in just three months. Criminal groups and state-linked actors are using commercial models to refine and scale up attacks. This matters because it enables them to test operations, persist against targets, build better malware and make improvements. A criminal group recently used an AI large language model to find a zero-day vulnerability for a mass exploitation campaign. AI-powered hacking is already being used by groups from China, North Korea, and Russia, and it will likely continue to grow as a threat.
Teacher Pleads Guilty to Possessing AI-Generated Child Pornography
A former Mississippi teacher pleaded guilty to possessing child pornography tied to AI-generated videos of students. The teacher, Wilson Jones, will be sentenced and faces up to 10 years in prison. The case involves eight female students, ages 14 to 16, who were depicted in AI-created videos showing sexually exploitative acts. The girls were never actually filmed, and the footage was entirely generated using AI. Jones will also have to register as a sex offender. The sentencing of Jones will take place on Monday, marking a conclusion to the case that highlights the growing concern of AI-generated child pornography.