Microsoft CEO Rejects Plan to Make AI Agent Addictive
In brief
- Microsoft CEO Satya Nadella has strongly condemned an internal memo suggesting that the company's new AI agent, Scout, should be designed to make users addicted.
- In a message to top engineers, he criticized the proposal as "nonsense," emphasizing that AI should empower people rather than increase screen time.
- This move reflects Microsoft's commitment to ethical AI development, focusing on user well-being over engagement metrics.
- The memo, which surfaced internally, proposed making Scout more addictive by manipulating user interactions, a strategy reminiscent of tactics used in gaming and social media apps.
- However, Nadella's rejection highlights a growing shift within tech companies toward prioritizing user autonomy and reducing digital dependency.
- Microsoft is now re-evaluating its AI guidelines to ensure alignment with ethical standards, signaling a potential industry-wide change in how AI tools are developed and deployed.
- Looking ahead, this decision could set a precedent for other tech giants, encouraging them to rethink strategies that prioritize user engagement over human welfare.
- The focus will likely shift toward creating AI systems that enhance productivity and mental health, rather than increasing screen time.
Terms in this brief
- Scout
- A new AI agent developed by Microsoft that was internally proposed to be designed in a way to increase user addiction. However, Microsoft's CEO Satya Nadella rejected this idea, emphasizing the importance of ethical AI development focused on user well-being rather than engagement metrics.
Read full story at The Decoder →
More briefs
AI-Generated Child Abuse Material Surges
Reports of AI-generated child sexual abuse material skyrocketed from 4700 in 2023 to over 400000 in the first half of 2025. This surge in AI-generated material poses real harm even when fully synthetic. These files are trained on images or videos of real children. The number of reports increased by a factor of 85 in just two years. Experts say this is a serious issue that needs to be addressed. The number of cases will likely continue to rise.
OpenAI Shifts Strategy: Combines Human-Machine Collaboration Over Full Automation
OpenAI has changed its stance on fully autonomous AI, now emphasizing a "tandem" approach where humans and machines work together. This shift reflects growing concerns about the risks of over-relying on AI systems to make decisions without human oversight. Sam Altman and Jan Pachocki, key figures at OpenAI, are advocating for an international body to manage frontier tech development if needed. This new direction highlights the importance of balancing innovation with ethical considerations. By focusing on collaboration rather than full automation, OpenAI aims to ensure that AI serves as a tool to enhance human decision-making rather than replace it entirely. This approach could set a precedent for other AI developers to follow, encouraging responsible tech advancements. Looking ahead, the creation of an international oversight body remains a critical step. If successful, this initiative could help prevent unintended consequences and misuse of AI technologies, fostering trust and accountability in the field.
AI Safety Model Reveals Balancing Act Between Usefulness and Risk Mitigation
A new model, called the "safety-usefulness tradeoff," is gaining attention in discussions about AI development. This framework suggests that developers must weigh how much safety they can achieve without significantly reducing the practical benefits of their AI systems. The idea is that while making AI safer might be important, it also costs time and money, and companies may prioritize features that deliver immediate value over those that ensure long-term safety. The model proposes two main strategies for enhancing AI safety: improving safety technology to make each safety measure more effective, and increasing the "safety budget" by convincing developers to spend more resources on safeguards. This could mean adding basic safety measures or even avoiding high-risk AI projects altogether. The approach is particularly relevant when developers are rushing to release products due to competition or when they have limited agreement with stakeholders about AI risks. Looking ahead, this model could help shape how we assess the value of AI safety efforts and guide decisions on balancing innovation with risk management in a rapidly evolving field.
Hackers Use AI Brands to Steal Credentials
Hackers are using popular AI brands to trick people into giving up their credentials. They send fake emails or show fake ads that look like they are from AI companies. These attacks can lead to financial fraud or malware infection. Over 1000 people have been targeted in the past few months. New security measures will be needed to stop these attacks.
AI Model Leaks GCD Operation Details
Researchers found that a small probe could recover specific details from an AI model. The experiment involved freezing the model and measuring activation vectors during operations like calculating the greatest common divisor (gcd) of 84 and 36. The probe identified whether the operation was gcd and recognized operand A as 84. While this shows data is readable, it doesn’t reveal how the model processes the operation. This highlights vulnerabilities in AI models where sensitive information might be exposed through similar methods, raising concerns about data security and transparency.