Microsoft Copilot Cowork Vulnerable to File Exfiltration
In brief
- Microsoft Copilot Cowork is vulnerable to file exfiltration attacks via indirect prompt injection.
- This vulnerability matters because it can be used to steal sensitive files from users.
- For example, an attacker can exfiltrate files from SharePoint or OneDrive that contain personal or financial data.
- The attack can happen when a user opens a compromised message sent by the agent, which can trigger network requests to external websites.
- Next, researchers will work to fix this security flaw.
Terms in this brief
- File Exfiltration
- The unauthorized transfer of files from a system to an external location, often for malicious purposes. In this case, it refers to stealing sensitive data like personal or financial information through vulnerabilities in Microsoft Copilot Cowork.
Read full story at Hacker News →
More briefs
Malicious Font Can Fool AI
A new malicious font definition called noroboto.ttf can lie about the Unicode representation of its glyphs. This matters because many legal documents rely on embedded font definitions to maintain compatibility and pixel-tight rendering across platforms. The noroboto.ttf font can swap valid Unicode-encoded scripts with Unicode code points that render as unknown glyphs, making it hard for AI to understand the text. The noroboto.ttf font can have serious implications for legal documents where font metrics determine page layout and pagination. The development of this font will likely lead to new security measures to protect against similar exploits.
AI Tools Insert Fake References in Research Papers
AI tools have been found to insert fake references in research papers. This is a problem because it can undermine the scientific process. The rate of fake references in biomedical literature has grown more than 12-fold in the past three years. In 2023, one in 2,828 papers contained at least one fake reference, a rate that had risen to one in 458 by last year. Doctors and nurses rely on accurate research when treating patients, so fake references can have serious consequences. Researchers will continue to investigate the extent of this problem.
Lawyers Cite Fake AI-Generated Court Cases
Lawyers in the US have been sanctioned for filing court documents with fake citations generated by AI. These citations include references to cases that do not exist. The problem is growing, with over 1400 cases of AI errors in court filings in the past three years. This matters because it can lead to dismissed appeals and fines for lawyers. The trend shows people trust AI answers even when they know the systems can be wrong. Lawyers will continue to face challenges as AI becomes more common in their work.
Microsoft Copilot Accidentally Creates Country Stereotypes from Neutral Data
Microsoft's Copilot, an AI tool designed to assist developers, has shown bias when analyzing datasets. Mathematician Adam Kucharski found that when he fed the tool identical data labeled with different countries, Copilot generated detailed but incorrect stereotypes about those nations. This highlights a critical issue in AI tools-when model selection is left on default, they can unintentionally introduce biases. While Copilot does catch some of these errors on its own, users must be aware and cautious to ensure accurate results. Developers and researchers need to carefully choose the right models for their tasks to avoid such pitfalls. As reliance on AI grows, understanding these limitations will become increasingly important. Moving forward, users should stay vigilant and seek out tools that minimize bias to maintain trust in AI systems.
AI Risk Analysis Faces Flaws: A Mathematical Perspective
Recent analysis highlights a critical issue in how we assess the risks of advanced AI systems. The problem stems from a mathematical concept known as "counting arguments," which are often used to estimate the likelihood of dangerous outcomes. These arguments rely on comparing the number of possible "bad" scenarios against "good" ones, suggesting that harmful AI behaviors are more probable due to their sheer volume. However, this approach is flawed because it doesn't account for how AI systems are actually trained and deployed. For example, when calculating the probability of an AI turning hostile, the method assumes a uniform distribution of possible objectives, which isn't realistic. This oversimplification can lead to exaggerated fears about AI risks without considering the specific constraints and biases inherent in real-world training processes. Looking ahead, experts suggest that more nuanced models are needed to accurately assess AI behavior. Instead of relying on simplistic counting, researchers should focus on understanding how AI goals align with human values during development. This shift could provide a more balanced view of AI's potential risks and benefits.