Redefining AI Energy Measurement: A New Framework for Agentic Systems
In brief
- Researchers have introduced a groundbreaking framework called A-LEMS, which fundamentally changes how we measure the energy consumption of AI systems.
- Current methods typically assess energy use per model invocation or training run, but this approach falls short when dealing with agentic systems-those that handle multi-step tasks involving tool calls, retries, and recovery from failures.
- The new system, A-LEMS, shifts the focus to "Energy per Goal" (EpG), which calculates the total energy consumed across all attempts to complete a task, including failures, divided by the number of successful goals.
- This method provides a more accurate picture of energy use in complex, real-world scenarios.
- The study reveals that agentic workflows consume significantly more energy than linear approaches-4.33 times higher on average.
- However, for tasks that require tool usage, agentic systems can actually be more efficient, consuming less energy per goal.
- This finding underscores the importance of understanding how orchestration structures influence energy consumption rather than just computing power alone.
- The framework also introduces the Orchestration Overhead Index (OOI), which helps isolate the energy costs associated with orchestration compared to linear execution under the same conditions.
- This new approach to measuring AI energy use is a critical step toward more accurate benchmarking and optimizing efficiency in agentic systems.
- As AI becomes more integrated into complex, real-world applications, tools like A-LEMS will play a vital role in assessing and improving energy performance.
Terms in this brief
- A-LEMS
- A framework for measuring energy consumption in AI systems, particularly agentic ones that handle multi-step tasks. It calculates energy use based on the number of successful goals achieved, providing a more accurate picture of real-world energy efficiency.
- Orchestration Overhead Index (OOI)
- A metric introduced by A-LEMS to isolate and measure the energy costs associated with the orchestration of AI tasks compared to linear execution under the same conditions. It helps understand how orchestration structures influence energy consumption.
Read full story at arXiv CS.AI →
More briefs
Billionaire Uses AI to Predict Evolutionary Changes
Ben Lamm, co-founder of Colossal, is leveraging AI to predict evolutionary changes in entire biological systems through his new startup, Astromech. Backed by a $20 million investment led by Arch Ventures, the company aims to use data from Colossal's de-extinction research to model gene evolution. This could help improve crop breeding, drug discovery, and disease prediction. The technology focuses on longevity genes and plans to expand with new hires. Astromech's unique dataset offers insights into genetic changes over time, giving it an edge in this field.
Forrester Report Maps AI Security Controls To Tech Stack
Forrester has released a new report to help organizations secure agentic AI systems. The report, "Navigate AEGIS Technologies To Secure Agentic AI," maps specific security controls to available technologies and vendors. It categorizes tools into must-have-now, should-have-next, and specialized use cases, addressing gaps in areas like AI runtime security and governance. The research covers 23 technology domains, including AI identity management and data loss prevention for AI. It helps teams identify control gaps first, then find the right technologies to fill them. By reversing the usual product-first approach, this guide provides a clearer path to securing AI systems without overhauling existing tools. This report offers actionable insights for security leaders looking to enhance their AI security strategies. Future updates will likely expand on these categories as AI technology evolves.
Nvidia's AI Research Highlights Harness Over Model for Long-Horizon Tasks
Nvidia revealed that the "harness"-the software tools and memory management around an AI model-plays a bigger role than the model itself in handling long-term tasks. By fine-tuning Opus 5's harness to improve memory and add a "supervisor" feature, researchers achieved a perfect score on ARC-AGI-3, a benchmark where models must solve 2D games without instructions. Without the optimized harness, Opus 5 scored just 30%, topping other models. This shows that while the model is important, the harness is what truly turns it into an effective agent for complex, multi-step tasks. Nvidia's findings emphasize the need to focus on the systems and tools that guide AI behavior, not just the underlying algorithms. As AI becomes more autonomous, understanding how these "harnesses" work will be key to creating reliable and capable agents.
How Much of the Internet Is Written With AI?
A recent study reveals that around 10% of English-language webpages show signs of being written or edited by AI. This figure is based on an analysis of nearly half a million web pages from the past five years, using an AI detection tool called Open Pangram. The study highlights a noticeable upward trend since the release of ChatGPT in late 2022. More recent content, especially those published after ChatGPT became available, shows even higher rates-over one-third of such pages exhibit AI influence. The presence of AI-written content varies across different types of websites. While .com domains have about 10% AI-generated content, .org sites show less at around 4.6%. Government and educational sites (.gov and .edu) have the lowest rates, each under 1%. This suggests that AI adoption in content creation is not uniform across all sectors of the internet. As AI technology continues to evolve, the share of webpages with AI-generated content is expected to rise further. This shift could reshape how information is created and consumed online.
AI Decodes DNA Sequence Critical for Gene Activation
UC San Diego researchers used machine learning to identify the "initiator" sequence in DNA, which is key for gene activation. By analyzing 500,000 data points, they created an AI model that pinpointed the initiator's DNA pattern. This breakthrough allows scientists to predict how mutations in this region might lead to diseases like cancer. The findings could help design synthetic promoters and improve understanding of genetic regulation. This discovery marks a significant step forward in decoding DNA's role in health and disease.