NVIDIA Makes AI Workload Management Open Source
In brief
- NVIDIA has made its Run:ai platform fully open source, adding it to the NVIDIA DSX AI Factory Platform.
- This move offers production-grade AI workload and GPU orchestration capabilities to customers, developers, and partners.
- The release includes four modular components that can be used independently: KAI Scheduler for Kubernetes GPU scheduling, GPU Fractioning Operator for CUDA-native sharing, KAI Resource Management for organizational quotas, and Workload Map for consistent management.
- This decision aims to accelerate innovation across the DSX ecosystem, enabling faster delivery of AI factory solutions.
- Eight partners, including Crusoe and Mirantis, are integrating Run:ai into their offerings.
- Additionally, Model Streamer and Grove projects complement the platform by speeding up model loading and managing multi-component workloads.
- By open-sourcing Run:ai, NVIDIA allows ecosystem partners to improve GPU utilization and streamline shared AI infrastructure operations.
- Developers can now access tools that dynamically allocate resources across data processing, training, and inference stages.
- As AI adoption grows, expect more tools like these to emerge, supporting efficient scale in the industry.
Terms in this brief
- Run:ai
- A platform by NVIDIA that helps manage AI workloads and GPU resources efficiently. By making it open source, NVIDIA aims to accelerate innovation in their DSX AI Factory Platform, allowing developers and partners to better utilize GPUs for tasks like model training and inference.
- KAI Scheduler
- A component of the Run:ai platform that schedules GPU resources on Kubernetes clusters, ensuring efficient allocation of computational power for AI workloads.
- GPU Fractioning Operator
- Another part of Run:ai that allows multiple AI tasks to share GPUs effectively, optimizing resource usage for different processes like data processing and model training.
- KAI Resource Management
- A tool within Run:ai that sets up quotas and manages GPU resources at an organizational level, helping teams allocate computational resources fairly and efficiently.
Read full story at NVIDIA Dev Blog →
More briefs
Open-Source Tool Enhances AI Agent Efficiency
MuscleMemory, an open-source tool for AI agents, has been developed to improve their efficiency by leveraging past successful tasks. It observes how agents perform tasks and learns from their actions, providing concise notes for future similar tasks. In testing with Codex and Claude Code, it reduced input tokens and tool calls significantly while maintaining success rates. For example, in real Django issues, it solved problems using about half the tokens compared to agents without memory. The tool's effectiveness lies in its ability to remember successful outcomes and avoid repeating failures. It integrates seamlessly with various AI frameworks like Codex and Claude Code, offering a lightweight solution for project-specific setups. Users can install and configure it quickly, allowing their agents to operate more efficiently without extra setup. Looking ahead, MuscleMemory could revolutionize how AI agents handle repetitive tasks by reducing redundancy and enhancing decision-making through learned experiences. Its open-source nature encourages further development, potentially benefiting a wide range of applications.
New Tool Lets Mac Users Remove Apple's AI Features
A new open-source tool called RemoveMacAI is gaining popularity among Mac users. This tool allows them to disable and delete several of Apple's AI features, including Siri, Writing Tools, Genmoji, Image Playground, and the ChatGPT Extension. The tool also prevents these features from being reinstalled, freeing up valuable storage space-some users have reported saving over 35 GB. Many users feel that Apple's AI offerings are not practical or worth the space they consume. The project has already garnered nearly 2,000 stars on GitHub and dozens of forks, showing strong demand. While some praise the tool, others express disappointment that such a solution was necessary in the first place, highlighting the growing need for user control over AI features.
Open-Source AI Memory Projects Enhance Agent Continuity
AI agents, despite their ability to process vast amounts of information, often struggle with retaining context between interactions. To address this, developers are turning to open-source memory systems that preserve key details like user preferences and task progress. These systems use APIs, graphs, and benchmarks to ensure agents maintain consistency across sessions. Recent advancements in open-source projects focus on making AI memory more reliable and portable. By storing information in structured formats, these tools enable agents to recall past interactions accurately, enhancing their functionality. This development is particularly valuable for industries like healthcare and finance, where continuity is critical. Looking ahead, the integration of these memory systems into mainstream applications will be a key trend to watch. As AI becomes more dependent on retaining context, open-source projects are likely to play a pivotal role in shaping this capability.
AI Agents Get a Boost with New Open-Source Skills for Healthcare and Science
AI agents, often used in healthcare and science, have trouble applying guidelines correctly even when they know the rules. For example, asking an agent to classify genetic variants using specific criteria might result in correct references but wrong applications. Now, researchers have introduced 38 open-source skills tailored for 11 key areas like genomics and drug discovery. These skills help AI agents apply decision-making frameworks more accurately, improving tasks like variant interpretation and clinical trial design. The new skills are structured as markdown files that guide AI through complex decisions. They cover everything from genetic analysis to imaging techniques, ensuring agents follow proper procedures. Tests show these skills boost performance by 70-86% in head-to-head comparisons, with the biggest gains in critical thinking tasks. The skills are freely available under a MIT license, allowing anyone to use, modify, or extend them for their needs. This development marks an important step in making AI more reliable in healthcare and science. As these tools evolve, expect to see even greater improvements in how AI assists researchers and clinicians, potentially leading to faster and more accurate medical breakthroughs.
Alibaba’s Qwen3.8-2.4T-A95B Model Goes Open Source on Amazon SageMaker
Alibaba’s Qwen team has released the Qwen3.8-2.4T-A95B model as open weights, marking a significant milestone in AI accessibility. This powerful 2.4 trillion-parameter model is optimized for demanding tasks like multi-step coding and long-term planning. By making its weights available, Alibaba empowers developers to customize inference behavior and avoid per-token fees, though it requires specialized GPU infrastructure to host. The model features a hybrid architecture that combines linear attention with full attention, enabling efficient processing of up to 262K tokens-extendable to 1 million. This design is tailored for agentic AI workloads, where models must maintain state across many interactions without relying on inefficient chain-of-thought methods. The release includes support for Amazon SageMaker HyperPod, allowing seamless deployment using vLLM on high-performance instances like ml.p6-b300. With this move, Alibaba joins other companies in democratizing access to cutting-edge AI models. Developers can now experiment with a model capable of handling complex reasoning and tool usage, setting the stage for new innovations in AI capabilities and applications.