Redditor Builds 1-Terabyte Parameter LLM Using Optane PMem as RAM
In brief
- A tech enthusiast has made waves by using Intel's discontinued Optane Persistent Memory (DCPMM) sticks as RAM to run a 1-trillion-parameter large language model locally.
- The setup, detailed on the Local LLaMA subreddit, uses six used Optane PMem sticks totaling 768GB of memory.
- Despite Optane's slower performance compared to traditional DRAM, it offers lower latency than NVMe SSDs and was acquired at a fraction of the cost of equivalent DRAM capacity.
- The build includes an Intel Xeon Gold 6246 CPU, Samsung DDR4 ECC RAM for cache, and a Western Digital NVMe SSD.
- Running Kimi K2.5, the system achieves about 4 tokens per second using a hybrid GPU/CPU inference method with llama.cpp.
- While Optane's discontinuation limits this approach's scalability, the creator views it as a successful proof-of-concept.
- The experiment highlights the potential of repurposing older memory technologies for cutting-edge AI tasks, despite their limitations.
Terms in this brief
- Optane PMem
- Intel's Optane Persistent Memory (DCPMM) is a type of memory technology that combines the speed of RAM with the durability of storage. It allows for larger memory capacities and persistence, meaning data isn't lost when power is removed. This makes it suitable for high-performance computing tasks like running large language models locally.
Read full story at Hacker News →
More briefs
California Gas Stations Accused of Using AI to Raise Prices
A lawsuit claims many California gas stations used an AI tool to inflate prices. The lawsuit says this caused prices to surge in the state. The suit claims the AI tool allowed gas stations to avoid competing with other stations and charge higher prices to consumers. This could make gas prices rise by as much as 30 cents per gallon in some areas. The average gas price in California is $5.52 per gallon, the highest in the country. The lawsuit was brought against 1,732 gas stations in the state, including those owned by Walmart and Albertsons. The stations are accused of using the AI tool to set prices, which is illegal under a 2025 law. Gas prices in California will likely be watched closely as the lawsuit moves forward.
AI Beats Lawyers in UK Court
A woman used artificial intelligence to win a court case in the UK. She paid about $529 for the AI help and won $9,271 in unpaid fees. The AI handled pretrial work and a human argued the case. This is the first trial won by an AI law firm in the world. The AI law firm has processed over 600 claims and recovered about $662,000 for clients. The AI made the process more accessible and affordable. The woman was delighted by the result and said she could pursue the claim thanks to the AI. The use of AI in law will continue to grow.
NVIDIA Introduces Energy Efficient AI Servers
NVIDIA has introduced new AI servers that can run their cooling liquid at up to 45 degrees Celsius. This higher temperature limit makes them more energy efficient. The new servers are part of the Rubin generation of NVIDIA AI infrastructure. The new servers are the world's first to achieve 100% liquid cooling. This means every chip and component is cooled by liquid in a closed loop with no fans. Historically, cooling has accounted for up to 40% of a data center's electricity consumption. Raising chiller plant temperatures by just one degree can cut cooling energy costs by about 4%. A 50-megawatt hyperscale facility can save over $4 million annually in cooling-related energy and water costs. The new liquid-cooled infrastructure can reduce facility cooling water consumption from roughly 2.6 million gallons per megawatt per year to near zero. NVIDIA's 45-degree liquid cooling will enable data centers to operate more efficiently in the future.
Reflection AI Partners with SpaceX for AI Chips
Reflection AI will pay $150 million a month for access to Nvidia's latest AI chips. The deal is worth up to $6.3 billion and will last through 2029. The deal matters because it shows the value of open source AI. Reflection AI used this deal to promote its open-weight AI strategy. This strategy is an alternative to closed models like those used by Anthropic and OpenAI. The company will use the AI chips to build open models at scale. Reflection AI will have more computing power to work on its projects. The company will start using the AI chips on July 1, 2026.
NVIDIA Launches Safety System for Robots
NVIDIA has launched a safety system for robots called Halos for Robotics. This system is the first full-stack safety system for robots and physical AI. The system is important because it helps robots work safely with humans. Over 18,600 engineering years of autonomous vehicle safety development went into creating this system. NVIDIA will continue to work with companies to build safer robots using this system.