Artificial intelligence (AI) computing requires tremendous amounts of electricity, but targeted research might hold the key to greatly reducing that. However, a team of researchers in the U.S. has developed technology that could reduce the energy consumption required by AI processing by a factor of at least a thousand.
A group of engineering researchers at the University of Minnesota Twin Cities have demonstrated an AI efficiency-boosting technology and published apeer-reviewed paperoutlining their work and findings. The paper was published innpj Unconventional Computing, a peer-reviewed journal by Nature. In essence, they’ve created a shortcut in the normal practice of AI computations that greatly reduces the energy requirement for the task.
In current AI computing, data is transferred between the components processing it (logic) and where data is stored (memory/storage). This constant shuttling of information back and forth is responsible for consuming as much as 200 times the energy used in the computation, according to this research.
Thus the researchers have turned to Computational Random-Access Memory (CRAM) to address this. The CRAM the research team has developed places a high-density, reconfigurable spintronic in-memory compute substrate within the memory cells themselves.
This differs from existing processing-in-memory solutions such asSamsung’s PIM technology, as Samsung’s solution places a processing computing unit (PCU) within the memory core. Data still has to travel from memory cells to the PCU and back, just not nearly as far.
Using CRAM, the data never leaves memory, instead undergoing processing entirely within the computer’s memory array. According to the research team, this allows the system running the AI computing application an energy consumption improvement “on the order of 1,000x over a state-of-the-art solution.”
Other examples suggest the potential for even greater energy savings and faster processing. In one test, performing an MNIST handwritten digit classifier task, the CRAM proved 2,500 times more energy-efficient and 1,700 times as fast as a near-memory processing system using the 16nm technology node. This task is used to train AI systems to recognize handwriting.
Get Tom’s Hardware’s best news and in-depth reviews, straight to your inbox.
The importance of this kind of work cannot be overstated. Recent reports suggest AI workloads already consume almost as much electricity as theentire nation of Cyprusdid in 2021. The total energy consumption came in at 4.3 GW in 2023 and is expected to grow at a rate of 26% to 36% in the coming years. Arm’s CEO recently suggested that by 2030, AI may consume a quarter of allenergy produced in the U.S.
The first author of the paper Yang Lv, a University of Minnesota Department of Electrical and Computer Engineering postdoctoral researcher, and the rest of the research team have already applied for several patents based on the new technology. They plan to work with leaders in the semiconductor industry, including those in Minnesota, to provide large-scale demonstrations and produce the hardware to help advance AI functionality while also making it more efficient.
Jeff Butts has been covering tech news for more than a decade, and his IT experience predates the internet. Yes, he remembers when 9600 baud was “fast.” He especially enjoys covering DIY and Maker topics, along with anything on the bleeding edge of technology.
