MIT Neural Network Processor Cuts Power Consumption by 95 Percent

Neural network processing and AI workloads are both hot topics these days, driving multiple companies to announce their own custom silicon designs or to plug their own hardware as a top-end solution for these workloads. But one problem with neural networks is that they tend to be extremely power intensive, and not necessarily suited to mobile devices or the kind of low-power “smart” speakers that have recently become so popular.

MIT is claiming to have developed a neural network processor that addresses these problems, with an overall power reduction of up to 95 percent. If true, this could change the game for these kinds of applications. Instead of being forced to rely on cloud connectivity to drive AI (and using power to keep the modem active), SoCs could incorporate these processors and perform local calculations.

“The general processor model is that there is a memory in some part of the chip, and there is a processor in another part of the chip, and you move the data back and forth between them when you do these computations,” said Avishek Biswas, an MIT graduate student in electrical engineering and computer science, who led the new chip’s development:

Since these machine-learning algorithms need so many computations, this transferring back and forth of data is the dominant portion of the energy consumption. But the computation these algorithms do can be simplified to one specific operation, called the dot product. Our approach was, can we implement this dot-product functionality inside the memory so that you don’t need to transfer this data back and forth?

A typical neural network is organized into layers. Each node connects to other nodes above and below it, and each connection between nodes has its own weight. Weight, in this context, refers to how much of an impact computations performed in one node will have on the calculations performed in the nodes it connects to. Nodes receiving input from multiple nodes above it multiply the inputs they receive by the weight of each input. The result is called the dot product. If the dot product is above a certain threshold, it gets sent along to nodes farther down the chain. But this process is extremely memory intensive, with each dot product calculation requiring memory accesses to retrieve the weighted values. Those values then have to be stored, and each input to a node has to be independently calculated.

What MIT has done is create a chip that more closely mimics the human brain. Input values are converted to electrical voltages, then multiplied by appropriate weights. Only the combined voltages are converted back into digital representation and stored for processing. The prototype chip can calculate 16 dot products simultaneously. By storing all of its weights as either 1 or -1, the system can be implemented as a simple set of switches, while only losing 2-3 percent of accuracy compared with the vastly more expensive neural nets.

Not bad for an approach that can reduce power consumption up to 95 percent. And it’s a promising concept for a future in which the benefits of the cloud and AI aren’t limited to those with robust internet service in a mobile device or at home.

Top image credit: Chelsea Turner/MIT

Continue reading

Asus Announces Chromebox 4 With Support for 10th Gen Core Processors

Chromebooks are so plentiful these days they might as well grow on trees. There are fewer Chromeboxes, but Asus has been keeping its line updated and just announced its latest version.

AMD Launches New Ryzen 5000 Mobile Processors

AMD's Ryzen Mobile 5000 chips are now shipping in OEM systems. So what kind of performance improvements do they really offer?

How Are Process Nodes Defined?

We talk about process nodes a lot at wfoojjaec, but we don't always drill down into how nodes are defined or how they differ between manufacturers. Let's fix that.

Samsung Stuffs 1.2TFLOP AI Processor Into HBM2 to Boost Efficiency, Speed

Samsung has developed a new type of processor-in-memory, built around HBM2. It's a new achievement for AI offloading and could boost performance by up to 2x while cutting power consumption 71 percent.