Google Announces 8x Faster TPU 3.0 For AI, Machine Learning

For the past few years, Google has been building its own TPU (Tensor Processing Units) to handle various processing tasks related to artificial intelligence and machine learning. Google first announced the existence of TPUs in 2016, but said it had been using them internally for more than a year. The company’s second-generation TPUs have made the news recently for significantly improved performance, and the third-generation hardware will apparently continue that trend.

According to Google CEO Sundar Pichai, the new TPU 3.0 pods are 8x more powerful than the Google TPU 2.0 pods that we’ve previously covered, and Pichai declared that they’re power-hungry enough to need water cooling — something previous TPUs simply haven’t required. TechCrunch’s image shows a copper-plated cooler system, with water from the same pipe run through all four cooling plates.

So what do we know about TPU 3.0? Not much — but we can make a few educated guesses. According to Google’s own documentation, TPU 1.0 was built on a 28nm process node at TSMC, clocked at 700MHz, and consumed 40W of power. Each TPU PCB connected via PCIe 3.0 x16.

TPU 2.0 made some significant changes. Unlike TPU v1, which could only handle 8-bit integer operations, Google added support for single-precision floats in TPU v2 and added 8GB of HBM memory to each TPU to improve performance. A TPU cluster consists of 180 TFLOPS of total computational power, 64GB of HBM memory, and 2,400GB/s of memory bandwidth in total (the last thrown in purely of the purposes of making PC enthusiasts moan with envy).

Unlike TPU v1, which used 3.5-inch drive bays as its form factor, TPU v2 was welded together in groups of four ASIC chips. Google currently deploys TPUs in clusters of up to 64 boards, at 11.5 PFLOPS per cluster and 4TB of total HBM storage. Power consumption requirements were already estimated to be pretty high with last year’s model, and these solutions consume even more power, so the switch to water cooling makes sense — it’s probably the only way to deal with the heat output, especially if Google is stuffing 64 TPUs into a single cluster.

No word yet on other advanced capabilities of the processors, and they are supposedly still for Google’s own use, rather than wider adoption. Pichai claims TPU v3 can handle 100 PFLOPS, but that has to be the clustered variant, unless Google is also rolling out a new tentative project we’ll call “Google Stellar-Equivalent Thermal Density.” We would’ve expected to hear about it, if that was the case. As more companies flock to the AI / ML banner, expect to see more firms throwing their hats into this proverbial ring.

Continue reading

Micron Announces 176-Layer NAND, Volume Shipments Underway

Micron has announced its 176-layer NAND today in an impressive step forward for the industry.

In Massive Shift, Apple Announces New Macs With ARM-Based M1 Chip

Apple saw huge success the last time it switched architectures to Intel, but this time? The jury's still out, but one thing is certain: Apple is about to make a lot more money.

Asus Announces Chromebox 4 With Support for 10th Gen Core Processors

Chromebooks are so plentiful these days they might as well grow on trees. There are fewer Chromeboxes, but Asus has been keeping its line updated and just announced its latest version.

Seagate Announces Its Own RISC-V Cores for Future Storage Controllers

To hit its 50TB per-drive target over the next few years, Seagate decided it needed a custom storage controller. RISC-V offered a solution.