Google builds its first chip just for Machine Learning

Last week's Google IO conference generated a lot of interesting news. One of the more intriguing items was the revelation that Google is building its own chip for machine learning (ML). The Google chip is designed to work with Google TensorFlow language and is called the Google Tensor Processing Unit TPU).

 The company has been using the TPU for almost 1 year in applications such as speech recognition and for the AlphaGo AI that beat a world-class Go player. Google has been running TPUs in its data centers for over a year, and it's already used for optimizing search results and Street View. Here's what we know so far and what Google has not yet revealed.

To build the chip, Google hired top chip designer Norman P. Jouppi as a Distinguished Hardware Engineer. The company claimed it evaluated GPUs (graphic processing units) and FPGAs (field programmable gate arrays) before deciding on a custom ASIC design because the ASIC was more efficient at ML interference on both power and performance. The reasons the TPU is more power efficient than alternatives is likely because it was designed just for the inference part of neural net framework, which can still be effective with very limited math precision, while GPUs have additional logic to do higher precision math.

While an FPGA, like those from Xilinx and Intel's Altera ALTRdivision, can be configured for 8-bit math, it may not clock as fast as an ASIC and there is also additional circuitry overhead in the FPGA because it's very configurable. An FPGA is considered the best solution when the chip's low volume makes the non-reoccurring engineering costs of a full ASIC uneconomical or ongoing changes will be needed to the logic. The GPUs and FPGAs are more flexible and programmable, but the FPU is more tailored to one task. Using an analogy - FPGAs and GPUs are off-the-rack suits, while an ASIC is a custom tailored suit.