Intel’s Silicon Photonics Work Could Supercharge AI Neural Networks

Intel’s Silicon Photonics Work Could Supercharge AI Neural Networks

The various types of neural networks companies have experimented with so far have all been silicon-based. CPUs, GPUs, TPUs, and FPGAs may have different performance characteristics, but they’re built from the same sorts of materials. Intel, however, is working on building AI networks using silicon photonics, defined as the study and application of photonic systems that use silicon as an optical medium.

Two years ago, research work from MIT demonstrated that optical neural networks (ONNs) could be extremely useful for low-power, low-latency operations. This is possible because a common element of photonic circuits, known as a Mach-Zehnder interferometer (MZI), can be configured to perform 2×2 matrix multiplication. Placing the MZI’s in a triangular mesh allows for larger matrices to be created.

Casimir Wierzynski, Senior Director, Office of the CTO, Artificial Intelligence Products Group at Intel, writes:

As in any manufacturing process, there are imperfections, which means that there will be small variations within and across chips, and these will affect the accuracy of computations. In order to move ONNs closer to production, we wanted to understand how sensitive they were to typical process variations, especially as they scaled up to more realistic problem sizes. We also wanted to know whether we could make them more robust to these variations by considering different circuit architectures.

Intel’s new paper is a survey of two types of fault-resistant ONN. One of them had a more tunable design (GridNet), while the other was built with better fault-tolerance (FFTNet). As the name implies, Gridnet is designed as a grid, while FFTNet “arranges the MZIs in a butterfly-like pattern modeled after architectures for computing Fast Fourier Transforms.”

Intel’s Silicon Photonics Work Could Supercharge AI Neural Networks

Both ONNs were then trained to recognize handwriting. GridNet proved more accurate than FFTNet, at 98 percent to 95 percent, but FFTNet was specifically more robust when it came to handling manufacturing imprecision, which was simulated by adding noise and phase-shifting to each MZI. After simulating the real-world impact of this noise, GridNet became just 50 percent effective at handwriting recognition. FFTNet’s recognition level remained nearly constant.

Intel’s Silicon Photonics Work Could Supercharge AI Neural Networks

These simulations and prototype work suggest that optical neural networks could be valid alternatives to silicon-based designs. Wierzynski writes:

Larger circuits will require more devices, such as MZIs, per chip. Therefore, attempting to “fine tune” each device on a chip post-manufacturing will be a growing challenge. A more scalable strategy will be to train ONNs in software, then mass produce circuits based on those parameters. Our results suggest that choosing the right architecture in advance can greatly increase the probability that the resulting circuits will achieve their desired performance even in the face of manufacturing variations.

Being able to build effective ONNs even in the face of manufacturing variations means it is much easier to build them effectively while learning how to build them in the first place. That kind of capability could help with commercialization if optical architectures can scale up and compete with conventional silicon work.

Continue reading

People Are Using a Neural Network App to Create Fake Celebrity Porn

Machine learning has become so advanced that a handful of developers have created a tool called FakeApp that can create convincing "face swap" videos. And of course, they're using it to make porn.

MIT Neural Network Processor Cuts Power Consumption by 95 Percent

MIT has developed new neural network processing methods that could cut the power consumption of existing solutions by up to 95 percent. It's a sea change that could effectively reinvent the nature of AI — and where such workloads are performed.

Google Neural Network Can Isolate Individual Voices in Videos

In a new feat of machine learning magic, Google Research has developed a system that can replicate the "cocktail party effect," where your brain focuses on a single audio source in a crowded room.

IBM Aims To Reduce Power Needed For Neural Net Training By 100x

Custom silicon for speeding up AI inferencing is here, but IBM wants to go further and use a hybrid computing architecture and elements of resistive computing to also improve the efficiency of training neural networks.