Silicon Photonic Neural Networks

Neural networks has attracted a lot of attention in the recent years after a brief slump after the 1970s due to technology constraints. Their biggest advantages are their ability to carryout computationally intensive tasks such as image recognition, voice recognition, solving optimization problems, non-linear programming and classification. There have been great advances in this filed which has introduced various kinds of neural networks such as Hopfield networks, deep neural networks, convolutional neural networks and recursive neural networks. Despite the advances, their true capabilities remain constrained by the hardware available to them, which is in most cases, microprocessors. Microprocessors being based on electron devices such as MOSFETs, HEMTs and FINFETs are still limited to operational speeds of a few gigahertz. Even thought they have scalability superiority over other technologies, enabling them to incorporate neural networks with hundreds of thousands of neurons per layer, some applications require speed over complexity. These applications are usually in the area of high-speed communications, defence and data processing in data centers. The silicon photonic technology is capable of offering tens of gigahertz of frequency with its simplest devices. This enables neural networks to gain a speed advantage by several tenfolds. Silicon photonic devices such as Mach-Zehnder modulators that can operate at 70 GHz with a meagre 25fJ per bit energy consumption have already been demonstrated.

Photonic circuits are a reliable candidate for high-performance neural networks circuits due to its ability to carry out linear mathematical operations and provide outstanding interconnectivity. Matrix multiplication of weight matrices with input vectors lie at the foundation of neural networks. Implementation of such operations can easily be achieved in silicon photonics by using networks of Mach-Zehnder modulators or similar optical devices. Multiplication, inner products and addition are also easily achieved by passive as well as active electro-optic devices. In relation to connectivity, photonic circuits also offer long distance connectivity with low loss and little dissipative losses compared to metal wire connections. Optical waveguides, which are the optical substitute of the the metal wires, have a great advantage over their metal counterparts in terms of maintaining the integrity of signals they carry. They are less prone to parasitic effects and signal distortion. Cross-talk effect are also reduced due to infinitesimal coupling. Another unique feature of neural networks that benefit form the advantages of silicon photonics is their requirement of massively parallel input and output architecture, also know as signal fan-in and fan-out. Electronic connectivity stands tall when it comes to point-to-point connectivity assisted by buffers but they are by design low bandwidth and high-latency solutions. Photonic connections are regarded as the leading alternative that can offer dense fan-in and fan-out without sacrificing either bandwidth or low-latency.

Several demonstrations of photonic neural networks have shown that unitary matrix transformations can be implemented by using an array of beam splitters and phase-shifters. An alternative approach is to use a mesh of Mach-Zehnder Interferometers (MZI). Non-unitary matrix operations have also been demonstrated with a layered approach using meshes of MZIs and tunable attenuators. Yet another approach benefits from the modulation of effective index and employs it for weight implementation. This can either be implemented with free carrier dispersion based devices or devices that integrate other materials in their architecture, such as lithium niobate and graphene.

The activation function required by artificial neurons is easily implemented with the transfer function of several photonic devices such as micro-ring resonators, MZIs, photodiodes, etc. In some cases, a combination of optical and electronic devices are used for introducing the non-linearities. A key strength of an opto-electrooptic O-E-O networks is that the electrical output can be modulated onto an optical carrier, thereby increasing the gain per layer. In fully optical networks, a similar method of actuating output light with input light is used, known as regeneration.

A hardware architecture for ANs in neuromorphic networks typically requires fan-in, fan-out and weight implementation. Neurons with such a structure are implementable in a variety fo topologies which can be anything from feedforward NNs to recurrent networks. The weight implementation in these networks determine which form of training can be carried out. The backpropagation algorithm is widely used for training neural networks. It involves propagating an input in the forward direction and calculating the output error and then re-adjustment of the weights from output to input following the error gradient.

Neural networks can be used to solve complex mathematical problems, solve nonlinear differential equations and carry out nonlinear optimizations. Inhibitory neuron implementations also open up the doors for advanced sensing applications. Photonic reservoir computing is a new field related to neural networks that is also gaining momentum. In such systems, the input Is fed through a layer of random nonlinearities and interconnections and then passed on to a readout layer. The readout layer is trained to adapt the output to match the desired output.