Neuromorphic devices & systems

Technologies for computing tomorrow’s AI

The prospect of analog computing

To compute and train neuromorphic and AI algorithms, digital processors (CPUs, GPUs, TPUs, FPGAs and ASICs) are used almost exclusively today. One promising alternative to the large, costly and power-hungry digital logic is analog computing, where computationally expensive operations are offloaded to specialized accelerators comprising analog elements with the promise to accelerate existing schemes by factors of 1000 to 10,000.

Suitable compute elements are programmable resistive or optical devices that can be arranged to perform various mathematical operations. The main requirements for such architectures are vector-matrix multiplication and the ability to provide the transposed matrix for learning as well as a means to store analog synaptic weights. This mitigates the huge communication overhead for the operands in traditional systems.

Electrical crossbar devices & circuits

Electrical resistive devices arranged as crossbars constitute extremely powerful vector-matrix multipliers and, with additional memory functions, can directly be used as synaptic weights in ANN/CNNs. Thus they help to solve one of the most frequent and most costly operations in typical AI algorithms. These devices require that their programmed resistivity be maintained, which makes them memristive devices. They also require linearity in the transfer and symmetry of their memory and programming characteristics.

At IBM Research – Zurich, we are pushing the state-of-the-art of such devices and circuits based on phase-change and ferroelectric effects and with filamentary oxide devices.

Devices

Optical computing

 

Although the fundamentals of optical computing were demonstrated at the end of the previous century, it is only with modern silicon integrated optical circuits that optical compute densities gained the prospect of becoming competitive with electronic computing.

At IBM Research – Zurich, we are building a fully optical, integrated compute engine that exploits 2-D interference patterns in thin III/V films on silicon photonics circuits. This facilitates single-shot vector-matrix multiplication, comparable to electrical crossbars and featuring very high linearity, high dynamic range and matrix weights that are optically stored within the holographic interference layer. The entire optical accelerator technology will be packaged and thus provide standard electrical interfaces to integrate and interface seamlessly with other AI system components.

Oscillating neural networks

Rhythmic patterns are ubiquitous in neural systems. Oscillatory activity known as “brain waves” play a major role in motor, sensory, and even cognitive functioning. In an oscillatory network the neurons may fire at different times but with the same period [More]. An important property is stability to perturbations. Even if the oscillation is briefly perturbed, it will return to its original magnitude and period, albeit with a possible phase shift [More]. This phase shift is quite important, and it is assumed that the information flow in the network is encoded in the pattern of timings of the individual elements.

We are developing oscillating neural networks based on electronic relaxation oscillators. We fabricate nanoscale electrical switches that exploit the metal-insulator transition of vanadium dioxide (VO2). Electronic circuits are created using the capabilities of the Binnig and Rohrer Nanotechnology Center. We perform device and circuit testing and simulation targeting applications in the field of neuromorphic computing and machine learning.

Computational Architectures Based on Coupled Oscillators

M.J. Cotter et al.
In Proc. IEEE Computer Society Annual Symposium on VLSI, pp. 130–135, 2014.

Computing with Oscillators

E.M. Izhikevich
Neural Networks 5255 1‐30, 2000.

Neural network using oscillators

T. Endo and K. Takeyama
Electron. Comm. Jpn. Pt. III 75, 51‐59, 1992.

oscillating neural networks
oscillating neural networks

 

 

 

Ask the experts

Bert Jan Offrein

Bert Jan Offrein
IBM Research scientist

Siegfried Karg

Siegfried Karg
IBM Research scientist

Folkert Horst
IBM Research scientist

Elisabetta Corti

Elisabetta Corti
PhD student

EU project

ULPEC logo

ULPEC
For an Ultra-Low Power Event-Based Camera