Reinforcement learning using optomemristors

Lighting Up Artificial Neural Networks

A team of international scientists have performed difficult machine learning computations using a nano-scale device, named an “optomemristor”.

The chalcogenide thin-film device uses both light and electrical signals to interact and emulate multi-factor biological computations of the mammalian brain while consuming very little energy.

To date, research on hardware for artificial intelligence and machine learning applications has concentrated mainly on developing electronic or photonic synapses and neurons, and combining these to carry out basic forms of neural-type processing.

However, powerful processing mechanisms that exist in real brains – such as reinforcement learning and dendritic computation – and that help us learn new skills and carry out everyday tasks, are more challenging to implement directly in hardware.

This new work, in Nature Communications, helps to fill this missing ‘hardware gap’ via the development of an “optomemristor” device that responds to multiple electronic and photonic inputs simultaneously.

Complex learning and processing is made possible in the mammalian brain by the rich biophysical mechanisms that govern the functionalities of the brain’s neurons and synapses.

One key aspect is multifactorial computation – such as three-factor learning – which allows the brain to efficiently learn using positive and negative reinforcements, for example when playing a sport or navigating a maze. The optomemristor approach facilitates such three-factor learning, in one single device.

Dr. Syed Ghazi Sarwat carried out the optomemristor experiments as a DPhil student at the University of Oxford and is currently at IBM Research Europe, where he teamed up with colleague Dr. Timoleon Moraitis to apply the devices to maze solving. Dr. Sarwat continues “Our research shows a practicable hardware approach to efficiently mimic reinforcement learning, a form of machine learning that we use in the paper to enable an artificial rodent to learn to navigate through a maze!”

Professor Harish Bhaskaran, who led the study at the Department of Materials, University of Oxford adds, “We demonstrate how neural operations that are based on the interaction of multiple signals can be carried out using relatively simple hardware. This is illustrated in our demonstration of a linearly non-separable problem of classification (XOR) that requires multiple layers of conventional artificial neurons for its solution, unlike the brain that uses a single biological neuron.”

“Indeed, by emulating the so-called ‘shunting inhibition’ function of dendrites of biological neurons, we illustrate how our optomemristor can effectively provide a single-neuron solution for difficult computational problems”, continued Dr. Timoleon Moraitis.

The demonstrations are at an early proof-of-concept stage and show promise to tackle some important challenges in machine learning. Some key issues arise when considering the scaling-up of such concepts and integrating them alongside other hardware blocks. The team is nonetheless enthused. “All new concepts have significant risks, but this is a new way of thinking about so-called multifactor computations, and that is exciting,” says Professor David Wright at the University of Exeter.

‘Chalcogenide optomemristors for multi-factor neuromorphic computation’ is published in Nature Communications.

The DOI number for this paper is: doi.org/10.1038/s41467-022-29870

The work was supported by EPSRC grant numbers EP/R001677/1, EP/M015173/1 and EP/J018694/1 and the John Fell Fund 553.

Date: 27 April 2022