Artificial brains vs. von Neumann bottlenecks – AGH UST researchers work on synthetic intelligence

Von Neumann bottleneck

Modern computers are made of a CPU (Central Processing Unit) and memory, between which data travel. This system is based on the von Neumann architecture, named so after a prominent Hungarian mathematician and logician of Jewish origin who created the theoretical foundations of computing. Due to the operational separation of the two components, a computer can be used to perform various tasks with respect to the program currently running. However, this solution also has one fundamental drawback: the data bandwidth between the two units is limited, which makes the CPU inert before it receives information. To make matters worse, the throughput remains at a similar level despite the fact that the processing speed and memory capacity increase.

The problem described above is referred to in the literature as a von Neumann bottleneck. Interestingly, this is something that brains are not subject to, and they can be treated as computing machines. The reason for this is cells that combine the functions of memory and a CPU. Research on brain pathways has shown that synapses, which we have evolution to thank for, can store information and process them at the same time. It happens because they memorise the flow of electrons; therefore, they operate on the basis of local memory, which does not rest on classic but on fuzzy or many-valued logic. The question boils down to how to mimic this clever natural contraption in an artificial environment and how to create a new-generation computer that everyone would be able to use at home.

‘It’s actually difficult to separate the processing and memorising procedures in our brains. A new-generation computer would have to mimic the nervous system so that we could get rid of the von Neumann bottleneck and wouldn’t have to constantly transmit information back and forth between the CPU and memory, which consumes a considerable amount of energy of the device. Should transistors be capable of storing the data for a moment, this wouldn’t be an issue. Another desired quality of computer technology might also be the lack of synchronisation, just as in the brain, which allows it to carry out tasks in parallel. Because the nervous system doesn’t have an internal clock, its elements can, to a certain degree, operate independently of each other’, says Professor Konrad Szaciłowski drawing his inspiration from nature.

Artificial equivalents of synapses: memristors

A propect for another huge leap in computer technology is the so-called memristor (a portmanteau of memory resistor), which can be seen as an artificial counterpart of a synapse. It completes the quartet of fundamental electrical components which also comprises the resistor, capacitor, and inductor. However, its distinguishing feature is that it can be used to store information. This is so because its resistance increases when electrons travel in one direction and decreases when they go in the opposite way. Moreover, each memristor remembers its state and retains its most recent resistance even when the power is off. This makes it a perfect candidate for a component that can one day serve as a building block for a network which will mimic the functioning of the brain.

‘A memristor is a non-linear electrical component that solves a particular equation. It has properties similar to a synapse in the nervous system; therefore, it is often called an artificial synapse. It’s because you can force it to retain a kind of primitive memory, namely, it will remember whether an electrical impulse has flown through it or not. Whereas an artificial neuron is the usual name given to an electrochemical cell that generates electrical impulses under the influence of light. The case is now to merge these two different elements to create a completely synthetic system that will process data analogously to the processing that occurs in a biological nervous system’, says the leader of the project, explaining potential uses of the device.

The existence of a memristor, albeit in theory, was forseen by an American scientist Leon Chua in the 1970s (AGH UST Doctor Honoris Causa). He is deemed a visionary in the field of electronics. Initially, a memory resistor did not attract much attention and was seen merely as a curio. This changed at the turn of the century when others saw it as a potential candidate for an artificial synapse. Due to local memory, memristors are capable of mimicking the functioning of the brain whose particular synapses can operate in parallel. This removes the need to exchange data between the main memory and the CPU, which limits modern computers. Unfortunately, building such a device is not as easy as one may think.

 

Trials and tribulations with synthetic intelligence

Today, the construction of a memristor remains a challenge for engineers. All this because the materials needed to make them are highly vulnerable to environmental and external conditions. Therefore, scientists from all over the world are on the lookout for innovative solutions to overcome this problem. AGH UST engineers have also been involved in the search and have carried out intense research on memristive materials. Bringing specialised knowledge from the borderline of physics, chemistry, and materials engineering to the table, the AGH UST ACMiN researchers try to build a memory resistor that will be able to operate under home conditions. Having received funding from a university grant, they construct systems made of nanoparticles and light-sensitive post-perovskite materials.

‘Memristors based on perovskites have been studied for 6 to 8 years. In this specific case, we’re talking about the so-called lead perovskites, that is, complex compounds containing lead, iodine, and an organic cation. These compounds show good electrical conductivity and are commonly applied in solar cells. However, perovskite memristors have one basic flaw: they are extremely air- and humidity-sensitive. This is why we look for entirely new materials that will have properties similar to those of perovskites, i.e. show the so-called memristance but, at the same time, will not be so sensitive to environmental factors. It’s because we need memristors that we’ll be able to mass produce on a larger scale in the future’, says Professor Szaciłowski, describing the goal of the project.

 

 

Working notes in the so-called scientists’ kitchen; photo from AGH UST ACMiN scientists’ private archives

Currently, the AGH UST engineers are trying to build a memory resistor that will operate stably in the laboratory so that future students might experiment with it someday. When scientists finally manage to develop individual components, they can begin constructing a neural network capable of performing specific tasks. Such a system would be a material equivalent of the nervous system, mimicking the activity of synapses and neurons. When discussing artificial neural networks, the focus often turns to a digital model of the brain. However, in this case, it is more about synthetic intelligence that could be used to build hardware.

‘What we’re trying to build in our laboratory is not so much artificial neural networks, but rather synthetic neural networks because they actually exist physically. It’s not software, which has no tangible representation, but elements that have properties very similar to neurons. Based on that, we’re planning to create networks in the future which will not be merely digital simulations, and then feed them problems to solve. In other words, we want to make a bucketful of jelly with which we can play chess. Here, we’ve managed to use this perovskite structure as both a neuron and a synapse. Thus, at the level of an individual element, we combined information processing with memory, which means that we have somehow defeated the von Neumann bottleneck’, claims the project leader.

The project was funded by a university grant within the framework of the “Initiative for Excellence – Research University” project (the AGH UST 2020–2022, PRA-5).