Neuromorphic Chips to End the AI Power Crisis

Large language models (LLMs) must frequently access memory every time they generate a word. Professor Chung Dae-sung explains that the physical distance in the von Neumann architecture, where computation and storage are separate, causes data bottlenecks and power waste.

AI large language models face a structural limitation in that they must frequently access memory every time a word is generated.

The physical distance between the CPU and memory is very far in the electronic world, leading to data bottlenecks and power waste.

Major semiconductor companies, including Samsung Electronics and SK Hynix, are currently developing Processing-In-Memory (PIM) technology.

This technology aims to improve power efficiency by minimizing the physical distance between the CPU and memory by embedding minimal computing capabilities within the memory chip.

Professor Chung added that this difference in workload is incomparable.

Professor Chung emphasized that the frequency of memory access during AI language generation is incomparable to that of games.

Games primarily operate with the CPU performing calculations, which fundamentally differs from AI in data processing methods.

The core of current research is to physically emulate the brain structure at the hardware level.

Current neuromorphic technology aims to physically emulate the human brain structure at the hardware level.

Although artificial neural network software resembles the human brain, the hardware still maintained the von Neumann architecture, leading to significant data bottlenecks.

Structural mismatch is a limitation of 2nd-generation neuromorphic chips. Professor Chung pointed out that a transition to hardware resembling the human brain is essential.

Currently, 2nd-generation neuromorphic chips require combining dozens of silicon transistors to mimic a single synapse.

The need for a chip the size of a house to implement the brain's more than 100 trillion synapses poses a significant challenge in terms of physical space efficiency.

The core of 3rd-generation neuromorphic chips is 1:1 matching technology, where one transistor performs the function of one synapse.

This completely solves the physical space problem of existing chips, and electrochemical transistors are being researched as a representative technology.

Electrochemical transistors can dramatically increase the structural efficiency of neurons and synapses due to changes in their electrical properties based on chemical ion concentration.

To implement functions like synapses, where electrical properties change with variations in the surrounding chemical environment, the introduction of organic material technology is essential.

However, organic semiconductors are vulnerable to moisture and oxygen, resulting in poor stability, which is considered the biggest challenge in academia.

Currently, research to improve organic material stability is underway based on existing examples such as OLEDs, and alternative materials like oxides and quantum dots are also being studied.

These stability issues remain a key challenge for commercialization.

2nd-generation neuromorphic involves parallel placement of chips with both computation and storage functions, and it is expected that PIM and neuromorphic will eventually share the same goal as they evolve to the analog chip stage.

Prototypes of 2nd-generation neuromorphic chips like Intel's 'Loihi' and IBM's 'Northpole' have already been developed, but a major reason they haven't been widely adopted in daily life is the absence of a software ecosystem.

Due to development environments accustomed to the von Neumann architecture, the development of neuromorphic-specific software is slow. Professor Chung stated that market formation and developer participation are crucial for commercialization.

Unlike von Neumann computers that constantly run and waste power, neuromorphic chips only activate pixels in the detected area when an event is sensed to acquire and compute information.

This function is considered an optimal alternative for low-power environments such as CCTV, which requires 24-hour monitoring, or battery-powered IoT devices.

Current wearable devices, such as wearable glasses and watches, suffer from short battery life due to the constant operation of GPUs.

Converting sensing functions like heart rate and body temperature measurement to neuromorphic chips can dramatically improve power consumption.

Following industrial machine vision, the wearable market is expected to emerge as a major application area for neuromorphic chips.

According to Professor Chung Dae-sung, 1st-generation neuromorphic technology, like voice recognition functions in smartphones, is already commercialized and integrated.

2nd-generation neuromorphic technology is in the testing phase by large corporations for applications like machine vision and autonomous driving assistance, while 3rd-generation technology, capable of processing even large language models (LLMs), is currently in the research stage.

As on-device decision-making becomes more important than central server communication, existing sensor companies will pursue chip development directly, and neuromorphic chip development companies will also concurrently develop sensors, leading to an integration trend.

Companies with both patents and silicon fabrication capabilities are expected to accelerate their entry into the neuromorphic market.

Once 3rd-generation neuromorphic chips, capable of immediate decision-making without central server communication, are completed, LLMs can be run on the device itself. Professor Chung predicted that this technology would eliminate dependence on HBM and GPUs and replace all AI application environments.

Professor Chung Dae-sung predicted that such technological advancements would usher in an era where devices the size of a smartphone can perform LLM functions without central server communication.

This signifies the pinnacle of edge computing and is expected to bring revolutionary changes to all AI application fields.

It is true that Intel and IBM in the United States are leading 2nd-generation neuromorphic technology.

However, in the development of 3rd-generation neuromorphic chips, China is moving ahead at an overwhelming pace.

Professor Chung assessed that China has an optimal environment for securing cutting-edge technology, with concentrated national-led massive capital investment.