The next data center bottleneck will be solved with 'light'... Silicon photonics boosts computing speed

With the advancement of large-scale AI models, the amount of data moving within data centers has increased exponentially. To cope with this change, new technologies beyond existing electrical signal-based communication methods are required, and silicon photonics is one of them. Silicon photonics has the potential to extend optical communication to chip-to-chip communication, dramatically increasing data bandwidth to an unprecedented level.

As artificial intelligence (AI) technology advances and large language models (LLMs) grow in size, the amount of data required for model training has exploded. It is now argued that simply increasing computational power alone is not enough to ensure the efficiency of AI training. Experts emphasize that how quickly and efficiently data is exchanged between computing devices like GPUs has become a critical factor determining overall system performance.

Even NVIDIA, a global AI semiconductor company, is reportedly yet to fully commercialize optical connections between racks within data centers. Many companies are currently attempting to increase bandwidth by raising the frequency of existing copper wires, but they face the problem of rapidly increasing signal loss as the frequency rises. This physical limitation of copper wires further highlights the need for optical communication technology.

Next-generation AI systems like NVIDIA's NVL72, showcased by CEO Jensen Huang in his keynote address, incorporate internal optical communication chips. In the industry, there's active discussion about whether NVIDIA has already adopted Co-packaged Optics (CPO) level technology. CPO is a technology that integrates optical communication modules into the same package as processor chips, playing a crucial role in achieving high bandwidth and low power consumption. However, it remains unclear whether NVIDIA's current technology level represents true CPO or an earlier form of optical communication integration.

Leading companies like NVIDIA and Marvell are actively seeking to adopt these technologies but encounter various difficulties inherent in integrated packaging. One of the biggest problems is the difficulty of module replacement in terms of serviceability. When optical communication modules are packaged with ASIC chips, individual components cannot be easily replaced, making maintenance complex.

Furthermore, heat generated by ASIC chips can affect adjacent photonics devices, leading to performance degradation. When manufactured together, the production difficulty sharply increases, which is a significant disadvantage in terms of yield. For these reasons, experts suggest a strategic approach focusing on which parts to transition to optical communication first, rather than when all systems will switch to CPO.

Due to these challenges, companies are approaching CPO technology adoption cautiously and are continuously working to secure both technical completeness and economic feasibility.

Traditional optical communication methods transmitted data by modulating light intensity, but such methods make it difficult to achieve terabyte (TB)-level high bandwidth. To solve this problem, researchers devised a method of encoding data by changing the phase of light while keeping the laser constantly on. This leverages the fact that the speed at which a material's refractive index changes when voltage is applied is much faster than the laser's on/off speed.

This phase modulation method is considered the only alternative that can meet the rapidly increasing bandwidth requirements, from gigabytes (GB) to terabytes. Data transmission through refractive index changes enables ultra-high-speed communication and has the potential to dramatically improve data movement bottlenecks within data centers.

Currently, most CMOS processes are optimized for silicon-based manufacturing, making it difficult to create efficient lasers directly from silicon. For this reason, lasers made from III-V compound semiconductors (such as gallium arsenide and indium phosphide) are bonded onto silicon wafers.

However, bonding III-V lasers to silicon wafers accurately and efficiently is extremely challenging. Tiny errors or defects occurring during this process can degrade the performance of the entire device and significantly reduce yield. Major companies like SK Hynix, Samsung Electronics, and Lumentum are focusing on research to ensure the efficiency and high yield of this III-V integration technology. Overcoming this technical hurdle is considered a key challenge for silicon photonics commercialization.

In conclusion, the success of III-V integration technology is expected to be a critical factor determining the growth of the silicon photonics market.

Electrical wiring, like copper wires, requires four separate copper wire channels to send four different pieces of information. This causes spatial constraints and complexity.

However, in photonics, a single optical fiber or waveguide can send light of multiple frequencies simultaneously. Different information is embedded in different frequencies of light, and then at the receiving end, a device called a 'demux' separates the light of each frequency to process individual signals. This is similar to multiple vehicles traveling simultaneously on a single highway towards different destinations.

This Wavelength Division Multiplexing (WDM) technology maximizes the bandwidth efficiency of optical communication and plays a crucial role in solving complex wiring problems within data centers. It is one of the reasons why photonics is gaining attention as a key technology to lead the era of high-bandwidth communication.

The ability to process multiple pieces of information through a single channel greatly contributes to increasing the integration density of data centers and reducing power consumption.

Traditional photonics devices, such as multiplexers (mux), demultiplexers (demux), and power splitters, were very large, on the millimeter (mm) scale. This size posed a significant limitation to increasing the integration density of optical integrated circuits. However, by utilizing an optimization algorithm called 'inverse design,' the size of these devices can be dramatically reduced to the 2-5 micrometer (μm) scale. This means a miniaturization of 50 to up to 1000 times compared to existing methods.

Inverse design algorithms do not rely on human intuitive thinking but rather find the most efficient structures based on physical laws and optimization criteria. For example, if the algorithm is tasked with finding the most efficient structure for separating a specific frequency, it may produce counter-intuitive designs that are difficult for human common sense to grasp. While these designs are theoretically very efficient, their non-intuitive nature necessitates discussion regarding practical application.

While inverse design has significantly reduced the size of photonics devices, it comes with several sacrifices. First, clear design constraints must be provided, and even an optimized result does not guarantee that the design will be precisely implemented during actual fabrication. In particular, intricate structures like 'perforated holes' are difficult to form as intended in real manufacturing processes.

Second, the cost (time and computational resources) of the optimization process is high, and if simulation results do not meet expectations, the inefficiency of having to repeat from scratch exists. Third, there is also the limitation of not being sure if the final design obtained from a specific starting point is indeed the most optimal design. These problems suggest that inverse design must consider not only miniaturization but also manufacturability and optimization efficiency.

High-performance computing semiconductors, such as GPUs and CPUs, have shown rapid development, with integration density increasing threefold every 1-2 years according to Moore's Law. However, the integration density of photonics devices has not kept pace with this speed. This is because photonics circuit units are still large, on the centimeter or millimeter scale, thus encountering physical integration limits.

To solve this integration density problem, there is a growing recognition that optimization techniques like 'inverse design' are essential. As mentioned earlier, inverse design has the potential to dramatically reduce device size, enabling miniaturization down to the millimeter scale. However, this technology also inherently has several limitations, such as issues with productivity, yield, optimization cost, and ensuring the optimality of the final design.

Therefore, researchers are continuing inverse design research to increase photonics integration density, while also making additional efforts to solve the problems associated with this technology.

The inefficiency of existing inverse design workflows in photonics device design is cited as a major factor slowing down technological progress. Typically, designers generate multiple candidate designs based on simulations from random noise or intermediate points. For example, let's assume 10 design candidates were created, and after actual fabrication and testing, 8 performed poorly and only 2 performed well.

The problem is that the information about the 8 poorly performing designs is simply discarded. No post-processing analysis is conducted to understand why their performance was poor, and only the 2 well-performing designs are reused. In other words, without a mechanism to learn from and reuse 'good and bad design' information, inefficiency arises because simulations must be run from scratch every time.

This approach leads to enormous waste of time and resources, acting as a fundamental reason for the slow progress in the photonics design field. Experts commonly agree on the urgent need for efficient design information reuse and the establishment of a learning system.

To overcome the inefficiencies of existing photonics design workflows, a new approach utilizing artificial intelligence (AI) models has been proposed. The research team stated that AI models can theoretically generate 10,000 photonics designs in just one second, suggesting the possibility of revolutionizing existing design methods. This significantly shortens the working time that would take weeks with traditional methods.

Another strength of this AI-based design method is 'manufacturing friendliness.' Since the research team trained the AI model only with design data that showed excellent performance, the designs generated by the model tend to be in a form that is easy to produce in actual manufacturing processes. This is significant because it not only speeds up design but also resolves bottlenecks in the actual manufacturing stage.

Therefore, this AI model demonstrates the potential to simultaneously solve two key challenges in the development of photonics devices: design speed and manufacturing yield. This is expected to play a crucial role in accelerating the commercialization of next-generation optical communication technology.

Such technological advancements are also expected to contribute to the mass production and cost reduction of photonics devices.

Inconsistencies between design and actual results during the photonics device manufacturing process are a major cause of performance degradation. The fab equipment used in the University of Michigan lab is theoretically known to be able to precisely pattern down to 20 nanometers (nm). However, in actual manufacturing environments, there is a problem that such fine patterns are not perfectly reproduced.

For example, even if a 20nm hole is designed, it can be irregularly formed as 25nm or 15nm on the actual wafer due to voltage changes or other process variables. Such discrepancies between designed and actually manufactured patterns inevitably alter the intended optical characteristics of the device and degrade final performance. This irregularity in the manufacturing process acts as a major bottleneck hindering the commercialization of optical integrated circuits.

Therefore, even with a perfect design at the design stage, it is difficult to mass-produce high-performance devices if the precision and reproducibility of the manufacturing process are not secured, which is the reality in the field. Developing precise process control technology and design technology that predicts and corrects manufacturing variations is essential to solve this problem.

The research team is conducting two key studies to support this hypothesis.

This feedback loop aims to continuously improve process efficiency and yield by iteratively repeating the design-manufacturing-measurement-analysis-design improvement cycle in a data-driven manner. If this system functions properly, it is expected to significantly shorten the time required for the commercialization of photonics processes.

Despite the growing attention to silicon photonics as a next-generation technology, the number of institutions specializing in its research is very limited. In Korea, only about 10 laboratories conduct related research, and in the United States, at most 30 laboratories focus on this field. Furthermore, very few of them possess their own fab facilities to handle everything from research to manufacturing.

Moreover, research laboratories integrating artificial intelligence (AI) and machine learning (ML) methodologies to develop design frameworks for silicon photonics are extremely rare. Currently, this field is in the preliminary research stage, and AI/ML-based innovative approaches are still in their early stages. This suggests that the silicon photonics field has immense growth potential and is poised for explosive growth through integration with AI.

The ongoing photonics research is characterized by an unusual interdisciplinary convergence. Originally, a researcher who had been conducting conventional photonics research coincidentally engaged in a conversation about 'inverse design.' During this discussion, they encountered non-intuitive photonics designs, like Swiss cheese with holes, which unexpectedly sparked a new idea.

This idea was to combine protein research with photonics technology. Merging seemingly unrelated fields like protein and optical technology is difficult to grasp by conventional wisdom. However, this 'absurd' combination has surprisingly begun to yield positive results, even astonishing the research team. This peculiar integration suggests new possibilities beyond the limitations of existing photonics research.

Up to now, AI models like Integrated Gradients and cVAE have been used as tools for photonics design.

However, the research team is going a step further, exploring ways to entrust the entire optical circuit design process to AI agents, moving beyond traditional design methods that rely on human intuition. Researchers are keenly interested in providing the AI agents with frameworks they developed, such as Integrated Gradients and cVAE, as tools, and then instructing the agents to create optical circuits autonomously.

This suggests that AI can move beyond merely generating designs to act as autonomous agents that find solutions to complex design problems themselves. AI agent-based optical integrated circuit design is expected to be a revolutionary turning point in the future development of photonics technology.