Closing the AI bandwidth gap with co-packaged optics

As frontier AI models push data center infrastructure to its limits, co-packaged optics is emerging as the packaging and photonics engineering response to a bandwidth challenge copper can’t solve.

BY Dr. Himani Suhag Kamineni, Director, Advanced Packaging Lab (APL), GF Labs; Distinguished Member of Technical Staff (DMTS)

AI infrastructure is hitting a bandwidth wall as workloads drive explosive growth in compute demand. Large language models now reach into trillions of parameters and the IEA reporting annual data center power consumption is projected to reach approximately 950 TWh by 2030, roughly doubling today’s levels. The speed that data moves between processors, memory and the network determines whether that compute is useful as AI workloads scale. The copper interconnects that have long linked processors to the network can’t keep up.

As data rates rise, copper faces physical limits in reach, bandwidth density and energy efficiency. Copper-based interconnects can operate reliably over distances under two meters, but the AI clusters that modern workloads demand, where thousands of processors function as a coordinated system, can no longer operate under those constraints.

Co-packaged optics (CPO) create a fundamental shift in how data moves to alleviate the demand from AI. By integrating photonic and electronic components at the package level, CPO enables terabit-scale optical I/O that raises GPU utilization and reduces energy consumption without sacrificing efficiency.

When copper interconnects hit the bandwidth ceiling
Traditional data center architectures have switches inside each server rack, connected to processors by short copper cables. That works when servers operate mostly independently, but AI clusters require thousands of processors working in constant coordination, which pushes switches further from the servers. The distance between servers and switches increases dramatically, so when high-speed data transfer is required, copper becomes impractical.

Optical networking extends interconnect reach from board-level distances to rack- and datacenter-scale (>10 meters to 100+ meters), while delivering bandwidth density on the order of ~1 Tb/s per millimeter in advanced co-packaged optics solutions. The energy benefits are measurable, reducing interconnect power from ~15 picojoules (pJ) per bit (pluggables) to ~5 pJ per bit today, with roadmaps targeting under 1 pJ per bit with CPO.

CPO redefines the AI interconnect stack
CPO moves the engine from the front panel into the package itself. In the architecture, a photonic integrated circuit (PIC) and electronic integrated circuit (EIC) are co-integrated on a shared substrate.

Electrical signal paths that previously ran centimeters across a circuit board are reduced to hundreds of microns inside the package. The reduction in signal travel distance directly translates to lower power consumption, higher bandwidth density and lower latency. Vertical component stacking enables compact, high-density system architecture that could not be achieved with traditional pluggable module designs.

CPO delivers system-level gains across performance, power, size and integration. CPO packaging enables designers to mix-and-match the best component for each product and integrate them in a single package, even if they were designed in different manufacturing processes.

Scaling CPO to production requires rethinking the packaging design, manufacturing process and the test strategy simultaneously.

Silicon photonics as the future of data center connectivity
Silicon photonics provides the manufacturing foundation CPO needs to scale. By building photonic devices and CMOS logic together on 300mm wafers, the industry can produce optical components at the massive volumes AI data center demand requires.

The industry is advancing through generations of silicon photonics capability from 100G to 200G to 400G per wavelength. Each generation requires co-development of the packaging features and photonic devices to connect them electrically, including copper pillars and the fiber interface components that connect them optically.

Increased bandwidth doesn’t have to result in sacrificing power efficiency. Each new generation of CPO improves efficiency by shortening electrical paths, improving the components that convert electrical signals to optical ones and reducing signal losses at each connection point.

Beyond 400G per wavelength, silicon alone hits its own physical limits. The next generation of silicon photonics platforms is incorporating new optical materials that can carry more data per wavelength, with a goal to reach multi-terabit per wavelength operation.

Advanced packaging extends integration beyond photonics
The packaging techniques, including 3D heterogeneous integration (3DHI), that make CPO work are not limited to photonics. Wafer-to-wafer (W2W) and die-to-wafer (D2W) bonding are uniquely positioned to solve data center problems, but the same capabilities apply across a much wider range of products.

In the data center alone, the applications extend beyond the optical transceiver. The bonding techniques that connect photonic and electronic ICs can integrate amplifier and driver circuits directly with silicon photonics, cutting the signal losses that grow with every additional interface at high data rates.

Outside the data center, the applications look entirely different. Bonding micro-LED structures to CMOS logic opens new paths for display integration. Applying this to power devices and image sensors is enabling next-generation platforms in automotive, industrial and consumer applications.

The advanced packaging capabilities built to meet the most demanding requirements in AI data center connectivity are proving valuable as they are applied to adjacent markets. Investing in the packaging building blocks for CPO is an investment in a broader advanced packaging platform.

Integrated CPO manufacturing demands end-to-end process control
Scaling co-packaged optics from lab demonstration to high-volume production introduces significant manufacturing challenges. Whereas chips are single devices made in a single process, CPO is a system built across manufacturing stages from photonic wafer fabrication, optical I/O integration, wafer-level optical testing, EIC and PIC assembly and final module test.

Optical testing introduces a new level of complexity. At the wafer level, testing requires coupling light in and out of devices at scale, with the alignment precision and throughput that high-volume manufacturing demands.

The dependency between stages has a direct implication for how CPO gets manufactured. Splitting the flow across multiple suppliers creates gaps in yield accountability that are difficult to close at data center volumes. Knowing where yield is lost and being able to fix it requires visibility across the entire chain.

The defining packaging challenge
AI scale-up networks are growing faster than copper-based architectures can accommodate, and the data center infrastructure industry is responding with a generational shift in how processors connect to the network. Co-packaged optics is at the center of that shift, converging silicon photonics process capability, advanced packaging techniques, heterogeneous integration and manufacturing infrastructure.

As these systems grow, the engineering decisions made at the package level increasingly determine system-level performance. Closing the bandwidth gap that AI infrastructure faces is the defining packaging challenge of this moment, and co-packaged optics is how the industry can meet it head on.