CAREER: Next-generation Optical I/O with Embedded Equalization for Disaggregated AI Computing

University Of Washington, Seattle WA

Investigators

Abstract

Ever-increasing size and complexity of artificial intelligence (AI) and machine learning (ML) models and datasets, recently reaching over a trillion parameters, have created a vital need for vast parallelization over thousands of processing and memory units. This requires low-latency and ultra-low power multi-Tb/s inter/intra-rack optical I/O as well as chip-to-chip interconnects. Although state-of-the-art inter-chip interconnects are realized with copper-based wirelines, they cannot meet the stringent speed, energy-efficiency, and bandwidth density requirements of emerging AI supercomputers. Silicon photonic transceivers have shown a great promise to address this challenge by ultimately co-packaging optical transceivers with high performance CPU/GPUs. Co-packaged optics for the next-generation AI computing should provide tens of Tb/s aggregate data-rates at sub-pJ/b energy-efficiency and low-latency. Despite recent efforts in developing such transceivers, proposed solutions do not yet satisfy the energy and latency requirements of future inter-chip links. In this work, we aim at solving these challenges for ultra-high data-rates by proposing a new equalization paradigm and a novel system-level architecture to make co-packaged optics “smarter” than being just an electro-optical bridge as has never been imagined before. Outcomes of this project can speed up AI/ML computing hardware by enabling a truly disaggregated computing architecture down to the package level. We also plan to integrate our research methodology and modern topics into the educational curriculum by developing a new course. This course will bridge the gap between the fields of integrated circuits and photonics and will have publicly available materials for dissemination at other institutes. Our other educational plans include outreach activities for college and K-12 students and engaging underrepresented graduate and undergraduate students in this project. The most critical challenge in building energy-efficient optical I/O at high data-rates is the power-hungry equalization circuitry conventionally implemented in the electrical domain. These equalizers can consume over 50% of the total link energy and area. We will overcome this issue by radically transforming the equalization techniques, embedding the necessary equalization functionalities in reconfigurable photonic devices rather than the electronic side for the first time. This approach will be deployed for both the transmitter and receiver sides and can achieve sub-pJ/b overall link efficiency. Moreover, while co-packaged optical I/O is becoming imminent for interconnect and switching networks, we will show that these chips can be “smarter” than being just an electro-optical bridge by adding more system-level capabilities to them in this project. These new capabilities can be implemented on a co-processor unit in co-packaged optical I/O chips. In doing so, not only we can overcome the latency overhead of using optical interconnects for chip-to-chip communications, but also, we will exploit novel system-level architectures that can be unlocked by co-packaged optics. The new disaggregated AI architecture can significantly speed up the compute time for training/interference of large-scale AI/ML applications by reducing communication bottlenecks and providing direct memory access for multi-GPU systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.