Elements: LHCb Upgrade II: Tackling the 200 Tb/s challenge

Massachusetts Institute Of Technology, Cambridge MA

Investigators

Abstract

The Standard Model of particle physics describes all known fundamental forces except gravity. It has proven capable of describing physical phenomenon to remarkable precision. However, it fails to describe all aspects of our universe, indicating there must be other forces beyond those that we currently know about. The LHCb experiment is one of the four main experiments at the Large Hadron Collider (LHC), CERN. It specializes in detecting differences between matter and anti-matter, using a heavy type of quark called a beauty quark. Billions of quarks are produced every second in collisions at LHC, which in turn decay to other particles, producing a so-called event in our detector. These events result in a data rate of around 40 Tb/s under current data-taking conditions. Not all this data can be saved, and the rate must be reduced in real-time by more than a factor ten. We call the algorithms used in real-time selection of events the "trigger". The LHCb Upgrade II experiment will run in the 2030s and will deal with a data rate of 200 Tb/s, the largest at the LHC. This rate has to be reduced by four orders of magnitude before being written out. To achieve this, the electronics present on or near the sub-detectors, called Field Programmable Gate Arrays (FPGAs), must be leveraged. Ideally machine learning (ML) algorithms would be employed on the FPGAs as these algorithms give improved performance. This is currently difficult for the LHCb experiment as the community-standard package that interfaces trained ML models to FPGA backends, hls4ml, has limited functionality for the two most prolific FPGA types used in LHCb. This project extends the functionality of hls4ml, significantly expanding the user-base. It also develops the first machine-learning based lossy compression algorithms to run on the electronics infrastructure within LHCb, with a speed on the order of tens of nano seconds, along with other algorithms to run in the LHCb Upgrade II trigger on both FPGAs and GPU architectures. The LHCb Upgrade II experiment will run at the Large Hadron Collider at CERN from 2033-2041 and will supersede the current LHCb Upgrade I (2022-2029). The current LHCb trigger system is unique in particle physics in that the full 40Tb/s of data produced by the experiment is fed into software, where it is processed using a GPU-based architecture which reduces the rate using a real-time event filter (trigger). The data rate of the LHCb Upgrade II experiment will be five times higher than the current rate, at 200 Tb/s. The data rate must be reduced by around four orders of magnitude before being sent to storage, and the large rates mean that data reduction must occur even before the computing farm. Without sophisticated trigger algorithms before the computing farm, many of the detector read outs will simply be thrown away, resulting in a significant loss of physics information. In recent years there has been rapid progress in fast machine learning (ML) algorithms run on FPGA accelerators present in the electronics that sit on or close to the detector readout. Despite the obvious relevance of using ML algorithms on FPGAs to overcome LHCb's 200 Tb/s problem, the collaboration has been slow to profit from this progress. This is in part because the community-standard package that interfaces trained ML models to FPGA backends, hls4ml, has limited functionality for the two most prolific FPGA types used in LHCb, Intel and Microchip. One of the sub-projects on this proposal will extend the scope of hls4ml to include these backends, significantly expanding the user-base of hls4ml. This project will then develop the first low-latency ML-based lossy compression algorithms to run on the electronics infrastructure within LHCb. This will allow the collaboration to assess the validity of fast ML on FPGAs as a solution to the 200 Tb/s problem. This project will also build on the work of the previous CSSI grant, OAC-2004645, and develop and implement reconstruction algorithms to run in the Upgrade II trigger which are tested and developed for both GPU and FPGA backends, allowing LHCb to have the flexibility to choose whichever architecture offers the most cost-effective solution in 2033. This award by the Office of Advanced Cyberinfrastructure is jointly supported by the Division of Physics in the Mathematics and Physical Sciences Directorate. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.