The Compact Muon Solenoid (CMS) is a general purpose detector at the LHC performing searches for new physics and tests of the current best model of Universe, the Standard Model. Huge technological challenges are faced in isolating events of interest from the large amount of background processes and in their subsequent reconstruction. The LHC collides protons at a rate of 40 MHz however rare events, such as those containing the Higgs boson, occur far less frequently. It is infeasible to retain every event as this would require an output data rate of 1 Tb/s, far exceeding current technological capabilities. To ensure that these rare events are extracted from the large volume of background processes, CMS utilises a sophisticated two-tier trigger system, performing a fast event reconstruction before deciding whether to retain the event or lose it forever. The first layer of the trigger system, the Level-1 (L1) Trigger, must be incredibly fast in performing these tasks which much be achieved in millionths of a second and is hence performed by high-performance custom-built electronics.
To increase the sensitivity to searches for new physics and measurements of particle properties, the LHC ceased operation in early 2013 for a two-year shutdown to allow for maintenance and upgrades. This will enable the accelerator to operate with a 14 TeV centre of mass energy, the highest yet achieved, and significantly increase the reach of its experiments. As a consequence a large increase in the number of simultaneous particle collisions, known as pileup, is expected to exceed the capabilities of the current trigger system. It is of utmost importance therefore that the trigger system be upgraded to ensure that CMS retains sensitivity to physics searches such as signatures of supersymmetry and measurements of particle properties including that of the Higgs boson.
The MP7 is the baseline trigger processor platform designed to operate in the challenging conditions that will be experienced in the upgraded LHC beam, performing the L1-triggering for the calorimetry systems. The capabilities of the MP7 will enable the performance of the upgraded system to far exceed that of the current trigger, improving the sensitivity and abilities of the CMS experiment. The power of the MP7 will allow the upgraded trigger system to far exceed the performance of the current trigger with far fewer processing boards. Currently the calorimeter trigger system processors fill a total of 13 telecoms racks, a system comprising entirely of MP7's would provide improved performance whilst only occupying 1/3 of a rack, greatly reducing network complexity and infrastructure requirements. The high-bandwidth and processing power of the MP7 from its powerful Virtex-7 FPGA and optical links will enable the upgraded trigger to exploit for the first time full-granularity information of the calorimetry systems. The high performance Virtex-7 FPGA will also allow for more sophisticated reconstruction algorithms to be employed and at a greater speed, greatly improving the performance of the trigger system and the CMS experiment as a whole.
The design of the MP7 is the product of the development of several generations of processing cards, the sophistication and power of which have greatly improved through each new design. The experience obtained during this research of cutting-edge technologies has enabled the MP7 to become an extremely powerful and flexible processing unit, capable of performing the highly challenging tasks required of the CMS calorimeter trigger.
The first card in the development of the calorimeter trigger processing boards was the Leaf card. Utilising two Xilinx Virtex-2 Pro FPGAs with 36 optical links running at 2 Gbps each, the Lead card performs a fast reconstruction and analysis of data from the CMS detector and is the board used in the current CMS trigger system. The Leaf Card exhibits a reconfigurable design, with the nature of the card being determined solely by the firmware, allowing identical boards to perform separate tasks of the reconstruction and processing of electrons and jets in events. Due to the limitations of processing power and bandwidth available at the time of design, the current trigger system can however only process information of a reduced level of granularity from the detector than could be achieved, with the processing spread over many boards. This presents several problems to the trigger system, firstly the lower granularity of information reduces the effectiveness of the triggering algorithms at selecting events of interest from background events, making the selection of events of interest in high pileup conditions extremely challenging. Secondly, the processing of an event had to be distributed across several boards, with each board having access to a small region of the detector. This restricts the choice of reconstruction algorithms that may be utilised and requires the duplication of data across backplanes to share information across detector boundaries, which is slow and greatly increases the complexity of the trigger system.
With the large increase in pileup expected with the LHC upgrade, the operation of the current trigger system will become severely degraded. In order to combat this problem, the calorimeter trigger upgrade program was initiated to develop the next generation of trigger processor boards to address the short comings of the current system, providing a large increase in triggering ability and power.
Developed as a joint collaboration between Imperial College, Princeton, LANL and CERN, the Matrix card was the first of the μTCA generation processor board designs. Possessing a large Xilinx Virtex-5 FPGA and 32 optical channels running at 3.2 Gbps, the Matrix card has powerful processing capabilities with large I/O bandwidth. The generic processing module design with reconfigurable optical links and firmware make the Matrix card flexible, with the ability to adapt the tasks performed by the board without the redesign of the hardware itself. Due to these qualities, the Matrix card is used in the LLRF control system for the FERMI free electron laser at Trieste and played a major role in the development of the CMS calorimeter trigger upgrade project and its algorithms.
Building on the experiences developed from the design of the Matrix card, the Mini-T5 was designed as a demonstrator board to test the feasibility of the baseline Time-Multiplexed Trigger (TMT) architecture for the CMS calorimeter trigger upgrade. Utilising a Xilinx Virtex-5 FPGA and 32 input and 20 output optical links running at 5 Gbps, the Mini-T5 provides a large optical bandwidth, with up to 160 Gbps input and 100 Gbps output optical I/O capability. The reliability of the Mini-T5 has also been well tested, subjected to an external loop feedback test for 12 hours the Mini-T5 was able to transmit 7×1015 bits without error.
Whilst the Mini-T5 provided a significant improvement in performance over what the current trigger processor boards can achieve, due the extreme data rate expected after the LHC upgrade and optical bandwidth limitations, in a TMT configuration the Mini-T5 could only concentrate 1/4 of the entire detector into a single chip. Although this vastly exceeds the capabilities of the current trigger, to ensure optimal performance of the upgraded trigger system the design of the Mini-T5 was refined. Utilising state-of-the-art optics and FPGA technologies the Mini-T5 design was perfected, leading to the development of the latest processor board in the calorimeter trigger generation, the MP7.
Publications:
- G. Iles et al, A demonstrator for a level-1 trigger system based on MicroTCA technology and 5Gb/s optical links. TWEPP-2010, Aachen, Germany, 20th-24th September 2010.
- G. Iles et al, Trigger R&D for CMS at SLHC. Proc of the Topical Workshop on Electronics for Particle Physics. CERN-2009-006, Paris, France, 21th-25th September 2009.
The experiences of the design of technologies for the CMS trigger system, has lead to the development of the MP7, the baseline candidate trigger processor platform for the CMS calorimeter trigger upgrade. Utilising a state-of-the-art Xilinx Virtex-7 FPGA for processing enables a significant increase in serial-I/O bandwidth over the Mini-T5, enabling the transmission and receiving of data at over 740 Gbps second in each direction. This capability, married with a TMT architecture, will allow for the first time the information of the entire detector to be readout and processed in a single MP7, greatly improving the power of reconstruction algorithms that can be utilised.
The capabilities of the MP7 are predicted to lead to significant improvements in the performance of the CMS calorimeter trigger, resolving the problem of the loss of granularity between processing stages that is experienced in the current trigger system. The large optical bandwidth of the MP7, in conjunction with a TMT architecture, will however enable all of the detector information to be concentrated in a single board, giving greater flexibility and power to the trigger system with no loss of data granularity. The high performance of the Xilinx Virtex-7 will also permit the utilisation of more sophisticated reconstruction algorithms, enabling for the first time event-by-event pileup subtraction which will be vital in the environment of the upgraded LHC. An additional benefit of the flexibility of the MP7 is a redundancy capability, with spare boards being able to be reconfigured to take over processing duties if required, a critical feature for operation at the LHC where trigger downtime would lead to a complete loss of data taking.