Minutes of CALICE Electronics Meeting, RAL, 06/12/02 ==================================================== Present: Adam Baird, Ivan Church, Paul Dauncey, Rob Halsall, Steve Hillier, Dave Mercer, Dave Price, Matthew Warren, Osman Zorba Minutes: Paul Next meeting: We should move to regular meetings every two weeks. Tuesday afternoons were chosen as best in general. However, Tue 17 Dec is not possible for the UCL people, so the next meeting is Mon 16 Dec at UCL. In general, these could vary between face-to-face or phone meetings. Outstanding questions: Rob presented a list of questions; 1. Can we make the Detector Front End LVDS Positive rather than Negative? 2. Can we make the Detector Front End Connector VHDCI SCSI or exisiting? 3. Is the Madison Universal SCSII cable ie UTP with overall shield ok? 4. Is the layout of Front Panel with 8 x dual VHDCI SCSI ok? 5. Analogue signals - full true differential? swing? termination? 6. Analogue voltage levels to the VFE 7. DAC modularity 8. Can we influence/have a copy of the design of the Detector FE drive circuitry? 9. Continuous clocked ADC implications? 10. Can we Sync readout control to the ADC clock? 11. Can we reduce the FE FPGA size & package to XC2V500/1000 FG456? 12. Can/should we fill up all spares of Connector with LVDS from the FE FPGA? 13. Trigger distribution - fanout, delay & Jitter, inside board, between boards 14. FE-BE FPGA control signals, speed? quantity? operation of? CMS protocol? 15. Trigger logic in BE FPGA, Backplane I/O fanout, connector 16. VME Readout speed 17. FE FPGA Design, new design 18. VME/BE FPGA Design, re-use CMS design? 19. NIM 2 LVDS module, use new RAL module? These were discussed in turn below. 1. Can we make the Detector Front End LVDS Positive rather than Negative? 2. Can we make the Detector Front End Connector VHDCI SCSI or exisiting? 5. Analogue signals - full true differential? swing? termination? 6. Analogue voltage levels to the VFE 8. Can we influence/have a copy of the design of the Detector FE drive circuitry? These all relate to the VFE and its PCB. Paul will connect Adam and Rob to the Orsay engineers (Julien Fleury and Christophe de la Taille) who are responsible for these items. There are travel funds available for Adam and/or Rob to go to Paris to discuss these issues if it would be useful. For 2, it is unlikely they can use the same SCSI connector as their mechanical clearance is only 4.5mm, although it is not clear if that includes the PCB width or not. 3. Is the Madison Universal SCSII cable ie UTP with overall shield ok? This has internal twisted pairs with an overall shield but not individual shields per pair. This would need to be tested to see if the noise performance would be adequate. The samples shown at the meeting indicated the cable is likely to be the widest part of the connector and it close to the 0.8" width of a single-slot VME module. There would be some motivation to have a cable with only the 38 lines needed (rather than the full 68 lines) to keep it narrower. This would imply custom-manufactured cables in any case; 150 pounds/cable (15k total) was put in the budget for this. One other issue is cable safety regulations as Fermilab is likely to be as strict as CERN. Paul will look into this. 4. Is the layout of Front Panel with 8 x dual VHDCI SCSI ok? Adam had laid out the physical components and although tight, 8 connectors seem to fit. The lug holes overlap but the pins which fit into them do not fill the holes, so this should work. 7. DAC modularity I.e. should there be one or two DAC's per FE FPGA? Clearly, two allows different calibration voltages for the two cables and so gives more flexibility and hence is more desirable. However, if there is a critical problem with layout etc, this could be reduced to one. 9. Continuous clocked ADC implications? The ADS8361 does not have an internal clock and so needs a 10MHz external clock for a 2us convert time. The serial data rate out is also on the same clock, so this is a factor 4 slower than assumed previously. Sending out the 16 bits on this clock would take 1.6us. If this was in addition to the ~2.5us for the settling time and convert, then the total sequence of 18 channels would take ~74us. In principle, the ADC can convert the second channel while the data for the first is clocked out, so this would reduce the total time to 45us. However, the discussion during the CDR raised this simultaneous convert/readout as potentially adding noise and the conclusion there was that it should be avoided. As long as all the control lines for the ADC are directly to/from the FE FPGA, then both methods could be tried and the extra noise measured. 10. Can we Sync readout control to the ADC clock? A board clock of around 40MHz seems reasonable, in which case the 10MHz for the ADC would be a simple factor 4 reduction of this. Hence, the ADC would be synchronised automatically to the rest of the readout sequence. 11. Can we reduce the FE FPGA size & package to XC2V500/1000 FG456? The CMS FED will use the FG676, which will be ~200 pounds. The FG456 has less I/O pins and is ~120 pounds but fits into the same footprint. Unfortunately, the CMS design uses some of the pins on the FG676 which are not present on the FG456 so using the FG456 would require some relayout of this area. The FG456 has 192 I/O pins which may be close to the number required if we track all unused cable lines to the FE FPGA (item 12 below). A careful count of the required I/O pins is needed and will come out of the schematics from Adam when complete. The total saving in pounds would be 80 x 8 = 640 per board and so 640 x 11 = 7k overall. The other issue is procurement; the FG676 has a 6-8 week lead time but the time for the FG456 is not known and should be checked. 12. Can/should we fill up all spares of Connector with LVDS from the FE FPGA? There are 30 unused pins in the 68-pin SCSI connector, allowing 15 LVDS. It would clearly give more flexibility to do this. The issue is which direction the signals are going. Having them as bus LVDS would allow each to be in either direction but this would require a 2.5V supply and would mean anyone using these lines would need to use bus LVDS also. An alternative which was thought preferable was to use standard LVDS and include termination resistors in the layout for every line as if they were all inputs. Then, for the output lines, the resistors are physically not mounted, so the decision can be made at a later date, when more information on the use of these lines is known. This would remove the need for a 2.5V supply. 13. Trigger distribution - fanout, delay & Jitter, inside board, between boards The total latency must be below 180ns, so we should aim for 10-20ns within the readout system. The maximum possible jitter is 10ns but again we should aim for substantially less than this. The idea is to distribute the trigger unclocked (asynchronously) throughout the system until it arrives at the FE, where it is clocked and delayed on a fast clock of a few ns period. E.g. this could be 8 times the 40 MHz board clock, i.e. 320 MHz, which has a 3.1ns period. This delay is in the FE FPGA as it allows different delays per cable. Putting it in the BE FPGA would require all cables to have the same delay or to use up all the digital clock managers (DMCs) in the BE for this purpose. Hence, keeping this in the FE was considered best. 14. FE-BE FPGA control signals, speed? quantity? operation of? CMS protocol? This is discussed below. 15. Trigger logic in BE FPGA, Backplane I/O fanout, connector This is discussed below. 16. VME Readout speed At least the initial versions of the VME firmware for the FED will only allow A32/D32 VME transfers. This has a realistic rate of around 20MBytes/s maximum. which would limit the readout to about 800Hz, even if this maximum could be achieved (which will not be trivial). The VME readout speed is much less critical if we can buffer the events on board until the beam spill has finished. However, with 16 cables per board, then each board now handles 16 x 18 x 6 = 1728 channels, which result in around 4kBytes of data per event. Hence, the 2MByte of memory hanging from the BE will only be able to buffer around 500 events, which would not be enough for a 1kHz rate during a spill length of 1 (or 2) seconds. To buffer 2k events would require the memory to be increased to 8MBytes. In the current design, the 2 MBytes is actually two 1MByte components. There are no spare address lines to extend the address range to cover 8MBytes, but by getting one large memory and relaying out the lines to both on the current memories, there would be sufficient connectivity. This will require quite a bit of relayout. 17. FE FPGA Design, new design This is discussed below. 18. VME/BE FPGA Design, re-use CMS design? This is discussed below. 19. NIM 2 LVDS module, use new RAL module? The new RAL module is a NIM unit with 16 signals in and 16 out. The NIM I/O is LEMO and the LVDS uses a SCSI connector. The cost is less than 1k. The trigger I/O we need is likely to fit into this number of signals. Hence, assuming the trigger logic is done in NIM (which seems most likely) then using this would obviously save us having to develop something equivalent. Reuse of CMS firmware: Rob described the BE functions of the CMS FED. There seemed to be a very close analogy with the functions required by the CALICE readout board so it seems likely that a lot of the firmware can be reused. There are some differences. Clearly, all the SLINK code will not be needed and the VME interface may need more than the 1kByte dual-ported RAM and 1kByte FIFO. In addition, adding a larger, single-component, memory will require some changes. The FE firmware is, however, very different, although some of the FE-BE interface handling might be reusable. FE-BE interface: Rob also described the data paths between the FE and BE. For configuration data, CMS use a single serial line for each direction. The BE treats this blindly by just sending whatever is loaded into the appropriate buffer via VME and saving whatever comes down the return line until read by VME. The FE is responsible for continually scanning the line from the BE for the start of a command packet and then interpreting this. Rob will circulate a description of the command protocol for this line. While the CALICE version will need different commands, the basic protocol should clearly cover any configuration requirements we have. The only apparent limit might be the 2kByte buffers if large amounts of configuration data were required. For event data, the FE's send a ready command together with the data length when read for readout. When all of these ready commands have been received, the BE then sends a start to synchronously start the FE's transmitting the data over 4 lines at a high rate. This is very similar to Paul's straw-man proposal in the minutes from the meeting on 06/12/02, except the length is sent with the ready instead of as the first word of data. This will therefore clearly also satisfy our requirements. Trigger logic in the BE: Paul showed a diagram of the logical functions of the trigger which needs to be embedded in the BE FPGA. The trigger I/O needs to go through the J0 and J2 connectors. There are 14 lines tracked from the BE to J0 and 90 to J2. Of these, only 8, i.e. 4 pairs, have termination resistors and so could be used as inputs. The trigger needs 16 in and 16 out (to match the NIM-LVDS module mentioned above) which will take up 64 lines on J2. These correspond to the I/O on Matt's board in the CDR. In addition, the trigger distribution to the other readout boards needs to be done and this could go through J0. Here, there needs to be one input (common to all boards) for the trigger itself. Also, there needs to be trigger outputs to be distributed across the backplane to these inputs. There is clearly a need for around 17 inputs, not 4, so extra termination resistors will need to be added to these lines. An alternative would be to use a non-LVDS FPGA-FPGA protocol for these lines. One issue is that three separate VME interupts are needed from the trigger, for a trigger itself and also for the beam spill turning on and off. It was not clear if these could be handled in the VME FPGA as it stands and this needs to be checked. Layout and schematics: Adam showed slides of the status of the layout and schematics. There appears to be enough room, certainly if the VFE-PCB uses positive voltages and so does not require level converters. The first draft of schematics are also complete, with no showstoppers found. Schedule: Having been approved, we are now working to the schedule outlined in the last meeting. For the next six months, this means schematic design should be complete by the end of Jan. The drawing office then does layout during Feb and Mar and two prototype boards are manufactured during Apr. To complete the schematics by the end of Jan, then the pin layout of the FE FPGA needs to be fixed. To do this will need a block-level FE design to define groupings of pins to signals; it was thought that any reasonable pin-to-signal designation within this could be accomodated within the FPGA. In addition, the component choices for the FPGA, ADC and DAC will be needed and also the VFE-PCB cable signal specifications must be fixed. The preliminary design review (PDR, also called IDR) should happen just before spending money on PCB production, which means towards the end of Mar. However, most of the components for the two prototype boards will need to be purchased before then, given the lead times of the parts. In parallel, the detailed FPGA designs need to converge by the start of May so the two prototypes can be tested. Also, the VME crate, PCI-VME interface and a PC will need to be available by the end of May. CMS are ordering crates within a short time and so one for CALICE should be added to that order.