Low level gigabit Ethernet analysis for the LHCb computing farm

This note presents the concept of Receive Descriptor Recycling to significantly reduce the performance drop associated with small packet Gigabit Ethernet traffic. High reliability of small-sized transmissions is crucial for correct calibration runs of the LHCb experiment, at the CERN LHC accelerator. Previous work applied to full link load Ethernet traffic, using UDP processes. This work covers more low-level details of the performance problem for small-sized traffic, using more lower-level Ethernet frames, and, with a deeper analysis at the PCI/PCI-X level. Measurements were performed at the LHCb online system, which is to a large extend made up of commodity equipment. Limits and trade-offs are inherent when optimising for small packet traffic. All important aspects in this context are covered. Results gathered show the Ethernet Controller's driver currently is the major bottleneck, preventing the system from reaching maximal Gigabit Ethernet performance. Receive Descriptor Recycling is implemented in the driver, and is shown to successfully remedy this situation.