Comparing Ethernet and Myrinet for MPI communication

This paper compares the performance of Myrinet and Ethernet as a communication substrate for MPI libraries. MPI library implementations for Myrinet utilize user-level communication protocols to provide low latency and high bandwidth MPI messaging. In contrast, MPI library implementations for Ethernet utilize the operating system network protocol stack, leading to higher message latency and lower message bandwidth. However, on the NAS benchmarks, GM messaging over Myrinet only achieves 5% higher application performance than TCP messaging over Ethernet. Furthermore, efficient TCP messaging implmentations improve communication latency tolerance, which closes the performance gap between Myrinet and Ethernet to about 0.3% on the NAS benchmarks. This shows that commodity networking, if used efficiently, can be a viable alternative to specialized networking for high-performance message passing.