Some of the material in is restricted to members of the community. By logging in, you may be able to gain additional access to certain collections or items. If you have questions about access or logging in, please use the form on the Contact Page.
Small, M. (2012). Optimizing MPI Point-to-Point Communication Performance on RDMA-Enabled SMP-CMP Clusters. Retrieved from http://purl.flvc.org/fsu/fd/FSU_migr_etd-5434
Clusters of Symmetric Multiprocessing (SMP) nodes with multi-core Chip Multiprocessors (CMP), also known as SMP-CMP clusters, are ubiquitous today. Message Passing Interface (MPI) is the de facto standard for developing message passing applications for such clusters. Most modern SMP-CMP clusters support Remote Direct Memory Access (RDMA), which allows for flexible and efficient communication schemes but introduces a new model that can be challenging to exploit. This dissertation research explores leveraging the flexibility provided by RDMA to optimize MPI point-to-point communications for both small and medium to large messages on SMP-CMP clusters. For small messages, a scheme is devised that improves the buffer memory management in the existing RDMA-based small message channel design; and a novel shared small message channel design is developed that reduces both individual channel resource requirements as well as the number of channels needed by an MPI application, greatly improving small message channel resource utilization and scalability without adding significant overheads or sacrificing the performance benefits of RDMA. MPI medium and large messages are realized by various rendezvous protocols whose performance is very sensitive to the timing of the critical events in the communication (protocol invocation scenarios). As such, existing MPI implementations that use a fixed protocol across all communications suffer from various performance problems such as unnecessary synchronization and communication progress issues. In my research, I explore the idea of protocol customization that allows different protocols to be used for different situations. First, a repository of protocols that can collectively provide near-optimal performance for all protocol invocation scenarios is developed. This repository provides the foundation for profile-driven and compiler-assisted protocol customization for performance improvement. Furthermore, a communication system with dynamic protocol selection is developed that integrates four protocols into a single system and is able to choose an optimized protocol to suit the run-time characteristics of a particular communication while fully supporting MPI semantics. These techniques reduce unnecessary synchronizations, decrease the number of control messages that are in the critical path of communications, and improve the communication progress, which results in a significantly better communication-computation overlap capability.
eager protocol, MPI, mpi large message, mpi small message, RDMA, rendezvous protocol
Date of Defense
November 5, 2012.
Submitted Note
A Dissertation submitted to the Department of Computer Science in partial fulfillment of the requirements for the degree of Doctor of Philosophy.
Bibliography Note
Includes bibliographical references.
Advisory Committee
Xin Yuan, Professor Directing Thesis; Valerie Shute, University Representative; Robert Van Engelen, Committee Member; Ashok Srinivasan, Committee Member.
Publisher
Florida State University
Identifier
FSU_migr_etd-5434
Use and Reproduction
This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). The copyright in theses and dissertations completed at Florida State University is held by the students who author them.
Small, M. (2012). Optimizing MPI Point-to-Point Communication Performance on RDMA-Enabled SMP-CMP Clusters. Retrieved from http://purl.flvc.org/fsu/fd/FSU_migr_etd-5434