Behind-the-Scenes Story on Applying Cross-Layer Coordination to Disks and Raids

Coordinating storage components across abstraction layers has demonstrated significant performance gains. However, when applied near the physical storage, this approach relies on exposing and exploiting low-level hardware characteristics, perhaps a large number of them, to cope with complex modern disks and RAIDs. Through implementation and validation of prior research on track-aligned accesses and its incorporation in RAIDs, as well as through experimentation with our proposed queue coordination in RAIDs, we confirmed that cross-layer coordination can indeed yield high performance gains. On the other hand, the effective use of cross-layer coordination involves overcoming several challenges: (1) developing efficient and automated ways to extract and exploit hardware characteristics due to rapidly evolving disks, (2) fostering a greater understanding of the legacy storage data path, so that we can better predict the benefits of low-level optimizations and their intertwined interactions, and (3) inventing efficient and automated ways to tune the low-level parameters.