New Storage Performance Development Kit Goes Live
By [email protected] on September 22, 2015
New non-volatile memory (NVM) technologies are transforming the storage landscape—and removing the historical I/O bottleneck caused by storage. With the performance of next-generation technologies like 3D XPoint, latency will now be measured not in milliseconds but nanoseconds.
While this is a huge step forward, fast media alone doesn’t get us to blazingly fast application performance. The rise of next-generation storage simply shifts the I/O bottlenecks from storage media to the network and software.
At Intel, we seek to address these bottlenecks, so developers can take full advantage of the potential performance of new NVM technologies. That’s a key goal driving the new Storage Performance Development Kit (SPDK), announced today at the Storage Developer Conference (SDC) in Santa Clara.
This new open source initiative, spearheaded by Intel, applies the high performance packet processing framework of the open source Data Plane Development Kit (DPDK) to a storage environment. SPDK offers a way to do things in Linux user space that typically requires context switching into the kernel. By enabling developers to work in user space, it improves performance and makes development simpler for storage developers.
As part of the SPDK launch, Intel is releasing an open source user space NVMe driver. Why is this important? For developers building their storage application in user space, this driver enables them to capitalize on the full potential of NVMe devices. Andy Warfield, CTO of Coho Data says, “The SPDK user space NVMe driver removes a distracting and time consuming barrier to harnessing the incredible performance of fast nonvolatile memory. It allows our engineering team to focus their efforts on what matters: building advanced functionality. This translates directly into more meaningful product development and a faster time to market.”
A lot of storage innovation is occurring in user space. This includes efforts like Containers, Ceph, Swift, Hadoop and proprietary applications designed to scale storage applications out or up. For applications like these, the SPDK NVMe polled-mode driver architecture delivers efficient performance and allows a single processor core to handle millions of IOPS. Removing or circumventing system bottlenecks are key for realization of the performance promised by next gen NVM technologies. For this reason, companies like Kingsoft Cloud have joined the SPDK community, and their experiences and feedback have already influenced the course of the project. Kingsoft Cloud is a leading cloud service provider, which provisions online cloud storage and hosting services to end users in China. “We will continue to evaluate SPDK techniques and expect to leverage SPDK for improving Kingsoft cloud’s scale out storage service” said Mr. Zhang, chief storage architect at Kingsoft Cloud.
With this announcement, storage developers can join an open ecosystem that is leveraging shared technology. This adds up to faster time to market while positioning organizations to capitalize on future NVM technology and storage advancements.
