Storage services form the platform on which widely-used cloud services, mobile applications, data analytics engines, and transactional databases are built. Such services are trusted with irreplaceable personal and commercial information by users, companies, and even governments.
The designers of storage services often have to choose between performance and reliability. If the developer makes the system reliable, performance is often significantly reduced. If the developer instead maximizes performance, a crash could lead to data loss and corruption.
In this talk, I describe how to build systems that achieve both strong reliability and high performance. In many systems, reliability is maintained by carefully ordering updates to storage. The key insight is that the low-level mechanism used to enforce ordering is overloaded: it provides durability as well as ordering. I introduce a new primitive, osync(), that decouples ordering from durability of writes. I present Optimistic Crash Consistency, a new crash-recovery protocol that builds on osync() to provide strong reliability guarantees and high performance. I implement these techniques in the Optimistic File System (OptFS) and show that it provides 10X increased performance for some workloads. With researchers in Microsoft, I employ the principles of Optimistic Crash Consistency in a distributed storage system, resulting in 2-5X performance improvements.
Speaker Biography
Vijay Chidambaram is a Ph.D candidate in the Department of Computer Sciences at the University of Wisconsin-Madison. His current research focus is to ensure the reliability of applications in the rapidly changing landscape of storage and cloud computing. Specifically, he has contributed new reliability techniques in (local and distributed) storage systems, and built frameworks for finding reliability bugs in applications. His work has resulted in patent applications by Samsung and Microsoft. He was awarded the Microsoft Research Fellowship in 2014, and the University of Wisconsin-Madison Alumni Scholarship in 2009.