https://blog.shuaizhang.cc/en-us/tags/distributed-storage/Posts gathered under this term.Hugoen-USSat, 08 Feb 2020 13:01:05 +0000https://blog.shuaizhang.cc/en-us/posts/distributed-storage-from-scratch-3/Sat, 08 Feb 2020 00:00:00 +0000https://blog.shuaizhang.cc/en-us/posts/distributed-storage-from-scratch-3/This article presents the high-level design of a distributed storage system built from scratch and introduces the user-facing API and the division of responsibilities among core modules.https://blog.shuaizhang.cc/en-us/posts/distributed-storage-from-scratch-2/Sat, 28 Dec 2019 00:00:00 +0000https://blog.shuaizhang.cc/en-us/posts/distributed-storage-from-scratch-2/Starting from file systems, relational databases, and disk characteristics, this article supplements the Distributed Storage From Scratch series with background knowledge.https://blog.shuaizhang.cc/en-us/posts/distributed-storage-from-scratch-1/Thu, 26 Dec 2019 00:00:00 +0000https://blog.shuaizhang.cc/en-us/posts/distributed-storage-from-scratch-1/As the prologue to this series, this article explains the goals, overall architecture, and upcoming content plan for building a distributed storage system from scratch.https://blog.shuaizhang.cc/en-us/posts/google-slicer/Sun, 13 Oct 2019 00:00:00 +0000https://blog.shuaizhang.cc/en-us/posts/google-slicer/This article introduces the control plane, client-side routing, and load-balancing design of Google Slicer, an auto-sharding component.https://blog.shuaizhang.cc/en-us/posts/percolator/Sun, 09 Jun 2019 00:00:00 +0000https://blog.shuaizhang.cc/en-us/posts/percolator/This article outlines how Google Percolator implements cross-row transactions and reliable notifications on top of Bigtable, with a focus on the transaction component.https://blog.shuaizhang.cc/en-us/posts/transactional-consistency-models/Sun, 26 May 2019 00:00:00 +0000https://blog.shuaizhang.cc/en-us/posts/transactional-consistency-models/This article introduces the formal framework of a paper on transactional consistency models and explains the intuitive meaning of concepts such as atomic visibility.https://blog.shuaizhang.cc/en-us/posts/cap-acid-what-can-we-do/Sun, 20 May 2018 00:00:00 +0000https://blog.shuaizhang.cc/en-us/posts/cap-acid-what-can-we-do/Starting from the CAP theorem and ACID properties, this article discusses the trade-offs among consistency, availability, and transaction design in distributed storage systems.https://blog.shuaizhang.cc/en-us/posts/anna-kvs/Sun, 08 Apr 2018 00:00:00 +0000https://blog.shuaizhang.cc/en-us/posts/anna-kvs/This post explains how the Anna key-value store uses the actor model and lattice-based conflict resolution to achieve high performance and tunable consistency.https://blog.shuaizhang.cc/en-us/posts/storage-engine-basic/Fri, 06 Apr 2018 00:00:00 +0000https://blog.shuaizhang.cc/en-us/posts/storage-engine-basic/This article summarizes common design methods for single-node storage engines from the perspectives of data and index layout, hash tables, tree structures, and more.https://blog.shuaizhang.cc/en-us/posts/facebook-f4/Mon, 02 Apr 2018 00:00:00 +0000https://blog.shuaizhang.cc/en-us/posts/facebook-f4/This article summarizes the architectural design of Facebook F4 for warm data object storage, along with its erasure coding and cross-data-center fault-tolerance scheme.https://blog.shuaizhang.cc/en-us/posts/facebook-haystack/Sun, 01 Apr 2018 00:00:00 +0000https://blog.shuaizhang.cc/en-us/posts/facebook-haystack/This post summarizes the design and optimization ideas behind Facebook Haystack's single-machine object storage engine for hot photo storage scenarios.https://blog.shuaizhang.cc/en-us/posts/amazon-dynamo/Mon, 12 Mar 2018 00:00:00 +0000https://blog.shuaizhang.cc/en-us/posts/amazon-dynamo/Reviews the consistency, replication, conflict handling, and routing designs in the Amazon Dynamo paper, and summarizes the key engineering trade-offs it made for highly available key-value storage.