Designing a DHT for Low Latency and High Throughput Export

@inproceedings{citeulike:780114, abstract = {Designing a wide-area distributed hash table (DHT) that provides high- throughput and low-latency network storage is a challenge. Existing systems have explored a range of solutions, including iterative routing, recursive routing, proximity routing and neighbor selection, erasure coding, replication, and server selection. <P> This paper explores the design of these techniques and their interaction in a complete system, drawing on the measured performance of a new DHT implementation and results from a simulator with an accurate Internet latency model. New techniques that resulted from this exploration include use of latency predictions based on synthetic coordinates, efficient integration of lookup routing and data fetching, and a congestion control mechanism suitable for fetching data striped over large numbers of servers. <P> Measurements with 425 server instances running on 150 PlanetLab and RON hosts show that the latency optimizations reduce the time required to locate and fetch data by a factor of two. The throughput optimizations result in a sustainable bulk read throughput related to the number of DHT hosts times the capacity of the slowest access link; with 150 selected PlanetLab hosts, the peak aggregate throughput over multiple clients is 12.8 megabytes per second.}, author = {Dabek, Frank and Li, Jinyang and Sit, Emil and Robertson, James and Kaashoek, Frans M. and Morris, Robert}, citeulike-article-id = {780114}, citeulike-linkout-0 = {http://www.usenix.org/publications/library/proceedings/nsdi04/tech/dabek.html}, journal = {NSDI '04}, keywords = {dht, high, latency, low, throughput}, pages = {85--98}, posted-at = {2006-07-30 09:46:57}, priority = {0}, title = {Designing a DHT for Low Latency and High Throughput}, url = {http://www.usenix.org/publications/library/proceedings/nsdi04/tech/dabek.html} }

TY - CONF ID - citeulike:780114 L3 - citeulike-article-id:780114 N2 - Designing a wide-area distributed hash table (DHT) that provides high- throughput and low-latency network storage is a challenge. Existing systems have explored a range of solutions, including iterative routing, recursive routing, proximity routing and neighbor selection, erasure coding, replication, and server selection. <P> This paper explores the design of these techniques and their interaction in a complete system, drawing on the measured performance of a new DHT implementation and results from a simulator with an accurate Internet latency model. New techniques that resulted from this exploration include use of latency predictions based on synthetic coordinates, efficient integration of lookup routing and data fetching, and a congestion control mechanism suitable for fetching data striped over large numbers of servers. <P> Measurements with 425 server instances running on 150 PlanetLab and RON hosts show that the latency optimizations reduce the time required to locate and fetch data by a factor of two. The throughput optimizations result in a sustainable bulk read throughput related to the number of DHT hosts times the capacity of the slowest access link; with 150 selected PlanetLab hosts, the peak aggregate throughput over multiple clients is 12.8 megabytes per second. JF - NSDI '04 EP - 98 TI - Designing a DHT for Low Latency and High Throughput SP - 85 KW - dht KW - high KW - latency KW - low KW - throughput AU - Dabek, Frank AU - Li, Jinyang AU - Sit, Emil AU - Robertson, James AU - Kaashoek, Frans AU - Morris, Robert UR - http://www.usenix.org/publications/library/proceedings/nsdi04/tech/dabek.html ER -