IPFS

Distributed Hash Table (DHT)

IPFS Amino DHT

One of the core components of IPFS is the Distributed Hash Table (DHT), which enables nodes to locate and retrieve content from other nodes on the network. The IPFS DHT is a distributed key-value store that maps content addresses (aka CIDs) to the nodes that are currently storing that content. It works by dividing the content address space into small, hash-based “buckets” that are distributed across the network. When a node wants to find a piece of content, it queries the DHT with the content’s CID, and the DHT returns the nodes that are currently storing that content, in the form of Provider Records. This allows content to be located and retrieved in a decentralized, efficient, and fault-tolerant manner.

Lookup Performance

Measuring the time it takes to publish and retrieve information from the IPFS DHT is crucial for understanding the network’s performance and identifying areas for improvement. It allows us to assess the network’s efficiency in different regions, which is essential for global-scale applications that rely on the IPFS DHT. Measuring performance across different regions helps identify potential bottlenecks and optimize content delivery.

DHT/IPNI Lookup Comparison

Lookup performance comparison between Amino DHT and IPNI for several settings and percentiles

DHT Lookup Performance

DHT Lookup Performance

We have instrumented the following experiment to capture the DHT Lookup performance over time and from several different geographic locations. We have set up IPFS DHT Server nodes in four (4) different geographic locations. Each node periodically publishes a unique CID and makes it known to the rest of the nodes, who subsequently request it through the DHT (acting as clients). This way we capture the entire performance spectrum of the DHT, i.e., both the DHT Publish and the Lookup performance from each location. In this plot we present the observed DHT Lookup performance over all regions, as well as per region in various percentiles.

Amino DHT Lookup performance across different regions in various percentiles

DHT Lookup Performance Distribution

DHT Lookup Performance Distribution

We have instrumented the following experiment to capture the DHT Lookup performance over time and from several different geographic locations. We have set up IPFS DHT Server nodes in four (4) different geographic locations. Each node periodically publishes a unique CID and makes it known to the rest of the nodes, who subsequently request it through the DHT (acting as clients). This way we capture the entire performance spectrum of the DHT, i.e., both the DHT Publish and the Lookup performance from each location. In this plot we present the CDF of the DHT Lookup performance over all regions, as well as per region.

CDF of the DHT Lookup operation

Publish Performance

Publish performance is equally important to retrieve/download performance. It provides information regarding the time needed to make a piece of content available in the IPFS network for others to fetch. It includes the time to find the 20 closest peers to a CID and push Provider Records to those peers.

DHT/IPNI Publish Comparison

Publish performance comparison between Amino DHT and IPNI for several settings and percentiles

DHT Publish Performance

DHT Publish Performance

We have instrumented the following experiment to capture the DHT Publish performance over time and from several different geographic locations. We have set up IPFS DHT Server nodes in four (4) different geographic locations. Each node periodically publishes a unique CID and makes it known to the rest of the nodes, who subsequently request it through the DHT (acting as clients). This way we capture the entire performance spectrum of the DHT, i.e., both the DHT Publish and the Lookup performance from each location. In this plot we present the observed DHT Publish performance over all regions, as well as per region in various percentiles.

Amino DHT Publish performance across different regions in various percentiles

DHT Publish Performance Distribution

DHT Publish Performance Distribution

We have instrumented the following experiment to capture the DHT Publish performance over time and from several different geographic locations. We have set up IPFS DHT Server nodes in four (4) different geographic locations. Each node periodically publishes a unique CID and makes it known to the rest of the nodes, who subsequently request it through the DHT (acting as clients). This way we capture the entire performance spectrum of the DHT, i.e., both the DHT Publish and the Lookup performance from each location. In this plot we present the CDF of the DHT Publish performance over all regions, as well as per region.

CDF of the DHT Publish operation

Measurement Tools

Our network monitoring employs advanced crawling and probing tools and techniques to gather comprehensive data about network health, topology, and performance