This chart shows the average download latency in seconds per day of our IPFS Kubo node in the selected region. The first dropdown menu selects the region. The second dropdown menu selects how Kubo found the content: through the DHT, through IPNI (the indexer returned a provider record), or through Bitswap (a peer that was already connected sent the first block, so no content routing was needed). We run the measurements with our Tiros measurement suite.
A Download is defined as requesting a random CID that we have learned about with our Bitswap sniffer tool from the network using the different content routing systems DHT and IPNI (and also Bitswap which is technically not a content routing system). The Download measurements show the latencies until we have received the first byte. This decouples the Download measurement numbers from the file size of the random CID.
We are showing average latencies here as opposed to percentiles because they are additive and only this allows us to correctly visualize the different steps as stacked bars. The tooltip shows more information like the total P50 and P90 percentile latencies, plus the number of samples used to calculate the aggregations.
In our measurement setup we effectively run the command ipfs cat $CID and read at most 100MiB of data. This limit was chosen to not generate unnecessary network load. The command times out after 10 seconds. Failed and timed-out downloads are not included.
ipfs cat $CID
The download performance is broken down to each stage of the process:
Time to broadcast: The time it took until the DHT and IPNI content routing subsystems were activated to resolve the CID's providers. Kubo implements a "Provider Search Delay" of 1s in which it asks its immediately connected peers via Bitswap if they happen to have the content for the CID. Kubo waits up to 1s for a positive response before it broadcasts the request to the DHT or IPNI. This "Time to broadcast" bar should almost always be around 1s.
Provider found delay: The time it took until the provider was found via the DHT or IPNI relative to the time when the broadcast operation started.
Provider connection delay: The time it took until our Kubo node has established a connection to the provider relative to the time when the provider was found.
TTFB delay: The time it took until the first byte of the file was received from the provider relative to the time when the provider connection was established.
Time to first block: Only for Bitswap downloads. The time it took until the first block of the file was received relative to the start of the download.
The resource provisioning of our Kubo node under test is as follows:
The line on the right y-axis shows the number of downloads (samples) per day. Dashed lines mark the day we changed the resources of our Kubo node and the day we deployed Kubo v0.39.0.
Fetching a file, using Kubo, through Amino, Bitswap and IPNI