Optimistic Provide: Optimize the IPFS DHT
Dennis Trautwein
About this talk
Presented at P2P Paris in April 2022, this talk introduces "Optimistic Provide," a proposal to make the IPFS DHT provide operation roughly an order of magnitude faster. Trautwein first reviews how Kademlia, routing tables, and the lookup algorithm drive content advertisement, then shows measurement data where the median provide takes around 30 seconds and the 95th percentile exceeds two minutes, even though the closest peers are typically discovered in under one second. He proposes two approaches: running multiple parallel lookups from different seed peers and terminating once their query queues intersect, or estimating peer proximity using a local network-size estimator (based on a published technique that avoids gossip) to decide when a discovered peer is "close enough" to receive the provider record. Preliminary results show the multi-query approach intersects too early and yields peers that are too far in XOR distance, while the proximity-estimation approach produces closeness in the same order of magnitude as the status quo and a network-size estimate that tracks crawler ground truth. Next steps include sweeping parameters, evaluating the network-size estimator across a distributed fleet, and measuring the actual end-to-end publication-time improvement.