Talks

libp2p-day Istanbul Nov 2023

Nebula: A Network Agnostic DHT Crawler

Dennis Trautwein

Dennis presents Nebula, ProbeLab's network-agnostic DHT crawler, and demonstrates it live against several supported networks including IPFS/Amino, the Ethereum consensus and execution layers, Filecoin, Polkadot/Kusama/Rococo/Westend, and the newly added Celestia mainnet. He explains the crawl algorithm — connecting to bootstrap peers, generating keys that fall into each Kademlia bucket, and recursively asking peers for ever-closer nodes until the reachable set is exhausted — and walks through the CLI flags, dry-run mode, and the four newline-delimited JSON output files that capture crawl metadata, per-peer visits with protocols/agent versions/errors, and optional neighbor data for full topology reconstruction. He shows how the monitor subcommand reconnects to discovered peers with exponential backoff to track uptime and churn, feeding studies on routing-table health, replication factor, and the IPFS DHT incident earlier in the year. The talk also describes how Nebula was recently refactored to abstract over different DHT implementations so it can map disc-v5 ENR node IDs to libp2p peer IDs and extract Ethereum-specific fields like fork digest into a free-form properties map, and ends with a roadmap pitch covering IPNS, GossipSub, and bandwidth measurements.

Watch

libp2p-day Istanbul Nov 2023

pcp: A Fully Decentralized Peer-to-Peer File Transfer Tool

Dennis Trautwein

Dennis introduces PCP (peer copy), a decentralized command-line file transfer tool he built as an entry project into libp2p, positioning it as a decentralized alternative to centralized tools like croc and Magic Wormhole. The talk walks through how PCP encodes a short channel identifier as four BIP-39 words combined with a truncated timestamp, hashes that into a CID, and writes a provider record into the Amino DHT so the receiver can rediscover the sender's multiaddrs and establish a direct connection. He covers the supporting mechanics: AutoNAT v2 for address discovery, mDNS for local-network peers, Circuit Relay v2 reservations and DCUtR-based hole punching for NAT traversal, and a PAKE-style exchange over the remaining words to derive a session key and authenticate the transfer. Dennis closes by noting that the current 0.4 release is outdated, that an in-progress 0.5 rewrite to adopt newer libp2p features has stalled due to time constraints, and he calls on the community to help drive features like CID-based fetches, transfer resumption, and a JS or browser port.

Watch

IETF 118 Nov 2023

Design and Evaluation of IPFS: A Storage Layer for the Decentralized Web

Dennis Trautwein

Dennis presents the SIGCOMM 2022 paper "Design and Evaluation of IPFS: A Storage Layer for the Decentralized Web," giving an overview of IPFS as a transport-agnostic, content-addressed system built around CIDs and walking through the anatomy of a CID (multibase, version, multicodec, multihash). He explains the content lifecycle in which a publisher writes a provider record mapping a CID to a peer ID into the Kademlia DHT while the data itself stays on the origin node, and a retriever first asks connected peers opportunistically before performing a DHT walk to fetch the provider and peer records. He then covers the measurement methodology — DHT crawls every 30 minutes, controlled probe nodes deployed across seven AWS regions, and public gateway logs — reporting roughly 464,000 unique IPs across 150+ countries and 2,700 ASes, heavy AS-level centralization with the top five hosting over 50% of IPs, churn distributions by country, and lookup latency where 80% of EU retrievals resolved under 500 ms. He contrasts 2021 publication latency of up to two minutes against current medians around six to seven seconds, attributes the improvement to a much larger stable-peer baseline (85-90% today vs. 55-60% in 2021), and closes by pointing to ProbeLab's ongoing weekly measurements at stats.ipfs.network and open future work on content availability, adverse network conditions, and routing latency.

Watch

Protocol Berg v1 Sep 2023

The Best of Both Worlds: Exploring the Role of Centralization in IPFS

Dennis Trautwein

This Protocol Berg 2023 talk examines how IPFS uses semi-centralized "hybrid" components to address three structural challenges of a fully decentralized network: massive content publication, content retrieval performance, and adoption on resource-constrained mobile clients. Dennis walks through the design of Interplanetary Network Indexers (IPNI), Hydra Boosters, and HTTP gateways, then presents measurement results from ProbeLab showing IPNI indexes roughly two orders of magnitude more provider records than the DHT and reduces lookups from seconds to hundreds of milliseconds (single-digit milliseconds when CDN-cached). The talk also covers the trade-offs each component introduces — single-points-of-failure for indexers, route poisoning and eclipse-attack surface for Hydras, and loss of end-to-end CID verification plus operator-side privacy asymmetries for gateways. It closes with the decision to unplug the Hydra Booster shared database as the underlying DHT matured, framing this as a successful pattern of using centralized scaffolding to bootstrap a decentralized system that can later stand on its own.

Watch

IPFS þing Apr 2023

Data Driven Protocol Design and Optimisation

Yiannis Psaras

Yiannis Psaras opens the IPFS þing 2023 measurement track with an overview of ProbeLab's data-driven protocol design work across the IPFS network, walking through their core methodologies of DHT crawling, controlled probe fleets, and infrastructure log analysis. He recaps recent measurement campaigns including the Hydra dial-down experiment, the global hole-punching study with over six million attempts, the January 2023 incident in which more than 50% of IPFS DHT nodes became unresponsive, and the provider record liveness study that justified halving reprovide traffic in production from December 2022. The talk previews probelab.io as a public home for these results alongside stats.ipfs.network, and surveys ongoing optimization work including the double-hash DHT for reader privacy (IPIP-373), optimistic provide, and the IPFS magic numbers effort. Psaras closes by flagging GossipSub measurement, used heavily in Filecoin and Ethereum 2.0, as the team's next major focus area.

Watch

IPFS þing Apr 2023

DHT Double Hashing Updates & Migration Plan

Yiannis Psaras · Guillaume Michel

Yiannis Psaras and Guillaume Michel present the migration plan for the IPFS DHT reader privacy upgrade, formerly known as double hashing and specified in IPIP-373. The scheme combines a CID-agnostic DHT lookup using a salted second hash, prefix-based requests that return multiple provider records to give k-anonymity, and provider record encryption keyed on the CID itself, so intermediate DHT servers can no longer link requesters to the content they fetch. Because this is a breaking protocol change, the talk focuses on coordinating a synchronized switch using a hard-coded IPNS key in a Kubo release that nodes poll periodically, with bootstrappers, content providers, DHT clients, and DHT servers each following defined behaviors during a transition period in which both old and new DHTs run side by side. The presenters discuss timeline targets through Q2 and Q3 of 2023, the role of IPNI and cid.contact as a fallback that may simplify the dual-stack period, and threat model limitations: the upgrade defeats passive traffic sniffing and bulk surveillance but does not hide lookups when the CID is already publicly known.

Watch

IPFS þing Apr 2023

State of Content Routing through the DHT: Latest Developments & Measurement Results

Yiannis Psaras

Yiannis Psaras presents measurement results from two major IPFS DHT events: the Hydra dial-down and a January 2023 incident where roughly 60% of network nodes became unresponsive due to misconfigured Resource Manager defaults. He describes ProbeLab's controlled experimental setup of geographically distributed probes that publish and retrieve random CIDs, and reports that disabling the 2,000 Hydra heads produced a roughly 10-13% latency increase, broadly matching expectations. The Resource Manager incident caused an additional 9-15% slowdown on put operations and 7-20% on get operations, which was largely recovered after Kubo v0.18.1 shipped better defaults and around 8,000 nodes upgraded within ten days. Psaras also shares current DHT lookup performance numbers showing significant regional disparities between EU/US East/US West clients and other locations, along with observations on peer ID rotation churn and the shift toward a more stable population of long-lived online nodes.

Watch

IPFS þing Apr 2023

Measuring IPFS

Yiannis Psaras

In this short lightning talk at IPFS þing 2023, Yiannis Psaras introduces ProbeLab's data-driven approach to protocol design and optimization for IPFS. He presents probelab.io as a contextualized collection of measurement plots — including DHT lookup latency across multiple geographic regions — published alongside experiment descriptions and links to the GitHub repositories of the underlying tooling. He also describes weekly network health snapshots that compare time-to-first-byte for a curated set of websites loaded via Kubo versus HTTP, noting that IPFS frequently outperforms HTTP thanks to content addressing and universal caching. Current measurement focus is on the public IPFS DHT, Bitswap, and Hydras, with plans to consolidate everything under stats.ipfs.network and to extend coverage to additional content-routing subsystems in collaboration with other protocol teams.

Watch