Solid Migrator: Linkrot Mitigation: IPFS and NDN

The main cause of linkrot is the way URL’s are designed. The owner of the domain part of the URL is in full control over the contents of that URL. There is no builtin feature of the web to keep track of content changes, or even ownership changes over time. However there is a project that attempts to change the way you access data fundamentally.

IPFS

IPFS, or Interplanetary Filesystem, is a from-the-ground-up redesign of the way the web works. Fundamentally it replaces the URL and HTTP(S) protocols with something based on Content-addressing and Peer-2-peer resolution. To publish data on IPFS, you don’t have to enter a URL, instead the URL is automatically generated as a hash of the data. Because of this, if you have the IPFS URL, you know that the data returned will be exactly what it was when you entered the URL. The contents of a IPFS URL can never change. The contents can disappear however, there still needs to be a host that has a copy of these contents somewhere. However, it is trivial to pin the contents on a host you control. And if there are many hosts that have the contents pinned, the bandwidth is shared. So ideally this means that the more popular the data is, the faster it is delivered and there is no need for fat, expensive servers.

The obvious problem here is that data does change. So to keep up you would need to monitor what the URL of the latest version of the linked data is and update your URL’s. That doesn’t scale at all, so in addition to IPFS, they’ve design a replacement for DNS, called IPNS. To prevent a centralized service with a single point of failure, IPNS is built on top of a blockchain. The IPNS URL’s aren’t human readable, avoiding the domain squatter, but they are stable. IPNS is in fact similar to the DOI system, in that a static name can point to changing URLs. So the owner of the IPNS name, who has the cryptographic key to change the data in the blockchain, can update the IPFS URL whenever a new version of the data is uploaded into IPFS.

NDN and Eris

IPFS has a spiritual predecessor, called Named Data Networking, by Van Jacobsen. It also focuses on data or content addressing instead of DNS based URL’s. It started around 2006 and is still in an ongoing research phase. It doesn’t seem to have the same uptake as IPFS, however. Interestingly the NDN research directly mentions Ted Nelson, from Xanadu fame. NDN is part of a larger effort, by the IETF, called the Information-Centric Networking research group. A very recent effort that may be interesting is called ERIS. This aims to create a standard for addressing data by its content.

---