Solid Migrator

Application and specification to forward and/or move data between Solid Pods

The Problem

While we were working on solid-nextcloud, a community solid server shut down. This server was well-used and contained a lot of datapods. While there was time to copy the data and store it somewhere else, it did a lot of damage still. All links to these satapods broke and had to be fixed manually. Not every link could be altered, for example when the owner of the datastore containing the link couldn’t be reached.

This lead to the realization that storing linked data across personal datapods all over the internet leads to a brittle system, if not mitigated.

Linkrot is already a large problem for the current web. In 2003 an article in the washington post quoted Brewster Kahle, of internet archive fame, who estimated that the average lifespan of a web page was just 100 days. The first link I found was in an article that had made a cached copy of this article, to prevent linkrot I assume. In an ironic case of Murphy’s Law, this cache was taken offline.

Read more about the problem

Possible Solutions

Based on the research mentioned below (see the “Research” section) we’ve formulated what is out of scope (things we won’t do) and several possible solutions:

These solutions lead to the insight that two things must be created in order to solve the problem: a specification and an application.

Specification

Of the proposed solutions defined above, some require additions to one or more Solid specification. But the most basic specification is to create a new ontology that defines the semantics of forwarding.

Solid Migrator Application

We’re building the Solid Migrator as a set of solid application tutorials first. We found that all code examples for Solid are either outdated or use a complex toolchain setup. You can testdrive the latest version at https://pdsinterop.org/solid-migrator-app/www/

Research

Part of the research into the problem and possible solutions revolved around linkrot and how it can be mitigated. Further details are available on separate pages:

Existing attempts to mitigate linkrot

Team

These PDS Interop team members will be implementing the Solid Migrator roadmap:

Auke van Slooten @poef - Research & Specification
Ben Peachey @Potherca - Prototype Migrator
Yvo Brevoort @ylebre - Community Outreach & Server Extensions