Preparing for the Collapse of Digital Civilization
While visiting the USA, I came across a delightfully bizzare TV show - Doomsday Preppers.
For those who don't know, this pseudo-documentary follows the lives of certain... eccentric... families who believe that the end of civilization is coming and they better get busy preparing for that eventuality.
Whereas you and I might keep a bit of spare cash hidden away, along with some out-of-date cans of food, these guys go the whole hog. Vast basements packed with food, gallons of oil to run generators, bomb-proofing their buildings, and training with guns. Lots of guns.
It's easy to laugh at these folk - their paranoia seems completely off the scale compared with the likelihood of the threat. And yet - I find them admirable. Come the apocalypse, I would likely last all of five minutes - whereas the "preppers" could survive indefinitely. They have the resources, the training, the experience, and the mental fortitude which comes from relentless preparation for the collapse of their world.
Which, naturally, brings me on to Google Reader.
Whither Google Reader
Last week, Google announced that it was killing off its popular Reader product. Howls of anguish from the loyal users of Reader - it was as if someone had announced the collapse of civilization.
As we come to rely more and more on the Internet, it's becoming clear that there is a real threat posed by tying oneself to a 3rd party service. The Internet is famously designed to route around failures caused by a nuclear strike - but it cannot defend against a service being withdrawn or a company going bankrupt.
It's tempting to say that multi-billion dollar companies like Apple and Google will never disappear - but a quick look at history shows Nokia, Enron, Amstrad, Sega, and many more which have fallen from great heights until they are mere shells and no longer offer the services which many people once relied on.
See, for example, this article from 2007 - Will MySpace Ever Lose Its Monopoly.
Even if the company survives - and there are remarkably few 100+ year old companies - we are at the mercy of third party services being shut down - witness Ping, Mobile Me, Buzz, Wave, Reader, etc. etc. ad mortem.
There are two questions that we need to ask when considering whether to adopt a new service.
Firstly - can I export my data? Secondly - is there an alternative which I control and therefore isn't at risk of collapse?
Export
I like to pose this question to my photography friends - "What would you do if Yahoo! suddenly decided to delete all your Flickr photos?" Some of them have backups - most faint at the thought of all their work vanishing.
Luckily, services like Google, Facebook, Flickr and Twitter offer users a way to export their data. This is something you should do regularly - because you may not get much notice that a service is disappearing.
FormSpring recently announced that they were shutting down. They said:
Sunday, March 31st will be the last day you’ll be able to ask questions or post content on Formspring. You’ll be able to export your responses from now through Monday, April 15th, after which the site and apps will go offline, and any content will be permanently deleted.
That's a month to grab your stuff and go. Are you on a long vacation? In hospital? In prison? Otherwise without Internet access? Tough - your data is toast.
Avoid
Now we come to the "digital preppers" section. What can you do to ensure you never need to rely on anyone else?
Here's my rough guide to how you can self host many of your essential digital services.
RSS Reading
Before Google Reader, we had RSS readers running on our computers. We can now recreate the Google Reader experience by running Tiny Tiny RSS on your server. TT-RSS is a web app just like Google Reader - it fetches your feeds, lets you read them, share them, save them, etc. The only difference is that it runs on your server rather than Google's.
It's open source - so it will keep working even long after the original coders have left the project.
Run this on your server and never worry about the bottom dropping out of your world.
Photo Sharing
The Open Photo Project is the perfect resilient replacement for Flickr and other image sharing services. Indeed, you can export your photos, tags, and comments from most major platforms. You can host your photos - and a community - on you very own server.
OpenPhoto also has smartphone apps and is open source.
Mapping
Google offer Maps - for now. There are other mapping providers out there, but they are all at risk of companies going bust or deciding that they no longer want to provide a service.
Enter OpenStreetMap - think of it as Wikipedia for maps. A crowd sourced map - continually updated, with mobile apps, navigation, and beautiful imagery. There are multiple providers who use OSM as their back end. If you are really paranoid, you can download the entire planet's map. A mere 27GB (compressed). Not an insignificant download - but manageable. Subsequent downloads are much smaller.
File Storage and Sharing
This is where we start moving off the beaten path and head into the wilderness. One of the lovely things about DropBox is that they provide a very simple way to synchronise multiple computers - complete with a range of apps for mobile phones. Rsync is exactly the same as DropBox - only a lot more complicated. You can use it to keep multiple computers in sync with each other. When files change on one machine, those changes are securely pushed to another machine.
There are limitations - few mobile clients, and no easy way to share files with others, for example.
What Else?
There are a huge range of services we use which are operated under the capricious whims of distant companies. This is by no means an exhaustive list of every option available to you. What services do you use which you would like to see decentralised?
Jasper says:
Nice article! I have adopted this mindset as well: if I can't export my stuff, I'm not using your service. Some exceptions exist, when for example I'll be using a service for a short period of time to achieve a single goal or something.
Also, you should really include Diaspora in this list, as it allows you to host your own pod, forming a free (as in freedom) social network with other pods.
Terence Eden says:
Diaspora is an interesting one. I haven't used it personally, and was under the impression that it had withered on the vine. Will take a look, thanks.
j says:
I've been thinking a lot about this too. For the past two years I've been using Google Doc/Drive and it suits my needs perfectly until there's a problem, either on Google's servers or with my ISP or anywhere in between. Recently it seems like Google's been having problems and it's just so frustrating when I can't get my own files. Fortunately I only work with words downloading all my texts is not so hard. I worry about the long-term fitness of cool apps like Draft when there is no easy option to run them locally.
Thanks expecially for the rsync recomendation, It seems like something that will suit my needs very well. I hope your article gets out and gets people thinking about the services they use. This reminds me of the old Nokia n810, which had an open source operating system. After Nokia completely dropped support for the OS the community stepped in and released updates and even added functionality as recently at 2010. That is the sort of long term support that is extremely useful to a low-end computer user like myself.
Jon says:
check out http://tent.io ("Tent is a protocol for distributed social networking and personal data storage")
rantfoil says:
I think self-hosting is great for people who understand computers, but for the larger populace, it may be more important to find hosting providers who are non-profit. That's what I'm trying to do with Posthaven. Surely computing technology and storage costs will come down to a point where a paid service can simply pay for all content being stored forever.
Then at that point the only thing you have to worry about is whether or not the software is good. And there's no reason why software of this kind can't be as good or better than what you expect from for-profit corporations like Facebook or Google.
Nelson says:
Hey hey, instead of rsync why not own cloud? That's what I'm using and it works great....
Jonathan says:
My thought on all of this tech apocalypse talk is the that, while these services were great when this technology first came out and was changing once a month, a lot of this stuff has somewhat stabilized and should be able to be run on private software, or if web based access is a necessity, VPSs running open sourced web apps...
Why rely on outsourcing (which as the article stated, nobody's service is ever certain), when the personalized solution isn't that hard. Half of these services run on either open source web apps, or could be cloned with basic modifications to such apps. Even the pretty veneers they provide are open sourced in a lot of cases in the form of themes, etc for these install-able web apps (see the thousand faces of wordpress, etc.). Why rely on facebook or any of the blog services when you can have a VPS running wordpress (or drupal, or a million other services) installed in less than 30 minutes?
Take back your digital identity, I say... These sites were handy at one point but there's no guarantee they will exist tomorrow, they've proven to be cavalier with your data at best and the personal solution at worst offers an out-of-the-box experience like you get from these services and at best can give you something completely customizable.
pixelherd says:
I'm torn between the paranoid need to have control of all my digital things, and the lack of casual competence to make the self-hosted gubbins effective enough to fit with what I want to do. Clearly, the solution is to give money to somebody who is competent, but we return full circle to paranoia: how competent are they, really. My tinfoil hat is tight around the ears.