** Sun 13 March 2016
TL;DR: At the risk of stating the obvious, manual management of files on disks now in 2016 is increasingly old-fashioned and largely unnecessary, especially among the non-technorati. Encapsulated / managed cloud services and consumer web applications have made it anachronistic for most normal people. Whether this is a good thing can be debated, but it is happening nonetheless.
In this post, I explore this topic in some detail as it relates to my personal experience.
Data packratry
I’m something of a “data packrat”. Next week is my 31st birthday, and I’m carting around almost 2 decades worth of carefully curated files and other data with personal importance going all the way back to 1998. Among other things this includes:
-
All of my e-mail from 1998 to 2006 before moving to GMail for personal email
-
All school work high school onward. More LaTeX that I’d like to admit.
-
Materials for conference talks and tutorials
-
Sensitive personal documents: taxes, investments, expenses, and so forth
Legal documents: 2 startups’ worth of legal documents, apartment leases, and any other contractual matters
-
Miscellaneous computer backups (largely disorganized)
-
Digital photos
But, in general, I’m a weird person, so I have no idea how common this is.
For me, Dropbox came along and radically simplified my personal data management after many years of careful data stewardship on CD-R’s, DVD-R’s, and a succession of external hard drives. I also try to forget the number of times I forgot to e-mail myself an important file (or copy it to a thumb-drive). (I know I should be moving to Camlistore. One of these days!)
Just a couple of years ago, I went through my spindles of optical disks and external spinning-rust hard drives and consolidated the last of my precious bits in my Dropbox folder. It felt good to decommission all that old physical media, especially since I’d heard CD-Rs go bad after 20 or 30 years.
The Nomadic *Nix Hacker
I also work on any of 4 different computers at any given time: powerful desktop workstations at home and at my desk at Cloudera, a Linux laptop for hacking on-the-go, and a MacBook Pro for PowerPoint and other stuff (e.g. Adobe products) that don’t work well (or at all) in Linux. To keep myself from completely going crazy, I use some tools to make the cost of jumping from one machine to another low:
- GitHub: for code, dotfiles, and other environment configuration
Dropbox: As a replacement for USB sticks and e-mail for miscellanea (and big stuff that doesn’t make sense to put in git)
Thinking of my computers as fungible Chrome, git, and Emacs terminals has had huge benefits for my productivity, but that’s a whole different topic.
Ubiquitous accessible-everywhere cloud storage has been very helpful for me, but I suspect my needs are atypical. As indispensible as Dropbox has been for me as a file syncing tool for Linux, several things have made me wonder why average consumers (who never open the system shell) will continue to need the proverbial “magic folder”.
All your data are belong to us
The biggest state change to my personal data in the last 2 years was Google Photos. After a short trial, I imported all of my digital photos (not that much of a shutterbug, honestly) from 2005 onward and deleted all the associated files from my Dropbox. It felt amazing. Pedants will note that Google Photos is really just paying Google for cloud storage, but the liberating aspect of it is that you don’t have to wrangle files manually anymore.
After a little more data reorganization (moving “cold storage” to Google Drive, which I started paying for because of Google Photos), I was able to stop paying for Dropbox, and am unlikely to ever need to again.
Then I got to thinking, what data do most average consumers really need to keep in manually-managed cloud storage (aka Dropbox-like)? The kind of data that I’ve been hoarding for the last almost-20 years is either irrelevant or being rapidly replaced by free or very low-cost managed mobile or web applications:
-
E-mail and messaging: GMail, Facebook, WhatsApp
-
Photos: Facebook, Instagram, Adobe Cloud products
-
Documents: Google Docs, Office 365, Evernote
-
Code (for hackers): GitHub and friends
The only thing I am not sure about is managing ad hoc “sensitive documents”. For example: tax documentation, apartments and employment contracts, expenses, and so forth. For now, I’m still bearing the mental burden of arranging files in folders and giving them names (naming things is exhausting).
Conclusions
As always, it will be interesting to see how the technology develops, but it would be interesting if, for consumer use cases, manual file management could mostly become a thing of the past. The challenge will be how to handle the overflow of miscellanea not fitting into one of the tidy “managed” cloud buckets.
To me, the biggest opportunity that I think about is personal financial document management and taxes. Maybe in the future, we will have online “continuous financial tracking” so that, come tax season, we only have to reveal our personal finances to the IRS electronically and either send or receive a tax payment (or refund).