A lot of external USB and eSATA harddisks
Of course it is possible to travel with a few dozen virtual machines! After looking at the stack of external USB and eSATA harddisks I use to travel with essential materials I just had to take a picture of it. Sorry about the quality that isn’t up to snuff but I used the iPhone to take the snap.
Over the past year people have asked me, “when you have so much local storage, how do you decide what you travel with and how do you carry it with you”. Well, the picture here probably says more than a thousand words. Not that I couldn’t write a thousand words on it, if I really wanted to. And no, it is not a joke. I really do travel with 22 external HDD’s (over 13TB of storage).
Since I travel with the Clevo D900F “laptop” (ahum, sort of, mobile luggable workstation) I also travel with enough storage and data that if I need to grab a VM and do a quick fix on a bug somewhere, I can do so without having to wait until I can return to the larger workstations.
The current “travel batch” of disks consist of various brands because I usually get them as-needed but most are either Western Digital, Samsung, or Seagate’s in external enclosures that I just put together myself. The collection at this time are 8 x 1TB (USB), 6 x 500GB (USB), 2 x 500GB (eSATA 7200rpm), 4 x 320GB (USB), 2 x 160GB (USB).
Why so many? For the most part because I take a lot with me ranging from (encrypted) business data and mission critical data as well as quite a few development VM’s but also because I travel with my data in a redundant fashion. That is to say, I don’t have one set of files on one disk only, I have exact copies of those disks. So each external disk is a copy of another. Imagine a disk failure on a 1TB disk. That’s no fun. Especially if you can’t easily get to the data at any given point (or location) at any given time.
I would definitely advise everyone to do the same. If you travel and need to take important data and files with you, don’t cheap-out and go for double the backup. Anyone who has had a drive failure while traveling will know why spending a few extra bucks can save you a lot more in the long run when there’s a failure.
Cloud storage is not an option for me. I have never been a fan of storing important data somewhere where you don’t have any control over it. Recent revelations regarding, for example, DropBox, are only further proving my point. Would I ever trust a cloud storage service with business critical data of mine and my clients? No way! Even if I apply my own compression and encryption to it? Maybe but I’d still rather not.
Of course when you’re talking about the amount of data I travel with, cloud storage isn’t an option anyway. What am I going to do when I need to get 100MB of data from an external HD along with a 30GB virtual machine to work on the data? Download it via whatever crummy internet connection I might be hooked onto when traveling? And waste how much time doing that? Cloud storage can be useful, within reason, as long as you know what the drawbacks are and how much data you wish to put at risk.
As for those VMs, and I’m not sure what you use them for, but would a cloud-hosted version of said VMs work for you? Ones that have the horsepower of the servers they run on, but are accessible to you through ssh/vnc/whateveryouneed from wherever you are. A 3G/4g connection would be beneficial then, of course. Then again, this of course fully depends on what you need from these VMs.
It probably wouldn’t work too well. If it was just a quick fix and a remote compile/build it would work but usually I would also need to test the fix locally which then involves some graphically intensive processes.
Often VM’s are sufficient but if I need more direct hardware acceleration for 3D graphics I use VHD’s. Of course, I can’t keep all the VHD’s on the laptop at all times so those huge files are also compressed and archived to the external disks. If I need a specific environment I just take it off the external disk, onto a local one, reboot into the VHD, and voila, I can do my quick test/fix/etc.
Or you could just remote desktop into home like the rest of us do.
LOL! Actually, I did that for a while, long ago. But with everything-but-great internet connections at the time (while I already had an 8mbps upstream) that didn’t work too well. Of course, the real problem was that taking 2 month vacations and leaving stuff running at home without a way to swap a disk when there’s a failure on a RAID, or deal with powering on/off, etc. was just too risky.
If you really experience RAID failures so frequently that you have to plan for them when you go on trips, perhaps it’s time to reevaluate your storage plan.
Granted, my storage is significantly smaller (Just 12TB or so), but I find regular automatetd full backups between two JBOD setups is more than sufficient.
Plus of course a USB flash or external HDD you grab in case of fire. I’m sure you’ve got that and more. Wish I could afford the off-site colo fancy stuff, but I’m just a lowly code monkey.
I always have a few disks fail in the course of a year and it depends on brand and production batches a lot so the actual chance of this happening when I’m on a 2 month trip is 1:10 or something. At least with RAID6 I can incur 2 disks on a server failing but it’s still too much of a risk over a 2 month period when I can’t just fly back 6000 miles to swap a disk. 🙂
Of course there’s other things that can also fail when leaving systems on, unattended, for that long, at home. I have redundancy in the network switches and routers and modems and UPS’es to deal with potential power-outs but there’s no accounting for anything from a lightning strike to burglary. So it’s safer and cheaper to just turn everything off for those long periods and carry the data.
Another reason is that remoting into a system and starting one of many development VM’s is that my work regularly involves 3D graphics which already won’t work all that fluently in a VM. I can bear it on a fast local system but remote would really be a pain in the beeeehind. 🙂
I only can say: WHAT A INSANE AMMOUNT OF STORAGE!!! HAHAHAHAHA
Anyway i still preffer to use a remote programme for accessing my computer, because not always can have all the info stored on a removable drive and even dispersed on my my huge collection of drives.
As for the cloud storage i preffer to use a NAS attached to my router enabled VPN, because i dont trust in third persons, and if you have a 8 Mbps upstream it would be fine for you, its just a comment.
I don’t trust online/cloud storage either. Also, because I am legally liable for potential breaches and leaks of client data, in particular source code, I’d much rather keep it double-encrypted on local storage.
This time I had no other choice than to bring dozens of USB XHD’s with me because getting such amount of data between two offices on two different continents would take too long. Not to mention, internet/cable in the US has serious limitations (low speed, really puny compared to much of Europe, and with 100-250GB transfer limits in a month). 🙂
Yes but keep in mind that you have to handle those disks with care specially in traveling mode, anyway how do you manage to sync all the data among all your disks?
The way I traveled with the disks was easy. Instead of 1 set of files per HD there are always 2 HD’s with the same set of files (like a RAID1 mirror). One set of HD’s went into one of the suitcases and another set went along with me as carry-on luggage with the laptops + stuff in the roller bag.
This time it wasn’t a sync of any kind but rather a manual process of creating RAR compressed archives of the data on TrueCrypt volumes and then just hooking up the HD’s to two systems and telling it to copy. And yes, that took a loooong time to get prepared. Especially since filling a 1TB USB2 HD takes almost a full work-day.
“filling a 1TB USB2 HD takes almost a full work-day.” yes thats right, thats why im asking you for syncing because i have two lappies (one with OS X for phun) and a workstation and sometimes is very buggy to collect all the info
For regular sync stuff I rely mostly on a bunch of batch scripts (and PowerShell scripts) but what I did with the USB HD’s was sync (copy/mirror) all resulting archives and folders into a separate location on a server and then copied all that to the USB HD’s.