I can’t believe it has almost been 3 years since I wrote about my data storage and backup strategy! In that post, I mentioned near the end that I would be experimenting with online backup solution. To cut a long story short, I started with Mozy (crappy OS X client), then Carbonite (didn’t support external drive), and ended up using CrashPlan instead. It took a while (as in months) to back up all of my iMovie files, Aperture vault, plus other files totaling ~1.5TB to CrashPlan but it is all worth it because just over 2 weeks ago my Drobo started failing. The only reason I noticed it was failing is because I wanted to edit June’s NY Alt.NET meeting’s videos and iMovie hung on startup. Took me a while to pin it down to the Drobo/Hard Disks because I don’t have a lot of free time nowadays with 2 toddlers running around the apartment.

Turned out almost every file I tried to open on the Drobo failed with I/O error. And there didn’t seem to be a pattern as to which files are good and which are bad. Fortunately files I restored from CrashPlan seem to be fine so at least I have a good copy somewhere. After some more probing and waiting for Drobo customer support to response to my help ticket (not very helpful), I ended up purchasing the Synology DS1512+ NAS as a replacement.

Replacing my 2nd generation Drobo with a NAS has been on my plan for a while. Leah’s Macbook is getting full and some point in the future I will be replacing my 7 years old MacPro with a new machine, probably a laptop with SSD (Macbook Pro with Retina Display? Yum!) and it won’t make economic sense to store everything on the SSD or attached an external drive to a laptop. The Drobo failure just pushed the purchase decision up a few months.

The Hardware

The setup now consists of:

Synology DS1512+ with two Seagate Barracuda 7200 1.5TB and one Western Digital Caviar Green 1.5TB (repurposed from the Drobo) disks, using SHR (Synology Hybrid RAID) resulting in 2.68TB of storage, connected to the Cisco E3000 wi-fi router
MacPro now hardwired to the Cisco router via 50 feet long CAT5e cable
The NAS drive is mounted using AFP, with auto-mount setup

I’m still undecided on what to do with the Drobo enclosure and the remaining three 1TB drives. One of the drive will replace an old 500GB drive in the external drive that I use for mirroring my MacPro internal drive. I may create a RAID-0 setup for the MacPro using the remaining two 1TB drives. But then again, considering the amount of work involves I probably won’t.

The Restoration

CrashPlan

After setting up the NAS and created the volume which took less than an hour, the first thing I did is set up CrashPlan on the NAS itself. Since Synology NAS runs a variant of Linux (BusyBox), it is (relatively) easy to setup CrashPlan’s linux client and runs it headless. Relatively easy as in if you are familiar and comfortable with working in *nix terminal shell.

To administrate this headless client, I need to run the CrashPlan GUI client on my Mac and connect to the NAS via SSH. This turns out to be a little hacky. To make life a little easier, I make a copy of the GUI client, edited the necessary settings, and setup SSH local port forwarding. Even with this, I still need to open up a terminal, ssh into the NAS, then run the GUI client. I think it may be possible to script this process but I haven’t spent time to research that.

I have the CrashPlan+ Family Unlimited account and it is simply a matter of adding the Synology to CrashPlan as a new machine and started backing up files directly from the NAS.

iTunes

The first batch of files I restored were my iTunes library. The process to switch iTunes from using the Drobo to NAS was much simpler and more straightforward than I thought it would be. Simply change the iTunes library location in the Preferences. Clicked ‘Yes’ when it asked whether I want iTunes to move files to new location (this will only move some files). Then tried to play a file (any will do) and let iTunes figure out the file’s new location. Done. I didn’t use the library consolidate function because iTunes will try to copy the files to new location, which won’t work as the files were all corrupted on the Drobo.

I did use the consolidate function for Leah’s Macbook and the copy process took less than an hour over Gigabit ethernet. As the original files are left untouched at the old location, I’m planning to delete them to free up space after a week or so of usage.

Aperture

I keep the main Aperture library on local disk for speed with an Aperture Vault on the Drobo. Instead of restoring the vault, it will be much faster to just create a new vault on the NAS. Despite the fact I get pretty decent write speed to the NAS (~70Mbps vs. ~15Mbps with Drobo), it took a rather long time to create the vault. Probably due to the fact that Aperture vault consists of large numbers of both large files (images) and small files (delta edits, meta-data, etc.).

iMovie

This turns out the be the trickiest of them all for 2 reasons:

Apple leverages the HFS+ file system and stores meta-data in the file resources. Since the NAS volume is formatted as ext4 and thus does not offer this capability, iMovie will not work properly with NAS.
Apple decides to not even display the NAS volume in iMovie (most probably due to #1)

One solution is to create a disk image on the NAS that is formatted to HFS+, mount it in Finder, then move the iMovie event/project files to it. This definitely works for iMovie (and Final Cut Pro X) so I decided to keep that as an option and tried something less certain but less work. (I’m lazy, ok?)

Thanks to the power of Google, I found this undocumented terminal command that makes network volume show up in iMovie:

defaults write -app iMovie allowNV -bool true

I then moved the files to the NAS volume using iMovie just like I would with the Drobo. It took a long time for iMovie to start the file move operation but once it started the transfer speed was decent (~35Mbps). Much faster than the Drobo but no where near saturating the Gigabit ethernet.

I haven’t done a thorough check on the iMovie project yet. But the cursory check I’ve performed, it looks like this hack works. If and when Final Cut Pro X supports NAS, that will be a very compelling reason for me to upgrade.

Other files

Just straightforward CrashPlan restoration from my MacPro onto NAS volume. This will take a while as I’m only getting 9-10Mbps download speed from CrashPlan (my FiOS is 35Mbps both up and down). The restoration requires my MacPro to be running and with the heat during the day, I only run the machine during the evening so it will be at least a week before all the files are restored.

Lessons learnt

So what are the lessons from all this.

Drives fail, even redundant ones

One can only hope the RAID system will report the failure and continue to work while waiting for replacement disk. Drobo in this case did not do that. It just failed, epically and silently.

Online/offsite backup is necessary

Even with RAID system

Don’t be a cheapskate

Drink couple less Starbucks a month and you are protected.

Panic is avoidable

Because I knew I have a copy of the data safe with CrashPlan. Desperation will only come if you know there is a high chance you’ve lost your valuable data.

Fast internet connection means not a lot, if someone is throttling you

In this case, not CrashPlan but one of the backbone ISP between CrashPlan and me.

Will never buy Drobo again nor recommend it to anyone

I bought the Drobo because of its redundant feature. If a single disk failure corrupts data, I might as well go with a eSATA external drive and spare the expense. Even if Drobo sends me a replacement unit for free, there is no easy way to regain my trust again. How would I know if the new unit won’t fail silently like the last one?

After my first attempt at organizing an emergency bag (or Bug Out Bag, B.O.B.), I haven’t done anything to refine/complete the setup. This is mainly due to lack of time. However, recently I started listening to In the Rabbit Hole podcast and there are a trove of useful information. From the EDC episode, I realize my EDC (Every Day Carry) is really inadequate so last week I set out to improve it.

Previously I had a Fenix EO5 flashlight and Leatherman Squirt PS4 on my keychain. When Leah gave birth to the girls I gave her the flashlight and Leatherman for her diaper bag, and I bought myself a JETBeam BA10 and Leatherman Juice CS4 as replacement. They are obviously too big to be on the keychain so I carry them in my messenger bag for my daily commute. After listening to the podcast, I realized some important items are missing.

First, because of the increased number of items I want to carry with me, it is much easier and better organized if I store the items in a bag. I picked the Maxpedition E.D.C. Pocket Organizer and put my JETBeam and Leatherman on the left hand side. A spare battery is added for backup. A disposable lighter (I’m not a smoker but the ability to make fire is useful in any emergency) and a simple first-aid kit round out the left hand side of the organizer. On the right, I added the iPhone charger and cable so I will be able to keep my phone charged in an emergency. That plus a iPhone backup battery pack from MonoPrice and a face mask goes into the pocket. The face mask will help me through any dusty situation. Having the equipment in the Maxpedition mean I can just pick it up from my messenger bag and put it into the Timbuk2 Stork messenger/diaper bag when we go out during the weekends.

For the keychain, I added three items. First, a PocketToolX Brewzer. It is kind of mini-multitool in that it has a flat head screwdriver, a box opener, and a bottle opener. It is made of high-grade stainless steel and is TSA compliant. Then I have the LaCie iamakey USB flash drive. I can use it for carrying general files, but the feature I wanted it for is the built-in encryption software. This allows me to store scanned copies of our legal and travel documents (passports, driving license, greencards, etc.) on the flash drive securely. And since the software for decryption is on the drive, I can use it on any computer as I won’t need to download and install any software. Finally, the green tube is a glow ring. It is basically a plastic tube filled with a small amount of tritium gas (a radioactive isotope of hydrogen) which glows as it decay. Why is it useful? If I drop my keys while I was out in the dark, say while camping or a park, the glow ring will allow me to find it if it is within about 30 feet. It is not available for sale in the US and I purchased mine from Nite GlowRing in the UK.

I’ve also learnt a lot from the podcast about other items that I should include in the B.O.B. In addition to the B.O.B, I’m going to build a baby B.O.B. for the girls specifically and a mini B.O.B. for the car. All of these will be in future posts.

Category: Post

Old and new