Tale of a data paranoid

Ever since I had my first hard disk failure about 11 years ago and lost a substantial portion of my data, I am more concerned with data back than most computer owners/users. Every new computer that I purchased since then had also included provision for data backup. I started out with magnetic tapes that while worked, took long time to back up even a moderate amount of data. Not to mention the longevity of the tapes were always in question. Even with the popularization of CD-ROM/RAM or even DVD-ROM/RAM, I never used it as a backup medium. Capacity is too low and unless I paid for top quality disks, longevity again was suspect.

In the last few years since I switched to OS X, I have been using external hard disks as a way to backup my data. The main catalyst are applications such as Carbon Copy Cloner and SuperDuper! which mirror a drive to another exactly. The advantage of unix-based OS X means that once my drive is cloned, I can actually boot up from the external drive and use it from there. Try doing that with a Windows machine!
Hard disk has its own issue though. With the advent of large capacity hard disk, that means I have to constantly purchase new external drive to accommodate any new internal drive, leaving a number of smaller and less useful disks around the apartment. And with the arrival of Time Machine in Leopard, now I need double the capacity of my internal drive so I can have both Time Machine and clone.
This is clearly a non-scalable solution.
I first heard of Drobo when I saw a video demo/interview of the Drobo founder by Robert Scoble. At the time I thought it was an interesting way to make RAID/NAS drive simpler to use, but I felt that the price was too steep for just an fancy enclosure. Fast-forward to last week when I recorded the first ALT.NET meetup and ended up with nearly 2 hours of DV video files. Suddenly the free space on my 250GB internal drive in my Mac Pro shrunk to almost nothing.
This time I look for a more scalable solution. RAID is an option but due to the stupidity of Apple it is not trivia to add software RAID or cheap to add hardware RAID. Also, the need to decide which type of RAID I want to setup seem to defeat the objective I want a Mac in the first place; to get work done instead of tinkering with software/hardware. So I give Drobo another look and found that while it may not be as integrated as RAID, the flexibility it offers more than offset that. The ability to easily add new drive(s) to expand my storage capacity means I don't have to decide on how large a disk I should purchase every time I need an upgrade. Now, I just buy a new drive, put it in to Drobo and that's it. No need to worry whether I should do RAID 0, RAID 1, or RAID 5, etc.
So how easy it is to get Drobo setup?

I purchased the generation 2 Drobo with 2 x 1TB drives package on the Drobo Store. After unpacking everything it was simply a matter of hooking up the power supply and the FW800 cable to the back of the Mac Pro, and pushed the two drives into the slots. Then I installed the Drobo Dashboard from the CD and formatted the Drobo "drive". One of the question the Dashboard asks is what size should Drobo drive be formatted at. I was given the choice of 1, 2, 4, 8, and 16TB. Logic or even common sense would say that with two 1TB drives I should pick 2TB option. But then I remembered Leo Laporte mentioned this on one of the MacBreak Weekly podcast that the best option is 16TB no matter how much storage is actually in the Drobo. Why? Because this mean I won't need to re-formatted the drive in the future or have multiple "drives" shows up in the OS when I put in more storage capacity.
Then it was a few minutes wait while the Dashboard "formats" the Drobo and rebooted it. Once that is done, the Drobo shows up in OS X as an external drive of 16TB. The Dashboard also offers to format the drive as HFS+ but somehow it didn't work for me. I had to use Disk Utility to partition and format but that was easy. (I create two partitions: one 250GB for Time Machine, and one that occupies the rest for data)
And that's it really. Total time from unpack to having a 16TB drive on my desktop? 30 minutes max. What is more time consuming is the data transfer from the internal drive to Drobo. The FW800 connection is really fast but it is not infinite! It still takes around 20 minutes to transfer my 40GB Windows XP VM while Time Machine is also backing up to the Drobo. Having said that, the connection is definitely fast enough to use Drobo as a primary drive. I have no problem running XP VM off Drobo directly. In fact, it is slightly smoother now because the VM is no longer on the internal drive, blocking OS I/O. The acid test would be to edit HD video directly from Drobo.

One thing I do take time and care is the transfer of my media files. Having lost iTunes metadata before, this time I look up the instruction and do it "properly". Here are the three links that explains how to transfer:

Read and post comments |
Send to a friend

PowerShell rescues wedding photos

One of my friend reached out to me last week for help. The photographer she hired for her wedding had gone bankrupted. Since she has not gotten the album now she can't even get her wedding photos in either prints or digital form. All she has access to is the photographer's web site where the people can order prints from. She was hoping that I can somehow retrieve all the photos (~1220) from the site so that at least she would have digital copies of her wedding photos.

After poking around the site a bit, I found out that I was in luck. Sort of. All the photos are accessible but only in low resolution (500 x 366). But more importantly for me, the site developer had opted to have a nice AJAX interface and for some reason decided to include all the image URLs in the page!
So a plan of action quickly formed in my head:
1. Download the page source
2. Extract the image URLs from all the HTML/Javascript code
3. Download all the images, one by one
Initially I want to try out Automator on my Mac and see how easy it would be to do this, since this is exactly the type of repetitive tasks Steve Job told us Automator  is perfect for! Unfortunately after poking around for 30 minutes, I quickly came to the conclusion that Automator is woefully inadequate for this task. So I turned to PowerShell on my Windows VM instead. Having used PowerShell 2.0 CTP a lot in my last project, I was able to quickly develop a script that extracts URLs from the page source, download the file, and save it to disk. In fact, the most time consuming part is to figure out the regular expression for URL extraction!
From start to finish, it took me around 90 minutes to get all the photos downloaded. If I didn't waste time with Automator and better at regular expression I think I can do it in 15 minutes!
Now my friend can have her wedding photos. She may not be able to print them out, but at least she can view them on a computer screen.

Read and post comments |
Send to a friend

MovableType vs. WordPress

There are many discussions on the web about which one is better, and why one is more popular with a certain group of users than the other. Here is what I think why WordPress is more popular among the causal blogger than MT.

Note #1: I think MovableType is, in general, a better blogging system (e.g. more scalable, more flexible architecture, etc.) but oftentimes technical merits do not count for much when causal users evaluate software.

Note #2: I am comfortable in unix shell so that's not where I burnt my time on.

Fact: I've spent around 4 hours trying and failed to install the infrastructure (CGI and Perl's DBI and DBI::mysql or DBI::sqlite) on OS X to get MovableType to run.

Fact: I only need to spend around 30 minutes to get WordPress to run.

Read and post comments |
Send to a friend

Wolf in sheep’s clothing

After having been coding in Windows inside OS X, using Parallels, on my Mac Pro for a day and half I still feel very weird about the whole thing. (Getting my laptop's Windows installation transferred into a Parallels' virtual hard disk image was a long and tedious process and I won't bore you with the details)

Using Windows inside OS X just feel plain strange at the moment but I like it so far. Being able to run both OSes at the same time is so much more convenient, I don't have to switch screen/keyboard/mouse inputs, or keep two sets of emails and web bookmarks, and I can just drag-n-drop files between the two OSes. Despite running inside a vm, Visual Studio 2005 works very well. It runs and compiles almost as fast as on my Core 2 Duo 2.4 GHz Dell with RAID 0 drives in the office but I think the 2.66 GHz Xeon processor helps! 🙂

Anyway, the Mac Pro is everything that I have hoped for. Yes it is expensive but given my last mac lasts over 4 years I think it is a good investment. It does mean that I have to forgo the Xbox 360 for another 6 months or so but I don't have time to play game that much anyway. In fact, my Mac Pro is so good that I spent most of my time on the mac and didn't play even one second of Halo this weekend, normally the designated Halo time of the week.

Read and post comments |
Send to a friend

Website Built with WordPress.com.

Up ↑

%d bloggers like this: