Tale of a data paranoid

Ever since I had my first hard disk failure about 11 years ago and lost a substantial portion of my data, I am more concerned with data back than most computer owners/users. Every new computer that I purchased since then had also included provision for data backup. I started out with magnetic tapes that while worked, took long time to back up even a moderate amount of data. Not to mention the longevity of the tapes were always in question. Even with the popularization of CD-ROM/RAM or even DVD-ROM/RAM, I never used it as a backup medium. Capacity is too low and unless I paid for top quality disks, longevity again was suspect.

In the last few years since I switched to OS X, I have been using external hard disks as a way to backup my data. The main catalyst are applications such as Carbon Copy Cloner and SuperDuper! which mirror a drive to another exactly. The advantage of unix-based OS X means that once my drive is cloned, I can actually boot up from the external drive and use it from there. Try doing that with a Windows machine!
Hard disk has its own issue though. With the advent of large capacity hard disk, that means I have to constantly purchase new external drive to accommodate any new internal drive, leaving a number of smaller and less useful disks around the apartment. And with the arrival of Time Machine in Leopard, now I need double the capacity of my internal drive so I can have both Time Machine and clone.
This is clearly a non-scalable solution.
I first heard of Drobo when I saw a video demo/interview of the Drobo founder by Robert Scoble. At the time I thought it was an interesting way to make RAID/NAS drive simpler to use, but I felt that the price was too steep for just an fancy enclosure. Fast-forward to last week when I recorded the first ALT.NET meetup and ended up with nearly 2 hours of DV video files. Suddenly the free space on my 250GB internal drive in my Mac Pro shrunk to almost nothing.
This time I look for a more scalable solution. RAID is an option but due to the stupidity of Apple it is not trivia to add software RAID or cheap to add hardware RAID. Also, the need to decide which type of RAID I want to setup seem to defeat the objective I want a Mac in the first place; to get work done instead of tinkering with software/hardware. So I give Drobo another look and found that while it may not be as integrated as RAID, the flexibility it offers more than offset that. The ability to easily add new drive(s) to expand my storage capacity means I don't have to decide on how large a disk I should purchase every time I need an upgrade. Now, I just buy a new drive, put it in to Drobo and that's it. No need to worry whether I should do RAID 0, RAID 1, or RAID 5, etc.
So how easy it is to get Drobo setup?

I purchased the generation 2 Drobo with 2 x 1TB drives package on the Drobo Store. After unpacking everything it was simply a matter of hooking up the power supply and the FW800 cable to the back of the Mac Pro, and pushed the two drives into the slots. Then I installed the Drobo Dashboard from the CD and formatted the Drobo "drive". One of the question the Dashboard asks is what size should Drobo drive be formatted at. I was given the choice of 1, 2, 4, 8, and 16TB. Logic or even common sense would say that with two 1TB drives I should pick 2TB option. But then I remembered Leo Laporte mentioned this on one of the MacBreak Weekly podcast that the best option is 16TB no matter how much storage is actually in the Drobo. Why? Because this mean I won't need to re-formatted the drive in the future or have multiple "drives" shows up in the OS when I put in more storage capacity.
Then it was a few minutes wait while the Dashboard "formats" the Drobo and rebooted it. Once that is done, the Drobo shows up in OS X as an external drive of 16TB. The Dashboard also offers to format the drive as HFS+ but somehow it didn't work for me. I had to use Disk Utility to partition and format but that was easy. (I create two partitions: one 250GB for Time Machine, and one that occupies the rest for data)
And that's it really. Total time from unpack to having a 16TB drive on my desktop? 30 minutes max. What is more time consuming is the data transfer from the internal drive to Drobo. The FW800 connection is really fast but it is not infinite! It still takes around 20 minutes to transfer my 40GB Windows XP VM while Time Machine is also backing up to the Drobo. Having said that, the connection is definitely fast enough to use Drobo as a primary drive. I have no problem running XP VM off Drobo directly. In fact, it is slightly smoother now because the VM is no longer on the internal drive, blocking OS I/O. The acid test would be to edit HD video directly from Drobo.

One thing I do take time and care is the transfer of my media files. Having lost iTunes metadata before, this time I look up the instruction and do it "properly". Here are the three links that explains how to transfer:

Read and post comments |
Send to a friend

NY ALT.NET First meeting: OR/M

Tonight was the first NY ALT.NET meetup and I think it has gone very well. Over 30 people turned up and many have not participated in a fishbowl style discussion before. Stephen Bohlen first gave a quick, 10 minutes, overview of OR/M. Then after a quick pizza break Don Demsak, Stephen Bohlen, and Mark Pollack kicked off the discussion and soon more people joined in the fishbowl experience. After the meeting, I talked to a few attendees and all of them gave very favorable feedback about this more interactive style of meetup, verses the traditional presentation by a speaker style.
On the technical side, we decided to stream the video live after we received a few requests from people who couldn't attend in person. The logistic of setting a stream up using Ustream.tv is trivia but there are a huge amount of fingercrossing and wood touching because everything had to come together at the same time:
  • Mark's DV camera talks to my Macbook Pro via Firewire and record to tape at the same time.
  • Ustream.tv's player recognized Mark's camera
  • Ability to get onto Microsoft's guest wi-fi network
  • Ability to find a place to put the camera that doesn't get into people's view but close enough to get decent sound reception
Having said that, I can't imagine streaming live video on the internet to the public 5 years ago (or even 2!) without huge infrastructure expense, lots of testings, and poor results. Now, it is free, easy to setup, and provide great user experience.
Here is Stephen Bohlen gave an overview of OR/M before the main discussion (~10 mins):

The main discussion (~1 hr 47 mins):

Read and post comments |
Send to a friend

NYC ALT.NET user group

A group of like minded .Net developer and I have started a local ALT.NET user group for the NYC area. Our first meeting will be held on Thursday, September 25th, 6:30 PM at the Microsoft office in midtown Manhattan.

The topic for the first meeting is: Object-Relational Mapping: the philosophies and the tools. More details can be found on our user group web site: http://nyalt.net/ or you can RSVP directly on the meetup.com page.

Read and post comments |
Send to a friend

What? A year already?

How time has flied! It has been a year since I joined ThoughtWorks. In the last 12 months, I've:

  • Visited ThoughtWorks India in Bangalore
  • Worked in 3 totally different projects (.Net desktop app, build & deploy at an enterprise level, and Agile coaching) in 3 different locations (London, Connecticut, New York)
  • Became a committer of an Open Source project, DbDeploy.net
  • Attended ALT.NET conference in Seattle
  • Started ALT.NET NYC with a bunch of great, like-minded developers
  • Edited 4 ThoughtWorks IT Matters podcasts
  • Helped out with recruitment (code reviews, office interviews, phone interviews)
Skills and tools that I've learnt/developed that I didn't think I would otherwise:
  • Agile project management, planning, estimation (Mingle)
  • Presentation and coaching (PowerPoint!, public speaking)
  • Build & Deploy (CruiseControl.NET, Cruise, TeamCity, NAnt, MSBuild, PowerShell, etc.)
  • iPhone development
  • Mocking (NMock, Moq)
  • UI Automated testing (Selenium)
What I can't quantify is the amount of learning I've done during this time. Working with smarter people than myself drove me to self-improvement in ways that I did not think I can or would.
I don't know what would happen in the next 12 months but one thing for sure, I would definitely enjoy it.

Read and post comments |
Send to a friend

Website Built with WordPress.com.

Up ↑

%d bloggers like this: