The thing about shooting and editing video, there’s just so much data created. Heaps of the stuff.

As I write, I’m sitting on a pot of about 48 Terabytes of data, and this is growing at about 1-2 Terabytes per month. Every project sits on a disk, each disk is mirrored, and when full, ‘retired’ and put off-site. Certain jobs are archived off to BluRay data disks, other jobs get copied to USB drives and handed over to the client.

So I have a Mac that spends most of its time copying. Just sucking bits off one drive and blowing them onto another.

But a little experience got me shaken out of my little rut recently. I was doing a ‘crash edit’ job, taking rushes of a conference and editing them down into a summary – fast turnaround stuff. The conference was being recorded to DVCAM tape, and ‘in the olden days’ somebody would take note of the time when something interesting came up, the DVCAM decks would record ‘Time Of Day Code’, and therefore I could suck in ‘just the good bits’.

Many shows now get recorded to Grass Valley Turbo – a beast of a Hard Disk recorder, records in a MPEG2 variant. That means its half or even quarter of the size of DV, but cannot be edited natively. So you have to transcode it (takes longer than real-time – so why bother, stick to tape). Rick and I looked at the KiPro recently, which was great…

But…

Imagine this: a Mac’s recording DVCpro50 (near-as-dammit DigiBeta) to hard disk. At the end of each 90 minute session, it leaves a 40 Gigabyte QT movie, edit ready, on its internal hard drive. The file gets copied to the edit computer’s hard disk in about 15-20 minutes (as a backup – could even edit off the original drive over the network), editing starts immediately. Output is rendered from Final Cut Pro to hard disk ready for immediate playout.

All this is done over Gigabit ethernet. It’s just the usual Cat-5e network cable, can be run long distances, patched through to a well-designed facilities built in network, and get this: IT IS FASTER THAN FIREWIRE 800. Leaves USB for dead.

Trouble is, you need a good network engineer to configure the gigabit switches and ensure a private network (so you don’t slow down the rest of the building, or have them share your precious bandwidth). But it was truly a delight to work with, and will be trying to get this on all similar jobs.

So, back home, backing up yet another Terabyte drive, noting with newfound disatisfaction that it will take the usual 7 hours; I reminisced about the speed of Gigabit ethernet, and the fact that one can edit from a networked drive, and wondered if and how I could implement it for myself.

Going Gigabit would require a bit of investment.

However, lets start with a simple test: 1 Terabyte of mixed data (big and small files) on a standard LaCie hard drive. How long to dump that lot onto another hard drive?

USB2 – 16 hours.
FireWire 800 – 7 hours.
Gigabit ethernet – 4.5 hours (extrapolated from 40 GB files)
eSata PCIexpress – 3 hours
LTO-5 tape backup – 2 hours (assuming uncompressed) via iSCSI

So I bought a LaCie eSata card for the MacBook Pro to better than halve the time it takes to back up a drive.

Though the Gigabit ethernet is a wonderful technology when on-site or where ingest and edit are split apart by more than a few meters.

So why not USB-3? Well, all my main drives have eSATA sockets on them, so USB-3 – whilst being a great technology – isn’t quite prime time for me yet. When the next round of Mac laptops come out, they should have USB-3 compliant ports, so that will encourage more drives to be released in this format, and so the ecology will generally drift that way. The LaCie Rugged USB3 is a good start.

Why not LTO-5? Well, the base drive is £3k, and the tapes cost more than bare hard drives. I can get a Samsung Terabyte bare drive for £50, and an LTO-5 tape for £70. They’re easier to store, but they don’t ‘unarchive’ easily, and if I want higher capacity, I have to pension off the expensive drive.

So, right now, I have 30 drives with eSATA as well as FW800 ports, and for the minuscule investment of £40 for a dual channel SATA card for my MacBook Pro, I get to halve my duplication time. If a hard drive goes south, I can pop the backup in the ‘toaster’ and continue until my replacement drive arrives, then copy across.

But of course, I have to pull out the eSATA card if I want to use my SxS or SDHC cards… (Drums fingers) and if I didn’t have a17 MacBook Pro, all this would be theoretical. (Drums fingers again) Come on, Apple, get with the USB-3 equipped MacBook Pro…

PS: You do need a dual channel card if you’re using SATA to back up – unlike FireWire, you can’t daisy-chain, and whilst there are ‘SATA Backplanes’ that work a little like USB hubs, it is not really the same thing.