Live long and prosper

I’m on the sunny side of a disaster. Backups and archives functioned, and my Mac systems are back at cruising altitude. But, lest that one should sound smug, I am aware that archiving is an interesting case in point, especially for an EX1 owner.

I’m using hard disks, with archive to DVD-R and now Blu-Ray. That works out well – in that everything is on hard disks, and every project has double Blu-Ray media managed backup, and if the footage isn’t on tape, Blu-Ray backup too (though I back up the EX1’s QuickTime files, not the original data – oooh, I can hear the sharp intake of breath from here).

But that’s not what’s on my blog-radar today. What has caused a blip is reading a little note from the BBC News website:

http://news.bbc.co.uk/1/hi/programmes/click_online/7525143.stm

Yes – film is falling apart.

Sure, Star Wars was rescued, re-scanned, twiddled and is now a lot of ones and zeros on a great many hard disks. Lots of other films have gone or are going through this process.

But going digital can cause problems: what if the format you store it on becomes redundant?

I have this problem. I have a project dear to my heart – an interactive touch screen application I made more than 12 years ago. I used Macromind Director 4 then, and I updated it to 5, 6, and MX. But now its projectors do not run on Macs. I cannot open the older files in new versions of Director. Such is the pace of progress, the files are too old – Do Not Resuscitate.

That’s the fear of many archivists. How do you open an image saved from a ZX81? How can you grab an original visicalc file to present today? Digital files change so quickly, and so many works are in esoteric custom formats. JPEG and MPEG may be safe for now, but for how long?

The film industry have a solution: 35mm! Common standard, easy to store, can’t erase it by mistake, could last for ages if looked after.

But wait, aren’t we trying to resuscitate 35mm originals that are rotting before our eyes? Ah, but that was old fashioned film – not new tough film, but sloppy stuff made from recycled cow hooves. Today’s material is far more scientific. But expensive.

So what of all those careless snaps now, which will mean so much to generations hence? Not the major epics, but those little glimpses into a century forgotten. Hundreds of years from now. No printouts from your desktop are going to survive like those ancient sepia bromides. No CD-ROMs are going to survive unless hermetically sealed in a protective gas and shaded from light.

But digits are digits. I had another project in Director that I diligently upgraded, and copied, and exported and re-saved. And its still with me. Not sure if it was worth saving, but it’s still there. And that seems to be the secret: revisit your archive, copy it, view it, don’t just back it up. Use open standard formats. Use widely accepted media. I still have Syquest drives with data on it. The data isn’t quite worth spending specialist money on it. The contents aren’t particularly earth shattering. Not now. But in 300 years? Who knows? What’s worth saving?

And has anyone tried buying VHS tapes recently?

alt.edit.final.final.final

It’s the last 10 yards of an edit. Well actually, today it’s two edits symultaneously.

Unlike, I imagine, big beefy long-form edits, corporate edits spend the first 80% of their edit being chopped together with major things happening, large scale grading and audio finessing, and then spend the next 80% of the time being tweaked to death. A caption update here (oopsie – re-render), a shuffle of sections there (necessitating a complete reshuffle of the music edit), make this slower, make that faster. It’s what we do for a living, and quite frankly it’s the shift from the ‘Director’s cut’ to the ‘Release Print’. Not all Director Cuts are, if the truth be told, ‘better’.

So I’m doing major version controllage on filenames, i.e. I export my FCP movie as ‘Main edit – final’, then there’s a slight change, so it becomes ‘Main edit – final final’ and so on. Yes you do, don’t get all coy, we all do it. So we start being all professional about it and using numbers for version control, and the real ‘greybeards’ start their versions with a leading zero, knowing that all final edits end in double figures.

All this is fine. What gives me the time to vent frustration on this blog whilst YET ANOTHER version of my edit(s) ooze out of FCP is that rendering as you go can lead to problems. Rick – we’ve both been here, and the mystery is in rogue render files shared between sequences within a project.

When making tweaks to a sequence, requiring outputs two, three or four times a day, it feels good to render everything out and allow only the bits you tweak to change. Saves lots of time.

But I’ve found something horrible: what you see in the Canvas window does NOT necessarily equate to what you’ll get in the exported sequence. I had an imported movie from Google Earth Pro, which had a glitch in it, and an overlaid graphic sequence from Motion. Everything played fine within FCP. When I exported it, haunting flashbacks from non-working versions over-wrote what I saw in the Canvas. As soon as a transition started, the background (google earth pro movie) snapped from the latest version to an earlier lame version. So tweak the transition by a frame, force a re-render, but the export was the SAME. Ditto other glitches.

Bottom line: I’ve tried to flush out ALL renders for the whole project in order to get the Export to Quicktime (make self contained) work. Nope. Perhaps I should have checked the ‘re-render’ box instead, but Nope. Glitches still existed from phantom renders – and I think it was to do with other sequences in the same project using the same renders, therefore they DON’T get deleted even if you ask them to.

I finally got what I wanted by exporting to a different codec, making it self contained and re-rendering the whole thing. Took six times longer to do the final export of course, so only do it if you have to, or as your final stage. I wouldn’t want to do that for every output, but if what you see in FCP and what you get in export don’t match, do this.

So that’s my lesson today: all ‘final final final final’ movies should be rendered out from scratch in a lossless codec otherwise the ghost of renders past will be hanging around like a fart in a space suit.

Graphic artists borrow, artists steal

Sorry to inaccurately paraphrase Mr Picasso, but I’m trying to excuse the fact that I’m watching Top Gear. With a notebook. I’m pretending to be a petrol head, but I’m watching the editing like a hawk. Or a hungry chicken. Whatever.

I’m watching it through the BBC iPlayer. I can’t stand broadcast TV any more – I don’t just want to pause live TV, I want to be able to stop it, walk away, mow the lawn, make wife some tea, watch something else, then pick up where I left off. Frame by frame if I feel like it. iPlayer rocks even if it’s a fraction of the technical quality of broadcast TV. Broadcast TV and schedules and adverts and ‘did you see last night’ are so dead… But I digress.

It’s automotive pornography, it’s without any useful educational content, it’s rather divisive (ho ho! In the extreme dear friend), but the pictures are pretty and the editing is exciting. And there’s lots of bits of metal that go ‘brum’ loudly.

I don’t edit long segments about cars, but why am I occasionally pausing the video and going through it frame by frame, working out what’s done in-camera and what’s in post? Why am I making mental notes about ‘the sound of transitions’? I’m analysing the number of frames in sequences and in beard stroking moments, watching how edits contain more and more sub-15-frame content and ‘glitch’ edits. Pixelation is no longer a ‘whoah’ thing, it represents DV drop-out to the audience, a moment when tape and drum did not connect. So camera shake and rolling frame is out. A ‘blik!’ sound effect and a few random pixellated areas and perhaps a flash frame or two.

Our children are watching this too. My four year old son thinks that fire engines go ‘nee nahh nee nahh’, but of course they don’t – they never have in his lifetime. They go – well, you know what they do. It’s like green screen text in The Matrix – does anyone under forty who is NOT a geek know what a command line is? More to the point, what will my 4yo son make of film scratches, film jumping the gate in projectors? He sees glitches and freezes, he hears the ‘stut-stut-stut-stut-stuttering’ of internet movies starting up in iPlayer. The buddy-blocks of blown bandwidth.

It doesn’t stop there. He associates the ‘washing machine from hell’ flash-loading icon with a busy day for the internet. Ye gods, forget the Oracle of Delphi and the ides of March, we have the spinning beachball of death and the washing machine from hell to tell us it’s a bad IP day for mortals.

Top Gear has been educating an audience with a visual style that’s abrasive (like rinsing your eyes in mouthwash), fresh, dynamic (and very IP unfriendly) – and I’m finding my edit style adapting to match. It’s all very ‘now’, very ‘cold shower’, very ‘mouthwash’ and ‘9 volt battery on your tongue’.

So when I had the chance to show my 70+ aunt my current show reel (it was that kind of afternoon), she got it totally.

Which leads me to a giddy pontification: when octogenarians are totally into blipvert editing, where do we go next?

It's all in the recovery

I had a hard disk failure the other day. MacBook Pro. Happened without warning when on the phone to client. No warning. Just like a stroke. Sudden, devastating, terminal.

The next day, I’m due to fly out for an EX1 shoot, requiring the transfer of its SxS cards into a compatible device – the MacBook Pro being perfect for this. I do have a backup Mac – a MacBook, but it has no PCI Express slot. And as backup machines go, it can’t really do Colorista, Motion, DVmatte Pro (all require GPU). And so although I have software backups, the hardware isn’t really a backup. So within three hours of the Spinning Beachball of Death, I owned a brand new MacBook Pro 17″.

Lesson 1:
If your income is dependent on a certain type of computer rather than a computer per se, have two of them. Not a posh one and a skivvy one. My backup was a helper, an ‘it’ll do, it can help out’. But if I need to load up a project full of colour correction, esoteric codecs, and (since my main machine is dead and gone) my current copy of Final Cut Pro, a skivvy computer will say ‘no’.

I call AppleCare – I purchased the full-on AppleCare package for my Mac. Sure, they will take it away, replace the hard disk and send it back – without transferring data, so it will be basically factory fresh. This may take up to three weeks. Three WEEKS? Three days would be a disaster. Apple’s response will be ‘just use your backup’. I pay for rescue: if my car breaks down, I call a number, and somebody arrives within an hour or two and gets me going. I thought I paid for this service for my Mac, but no. AppleCare is not the AA. I wanted to turn up at a shop, for some kind person to rip out the old drive, put a new one in, and hand my Mac back with a hard disk I could have in a USB enclosure if it should ever work again. What I got was 30 minutes of phone support reiterating everything I’d spent two hours doing, and a courier firm who always phones when I’m out and doesn’t want to call mobile numbers.

I’ll have the old MacBook Pro repaired, and it will be backup hardware for when my main machine calls in sick.

So, I now have a brand new MacBook with a nearly empty hard disk. I also had loads of backups spread across 30+ hard drives – but I didn’t know which one because in order to use the disk cataloging software, I’d have to install it in the new machine. So I did, and I found a recent one. Great. I attached the hard disk and used Migration Assistant. Oh.

When you set up a brand new Mac, you’re asked to set up a user account, which I duly did. This is me, this is my password. Okay, so it was the same as the previous one, otherwise the new software would be frumpy being on a new Mac and all… But I can’t restore my old self, as I can’t restore my old ‘Me’ over my new ‘Me’. I don’t want to make a new me (Me1 rather than Me) as it sounds so lame to be a secondary also ran on one’s own computer.

So that leads me to Lesson 2:

When in possession of a new Mac, make TWO accounts. The first one is a disposable admin account. Nothing to see, nothing to do. Most importantly: It is NOT personal. Just make it as plain as you can. From that, THEN restore your personal account – your avatar of Macness – into the virgin machine. The funny thing is that this is Computer Administration 101 stuff. Of course that’s how corporate machines are set up. I should know that – I used to do it myself. But somehow, when you’re a one-man-band, the lessons of Big Corporate IT don’t seem to apply. But they do if you don’t want to… Spend the next X hours reducing your brand new Mac with patiently set up software back to its virgin state so you can try again…

Right, so the new Mac is virgin again and as it boots, it swirls the Apple Welcome message. I set up an Admin account. I can now migrate my old entity to the new machine.

This can be done from a Time Machine drive, or from all sorts of third party solutions like SuperDuper (http://www.shirt-pocket.com/SuperDuper), but then there’s version control. If your backups are spread across many drives to ensure no single point of failure, which one do you use? Well, from personal experience, not the one with the most recent modification date.

I restored from a backup that appeared to be from a week ago, but actually was six weeks earlier than that. Okay, no problem, I thought. I can restore other stuff to make up for the time difference. Email, documents, etc – they’re all done separately, so no problem there.

But here’s the crunch: I’d inadvertently restored from a backup BEFORE a major system change. Thusly, I had re-inherited all the little issues my updates had cured. Due to the time pressure of getting things done now, getting up and running as soon as possible, I had got from hard disk failure and attempts to rescue, to fully operational machine fresh out of shrink wrap in a day. No data loss, no info loss, but…

That leads me to Lesson 3:

Look after your tools like you look after your data. We all back up our work. It’s critical. Redundant backups everywhere. We lose no data. Tools? Heck, I’ve got the DVD install disks. I’ve got the URLs and the serial numbers. My hard disk’s tool kit is backed up every so often – bleah. Whatever. A fresh install cures all.

Fresh installs take time – lots and lots of time. And pain. And frustration. It’s a chance to make the previous installation BETTER by applying learned lessons.

So I restored my data from fresh backups and it’s all good. I restored my tools from a six week old backup, and it’s pants. I inherit a whole lot of dross that I solved ages ago. Back up your tools like you back up your data.

Lesson 4:

I love SuperDuper so much, it’s got me out of nasty situations and helped me no end. I don’t trust Time Machine as I need to know that there’s no silly gotchas in the restore process. But here’s the kicker, folks: If I had Time Machine running on a cheap USB drive when I was working at home (SuperDuper does the abroad stuff), I’d have saved the four hours it’s going to take me to reinstall FCS, Leopard and the rest to make my tools work as they should ‘out of the box’.

Summary:

  • If you earn money from your Mac, own two of the same (or thereabouts)
  • Always have two accounts on your Mac: You and Admin
  • Don’t get obsessive about backups – get regular about everything
  • Time Machine is better than it looks

 

But on the other hand, my new machine was budgeted for, has twice the RAM, twice the Hard Disk, twice the GPU. Sorry I didn’t get a maxed out iMac or a base line Octo-Core Mac Pro – but that’s how the Education By Fate works: great lessons, bills are kinda high.

Death of a hard disk

I had a hard disk failure the other day. MacBook Pro. Happened without warning when on the phone to client. No warning. Just like a stroke. Sudden, devastating, terminal.

The next day, I’m due to fly out for an EX1 shoot, requiring the transfer of its SxS cards into a compatible device – the MacBook Pro being perfect for this. I do have a backup Mac – a MacBook, but it has no PCI Express slot. And as backup machines go, it can’t really do Colorista, Motion, DVmatte Pro (all require GPU). And so although I have software backups, the hardware isn’t really a backup. So within three hours of the Spinning Beachball of Death, I owned a brand new MacBook Pro 17″.

Lesson 1:
If your income is dependent on a certain type of computer rather than a computer per se, have two of them. Not a posh one and a skivvy one. My backup was a helper, an ‘it’ll do, it can help out’. But if I need to load up a project full of colour correction, esoteric codecs, and (since my main machine is dead and gone) my current copy of Final Cut Pro, a skivvy computer will say ‘no’.

I call AppleCare – I purchased the full-on AppleCare package for my Mac. Sure, they will take it away, replace the hard disk and send it back – without transferring data, so it will be basically factory fresh. This may take up to three weeks. Three WEEKS? Three days would be a disaster. Apple’s response will be ‘just use your backup’. I pay for rescue: if my car breaks down, I call a number, and somebody arrives within an hour or two and gets me going. I thought I paid for this service for my Mac, but no. AppleCare is not the AA. I wanted to turn up at a shop, for some kind person to rip out the old drive, put a new one in, and hand my Mac back with a hard disk I could have in a USB enclosure if it should ever work again. What I got was 30 minutes of phone support reiterating everything I’d spent two hours doing, and a courier firm who always phones when I’m out and doesn’t want to call mobile numbers.

I’ll have the old MacBook Pro repaired, and it will be backup hardware for when my main machine calls in sick.

So, I now have a brand new MacBook with a nearly empty hard disk. I also had loads of backups spread across 30+ hard drives – but I didn’t know which one because in order to use the disk cataloging software, I’d have to install it in the new machine. So I did, and I found a recent one. Great. I attached the hard disk and used Migration Assistant. Oh.

When you set up a brand new Mac, you’re asked to set up a user account, which I duly did. This is me, this is my password. Okay, so it was the same as the previous one, otherwise the new software would be frumpy being on a new Mac and all… But I can’t restore my old self, as I can’t restore my old ‘Me’ over my new ‘Me’. I don’t want to make a new me (Me1 rather than Me) as it sounds so lame to be a secondary also ran on one’s own computer.

So that leads me to Lesson 2:

When in possession of a new Mac, make TWO accounts. The first one is a disposable admin account. Nothing to see, nothing to do. Most importantly: It is NOT personal. Just make it as plain as you can. From that, THEN restore your personal account – your avatar of Macness – into the virgin machine. The funny thing is that this is Computer Administration 101 stuff. Of course that’s how corporate machines are set up. I should know that – I used to do it myself. But somehow, when you’re a one-man-band, the lessons of Big Corporate IT don’t seem to apply. But they do if you don’t want to… Spend the next X hours reducing your brand new Mac with patiently set up software back to its virgin state so you can try again…

Right, so the new Mac is virgin again and as it boots, it swirls the Apple Welcome message. I set up an Admin account. I can now migrate my old entity to the new machine.

This can be done from a Time Machine drive, or from all sorts of third party solutions like SuperDuper (http://www.shirt-pocket.com/SuperDuper), but then there’s version control. If your backups are spread across many drives to ensure no single point of failure, which one do you use? Well, from personal experience, not the one with the most recent modification date.

I restored from a backup that appeared to be from a week ago, but actually was six weeks earlier than that. Okay, no problem, I thought. I can restore other stuff to make up for the time difference. Email, documents, etc – they’re all done separately, so no problem there.

But here’s the crunch: I’d inadvertently restored from a backup BEFORE a major system change. Thusly, I had re-inherited all the little issues my updates had cured. Due to the time pressure of getting things done now, getting up and running as soon as possible, I had got from hard disk failure and attempts to rescue, to fully operational machine fresh out of shrink wrap in a day. No data loss, no info loss, but…

That leads me to Lesson 3:

Look after your tools like you look after your data. We all back up our work. It’s critical. Redundant backups everywhere. We lose no data. Tools? Heck, I’ve got the DVD install disks. I’ve got the URLs and the serial numbers. My hard disk’s tool kit is backed up every so often – bleah. Whatever. A fresh install cures all.

Fresh installs take time – lots and lots of time. And pain. And frustration. It’s a chance to make the previous installation BETTER by applying learned lessons.

So I restored my data from fresh backups and it’s all good. I restored my tools from a six week old backup, and it’s pants. I inherit a whole lot of dross that I solved ages ago. Back up your tools like you back up your data.

Lesson 4:

I love SuperDuper so much, it’s got me out of nasty situations and helped me no end. I don’t trust Time Machine as I need to know that there’s no silly gotchas in the restore process. But here’s the kicker, folks: If I had Time Machine running on a cheap USB drive when I was working at home (SuperDuper does the abroad stuff), I’d have saved the four hours it’s going to take me to reinstall FCS, Leopard and the rest to make my tools work as they should ‘out of the box’.

Summary:

  • If you earn money from your Mac, own two of the same (or thereabouts)
  • Always have two accounts on your Mac: You and Admin
  • Don’t get obsessive about backups – get regular about everything
  • Time Machine is better than it looks

 

But on the other hand, my new machine was budgeted for, has twice the RAM, twice the Hard Disk, twice the GPU. Sorry I didn’t get a maxed out iMac or a base line Octo-Core Mac Pro – but that’s how the Education By Fate works: great lessons, tuition bills are kinda high.

The Old Army Colonel And His Son On Holiday

Another edit is in the can. In approximately 9 hours, I’m off to do a Z1 shoot and really wishing it could have been an EX1 job. Okay, so I could shoot HDV – but the shots would stand out like trout in a fishbowl.

Whatever. No. Tonight, whilst I compress FLVs and upload them, I’ve been going through Ripple Training’s Deep Dive course – all about Motion’s 3D. Lots of Alphabet street and ‘which way is up’ moments. Ages ago, I did a fair bit of early After Effects and even Specular’s Infini-D, but often got lost. I had to get some ‘Doe, a deer’ rules.

So I thought I’d share these silly mnemonics for those who only sporadically dip their toe into the 3D universe:

“X is a cross” (as in across, left and right, geddit?), so therefore Y is uppY downY. And “Z is like Zoom” in and out. Not strictly accurate, but when it’s late and you’re reaching for the right (wrong) slider…

And for Motion,

“Red-X” or “Red Cross”, “Green trees grow up”, “Blue oceans into distance”.

Of course this is all second nature to Motion Graphics designers, and sure – I can reel off pixel aspect ratios and Composite modes in a flash, but if you’re not doing it every day or even every month, we all have those ‘The Old Army Colonel And His Son On Holiday” moments.

If you still remember your Cosines from your Tangents, that is.

Bridge to Engine Room: We need more power!

Having switched from DV (and DV derived from HDV) to the EX1’s 720p, the quality is wonderful, but suddenly there’s lots to re-learn. Okay, so we’ve got 720p25 to progressive PAL worked out. That’s all fine and dandy. Good quality, easy peasy, quick and efficient. Move along, nothing to see… (it’s all about ensuring FCP does not try to do anything interlacy with your 720p footage – see Rick’s article on Ken Stone’s site).

But today I’m asked for some rushes from my 720p25 shoot as an NTSC DVD. Hey, no problem, just run it through DVFilm Atlantis (or  alternatives such as Graeme Nattress’s Standards Conversion plug-ins) and… Oh.

Glaringly obvious point, but of course most of these processes assume interlaced video on both sides of the conversion, taking their motion queues from the extra time info that’s not present in 720p25. The results limp along with blocky scaling and a definite cadence to dropped frames.

So this morning I’m rolling up my sleeves in the Compressor Kitchen, armed with knowledge from Ripple Training’s ‘Compressor’ course and a newfound confidence in switching on Frame Controls.

If you’ve not played with Compressor before, you’ll need to understand that experimenting with controls is very counter-intuitive, and until you learn (thank you Ripple) how to choose snippets of a given movie to play with, rather than using the whole chunk, any little twiddle can easily put hours on an encode time. So you don’t play. It seems that the merest click on something takes you from a bit of a longish wait to taking the rest of the day off. Or the weekend. And the end result (should you let it run its course) often isn’t visibly different.

Which is why I’m writing blog entries with my MacBook Pro raising a sweat on an encode from 720p25 to NTSC ProRez, thinking about buying an OctoCore or at least another rendering Mac. For those with long memories, when Sorenson Pro came out (how many years ago? Ouch), there was a similar period of much sighing of users and sweating of Macs as it was good but incredibly slow to compress. Now much is the same for H.264 and downconverting HD.

But 8 cores doth not make 8x speed. Making virtual clusters may reduce a 30 minute render to 20 minutes, but it’s not the step change I experienced when I switched from G4 PowerBook to Intel and render times reduced to a fraction. What I want is something in the corner that just does FTP uploads, WMV conversions, Downrez, and I want it to work overnight. I’ve attempted to steal my wife’s MacBook, but this hasn’t worked out. An OctoCore Mac Pro would be lovely, but expensive – I’d need a screen, and with all that power knocking around, it would be a sin not to use it for Motion, so that’s another seat of FCP… A Mac Mini would be charming, but maybe a tad underpowered? Could I control it through my MacBook Pro screen via WiFi? I’m sure it can handle FTP, and perhaps if I switched Episode to it, that would work out too. Or Flix Pro instead – wouldn’t want to be without Episode.

No. I think an iMac might find its way onto my desk. A fully fledged Mac that can do useful things, and if my MacBook Pro died, it would be capable enough – and at a pinch, transportable enough – to take on all work duties. How about a 1920×1200 iMac with maxed out RAM and a hairy chested graphics card? Oh dear…

Hmmm – and the recipe for Compressor? I’ll post it when I’m sure it’s efficient and repeatable. Watch this space.