Sunday, December 30, 2007

Video Cataloguing Software

Home/Small Business Video Cataloguing Software

I take a lot of digital pictures and digital movies of activities my family does. While I feel like I have a very good program for managing my photos (iPhoto), my options on the movie side are much less well developed. Here’s where I think we need to go to get it right for movie cataloguing and project development.

General Concepts:
Since I take a large amount of video, I need to be able to store, track and access all of the old movies in a form better than “grab that tape sitting on your desk somewhere”. While I do love the convenience of MiniDVs, and it can serve as a great long-term storage solution, it just isn’t convenient for working with except in the context of a one-off project (think, iMovie 06).

Why not just iMovie 08?
Let me count the ways and by extension, the features missing from iMovie 08. Here we go:
• This is a bit simplistic, and your experience may vary, but the Stability has to be improved. I’ve used two different DV cameras on two different recent model macs (a Quad-core MacPro and a MacBook Pro) from two different video cameras (granted, they are both Canon - a ZR200 and a XH-A1) and have had severe problems importing. The most prevalent problem is outright crashing of iMovie. This happens 8 out of 10 attempts. Unacceptable, Apple. Other attempts have included a host of other problems: dropped frames, unrendered thumbnails (with no clear way of forcing a rendering) and missing video (just doesn’t show up in the event). I ought to be able to start an automatic import and 99% of the time come back an hour later to a completed, error-free import.
• Looking at clips in some other fashion than the event library (not to mention the performance issues of the event library from time to time – I do not want to wait to look at clips). Right now, I can only look at clips in terms of events.
• Looking at full clip information. There seems to me that there is a host of other information that I would want to view about a clip that I don’t have access to right now. For instance, I would like to set the start date of the clip (for those times that my video camera internal battery may have been dead or incorrectly set). I’d like to be able to tag with location and people in the video and other category (keyword) type information (such as school concert).
• Speed of the viewing/scrolling/searching needs to be quick. While Jobs has bragged in past keynote speeches that iPhoto scrolls “like butter”, iMovie right now can only be described to scroll like a three-sided wheel.

Ideal Feature Set for Video Cataloguing Software (VC)
• Flawless imports with auto-clip splitting on breaks in the video (why doesn’t FC have this?) while adding some reasonable/intelligent file structuring in the back end.
• Event and clip tagging: This should always give you the option to tag all clips with the same information provided for the event, but allow each clip to have separate information.
• Video/Picture Adjustments: The adjustments that are available in iMovie 08, namely Exposure, Brightness, Contrast, Saturation, and White Point recentering are useful, but don’t get you all of the way there. Often with home movies, you have limited lighting options. You can’t just set up three tungsten lights at the right angles and have a reflector on the person in focus at all times. Nope, you pretty much have to live with what you’ve got naturally in the room (if my experience is any indicator, your wife or husband or family is probably annoyed enough already with your efforts to record the event to do much more). If that is the case, the video adjustments ought to allow for some varying exposure adjustments across the picture frame. Said another way, tone down the floor lamp and the portion of the wall that it overexposed, but leave the people alone. One of the simpler aspects of this feature, would be the removal of “noise storms”. I’m not sure what the real name of this is, but it is the noise that is present in a fixed perspective video. Obviously, the colors are not shifting around, but the noise makes it look like it does. Automatic de-interlacing would be appreciated. I’m sure that this can be done, but would require some fancy algorithms to get it done right. Initial iterations may require some manual interaction (which would work well for tapings where the camera doesn’t move at all), but later ones should be fully automatic. Turning real world lighting problems into Hollywood lighting would be a big selling point.
• Audio Adjustments: Home movies also have the unfortunate reality that people don’t always talk at the same volume nor are the same distance away from the camera’s imperfect microphone. Incorporate some automatic adjustments here – perhaps as simple as “auto volume balancing”. This feature would be smart enough to recognize silence and not crank up the volume when all you would hear would be whine of the camera’s tape motor, but would boost Grandma when she speaks quietly and tone down Junior when he screams uncontrollably about how his sister took his truck. On the same theme as above, turning real-world audio into Hollywood-level audio would be a huge selling point.

Video editing/exporting feature set
Here, I don’t have too much to say/complain about in terms of where current products are today. I think that iMovie (when working properly) hits the mark on simplicity and feature set for quick and dirty movie exporting. I use Final Cut Express for my serious projects, so I don’t think that my video cataloguing software needs to be all things in this regard. [However, I do think that for Apple to legitimately call this an HD program, they ought to beef up support for HD exports beyond the 720 x 540 option without having to use QuickTime option.]

One click YouTube uploading is great, but I never like to be stuck with a single option. The software should integrate YouTube, Yahoo Video and any other major contenders (it can’t be that hard to add more).

Network Storage Options
Your VC software should work with you to make effective use of network storage. Obviously, your software will be able to see any network drives you have attached to your computer, the difference for video, though is that the sustained speed of your connection to your network drive is critical. Here’s where your video cataloguing software needs to be smart. It should ask the user if they want it to do some performance benchmarking whenever it sees a new drive. If the performance of the connection will not sustain the fairly high requirements for a video import, the software should ask you how you’d like to accommodate this. The primary option should be to cache the import on the computer’s hard drive and push it up in chunks as the bandwidth of the connection allows. The software can estimate how many minutes of import it can handle before your current computer’s hard drive fills up.

Archive Feature
Many videos that are not in your favorite clips (or are from completed videography engagements) no longer need to be present on your primary hard drive or network attached storage. Archiving at full quality should be a core feature.

It is not clear exactly how your typical user would be archiving at the present moment, except by writeable DVDs. Ideally, Blu-Ray disks will be very common and the user will have access to a 45 GB Blu-ray RW, which can be added to incrementally by the user. The VC should assign unique volume names to the disk and track which clips are on each volume, such that the user can retrieve specific clips efficiently.

Access of Clips in Media Browser
No matter who’s written the software, all of the clips should be available by the “Media Browser” or its equivalent by simple dragging and dropping.

Who should do this?
Obviously Apple, Adobe, Avid and Sony. I’m not sure who else. Write me scathing comments about my stupidity if they already do.

Why won’t this work?
It can and should. Perhaps as part of a fuller suite, but I think it would be best as a stand-alone piece that would allow the user to choose their favorite editing solution.

Thursday, December 27, 2007

Hard Drive Graveyard

The HardDrive Graveyard (HDG)

I’m one of those people that just won’t throw away a hard drive. Not so much because I need the extra 8 GB that my four year old hard drive will give me, but because I just don’t want any of my private data getting out there. It may also be driven by some laziness in organizing all of my old data and the nagging fear that I haven’t gotten everything that I need from the old hard drive. So what to do?

General Concept
Ideally, there would be a ATX sized case with 12 flexible slots for desktop and laptop-sized hard drives. The slots would be flexible on both size and type (supporting SATA all the way back to the really old-school hard drives (think IDE).

The flexibility could be obtained by having a standardized connection at the back of the slot of the HDG. Then, through different trays (think MacPro) and adapters that would snap on the back of the hard drive, the device could handle any previous format. The specific trays and adapters could be sold with the HDG or after the fact for a minimum ($10) amount of money. Speed would not be a key factor of this device – this is not designed to provide primary, shared storage for a home network, but rather to only provide cheat and convenient access to old data.

The HDG would have a FireWire 800, USB 2.0 and Ethernet connection on the back. From there, it should have the same network accessibility as the best home-targeted storage area network boxes that are on the market (think Infrant and others). RAID features probably don’t need to be a big factor, but couldn’t hurt. I think a small number of added features here would help. One would be to allow the user to set a hard drive as active or old. All of the old hard drives would be logically grouped together by the server software and the first level of folders would be the separate hard drives with titles of their volume names. In addition, it would be nice to have an “OS Scrub” feature, that would remove all of the files that are on your hard drive that were there simply to allow you to run your computers (i.e., Windows or OSX system files). You shouldn’t need these files anymore, because you don’t even have the computer anymore. [Note: it would be really cool to rig up a virtualization program to make that last statement untrue. Parallels or Virtual Desktop could add some great features if they could figure out how to recognize a “computer” and just make that hard drive act as the old computer.] The space freed by the removal of system files could be used as additional storage in the context of the old stuff that you were doing, or as part of the next feature. The next additional feature (though not a critical one) is “Large Slow Storage” or LSS. If I am describing something that is already well-described, please forgive me, but it is a novel concept to me. LSS generally would be used for files that would not be accessed with any frequency, and when they were accessed, would not need lightning fast file delivery. Archiving video would be a good example of where this could be useful.

But for this concept to work well, the HDG will have to be a champ with power savings features: to both minimize customer operational costs as well as manage heat. The device should be able to individually turn off hard drives when they are not in use based on customer-set preferences. This would minimize time that the old hard drives are spinning and allow the user to get the quality of service that they need.

Total price for the HDG could not be more than $400 and the lower it can go, the better.

Why won’t this probably work?
• Storage is getting large too quickly. You can almost just copy your old hard drive lock, stock and barrel, onto the new hard drive and still have plenty of room.
• Storage prices continue to fall. Perhaps just a corollary to the first reason, but the fact that you can do the above, doesn’t mean that you necessarily will. The fact that you can do it ridiculously cheaply (at least compared to the last time you saw storage prices), you may very well do it without thinking.
• Better migration software has emerged (either within the OS or in addition to it), to allow users to move all of their files onto the new computer. My view is that OSX does a pretty good job on this, but I’m wary of trusting something like this without double-checking. That’s where your time can really be sucked up – others probably disagree.

Who should do it?
Obvious candidates are DLink, Linksys or Buffalo. Others, closer to start up, like Infrant, could really make a go of it here. Its probably not a huge market, but one that I would assume that they could pursue. Pursuit could also bring some modular benefits for them. The connectors/sleds/slots that are generic for this project, could be adapted and developed only once for the rest of their product line.

[Update 2014-11-28]
There are a great many devices known as Hard Drive Docks available that do something similar to what I described here.  I must have been a thought leader?  No, I'm just joking.