Movies, Video Are More Than Just a Little Flash
The great thing about going to a technical conference like FMS (Flash Memory Summit) where they have honest-to-gawd user panels is that you find out why the Tower of Babel was built. And along the way (almost by accident), you learn something you don’t want to share.
While there were a ton of sessions that spoke in acronyms and secret formulas, I focused on sessions I could understand – the M&E (media and entertainment) sessions.
There are really three different categories of filmmakers but the goals, issues are the same (it’s just a matter of scale):
– Large sites like ILM (Industrial Light and Motion), Pixar, Disney, Warner Bros., etc. that get solutions from firms like Harmonic and Quantum but also have the luxury of an IT (information technology) team that sweats the small things and keeps the content safe from outsiders
– Medium-sized organizations that has a team of creatives/techies like One River Media that can build out their storage solutions or tap the expertise of firms like Symply when they want to for special, really tricky/insane projects
– The Indie filmmaker with 1-3 people like Catania Group, e-llusion media, and the 10s of thousands of people around the globe who rely on organizations like Advantage Video Systems and manufacturers for field/studio and workstation solutions
Several bloodhounds got lost trying to find the session rooms but the stubborn folks (engineers, solution designers and marketers, along with some great filmmakers) did get some good insights on how shooters use flash and other storage stuff as well as what they’d like/need and why.
Filmmakers are pushing image resolution and frame rates higher and can easily end up using 100-300TB of storage. Multi-camera and VR (virtual reality) video can push storage requirements toward the BP point
Nearly all of today’s digital cameras use flash for content capture (about 15 percent of the filmmaker’s total project storage budget); and even the highest-capacity (expensive) cards are constantly being swapped out with the data immediately loaded onto a computer and then backed up to an SSD or HD field RAID unit
As if that wasn’t bad enough; today’s filmmaker has to serve up high-quality, sometimes immersive, content that not only looks great on SD/HD/UHD screens, it has to stream flawlessly across the Internet (OTT – over the top) to smartphones, tablets, computers, devices.
That isn’t easy because even the youngest filmmakers on the panels had touches of grey.
Most filmmakers start each project with fresh media simply because they can’t afford the risk of losing bits/bytes, frames. The cost of redoing a shoot (if even possible) is a whole lot more than the storage media
Nearly 80 percent of the total M&E storage budget is used in production/post-production where raw capacity and pretty good speeds are good enough.
The cost of pay-as-you-go cloud storage is appealing for mid to small filmmakers but they prefer to have their storage next to their workstations.
Working through the cloud with RAW video is impossible (pushing 50-100 TB of data can take days) so proxy (highly compressed, very low res video) is shared with specialty work groups. Large content producers do nearly all of their work in the private or public cloud but mid to small filmmakers use it primarily for backup of the backup of the backup but still…
Flash people make very compelling arguments for using their memory devices for … everything. However, if every filmmaker switched to flash-based storage, they could probably only meet 10-15 percent of the M&E industry’s requirements.
Of course, all they have to do is make more right?
A new fab costs about $2B and takes two years to get up and running. According to Coughlin Associates, there are 19 of these projects underway this year and next, so don’t expect prices to drop too quickly–especially since car, home, IT and sensor everywhere folks are planning to get their share of the production capacity.
Filmmakers like what flash-based solutions offer – speed and rugged/reliable storage, especially in the field and on the set. Sure, price is important; but when time is money, things like performance, dependability/reliability and capacity outweigh cost.
While IDC and Gartner like to point out that PC sales are flat to lower, that hasn’t stopped people like HP from delivering some powerful and expandable workstations and laptops that even died-in-the-wool Mac users are slowly beginning to use in their workflow. As an added incentive, Blackmagic DaVinci, Adobe, Avid, Autodesk and other production tool producers make their software available for both platforms.
Having software available for both platforms does solve part of the problem.
However, network issues like the need for high throughput, high capacity and low latency are major bottlenecks.
The hybrid and tiered flash solutions just might be the answer.
To illustrate where storage goes, Marco Solorio, of OneRiver Media, took the audience through the production of a recent racing documentary, 10/10ths, doing all the work in-house.
I won’t bore you with all the gory details, but let’s just say:
– Acquisition formats – 12-bit cDNG RAW, ProRes HQ, MPEG4
– Resolution formats – 4K, 2.5K, 1080HD
– Off-the-shelf media included 480, 215GB SSD; 128, 256GB CFast2, 64, 128GB SD; 64, 128GB compact flash
– Total assets – 10TB
– 32TB custom RAID for local workstation
– 20TB archive RAID onsite backup plus 20TB archive RAID offsite backup plus 8TB archive bare HDs backup plus 11TB cloud archive backup
He brought tears of joy to the storage folks’ eyes.
Panelists Tim Standing, developer of SoftRAID; and Chris Bross, of DriveSaver; emphasized points the audience probably knew deep down; i.e., that RAID is good for data protection but it is no replacement for backup and SSDs do fail.
For HDs, products like SoftRAID will thoroughly check every HD and provide the user with a complete health report. It will even do some predictive analysis on when failures might occur (great if you’re in the middle of a project so you can replace the drive at your convenience instead of in a panic).
However, the two emphasized that flash-based devices will fail and that they fail differently in unique and complex ways. One minute you’re storing/reading great 4K content and BAM! suddenly zip, zero, nada.
Bross offered some DIY (do-it-yourself) steps to possibly recover your content:
– Try different card readers, they aren’t all the same
– Call media support; they may provide software/service assistance
– Try free/inexpensive recovery tools available online
– SSD failures are complex to diagnose so only use vendor-specific tools
– Be careful … don’t make the problem worse
– When you’ve run outta’ options, go to professional recovery doctors
Bross and Standing said that they and others are working to determine symptoms that can indicate flash and SSD failure is imminent, but it’s still a work in progress.
For video/audio storytellers, storage isn’t just nice; it holds their reputation, their creative efforts and their future so it was especially interesting that Chis Haeffner, of OWC (Other World Computing), didn’t provide the usual product-centric presentation but suggested the video industry needed better collaboration from all involved and video storage classifications/standardization.
Every flavor of storage technology has a set of standards but he suggested the industry was at a point where broad collaboration was necessary with unifying compatibility and qualification standards that were established by camera, workstation/notebook, workflow software, storage producers and studio/independent filmmakers.
The whole idea was to make it easier for:
– Storage industry to test, qualify, and market storage
– Video industry to clearly communicate storage needs
– Customers to get what they need
It was a helluva’ an idea and had people from every segment nodding in agreement because they all know that video storytelling is nothing but data … data with a soul!
# # #