Little But More Than Just a Little Flash

Movies, Video Are More Than Just a Little Flash

“Second chances don't come around all that often. I suggest you take a really close look at it.” – Hank Pym, “Ant-Man,” Marvel Enterprises, 2015 — *“Second chances don’t come around all that often. I suggest you take a really close look at it.”* – **Hank Pym, *“Ant-Man,”* Marvel Enterprises, 2015**

The great thing about going to a technical conference like FMS (Flash Memory Summit) where they have honest-to-gawd user panels is that you find out why the Tower of Babel was built. And along the way (almost by accident), you learn something you don’t want to share.

While there were a ton of sessions that spoke in acronyms and secret formulas, I focused on sessions I could understand – the M&E (media and entertainment) sessions.

There are really three different categories of filmmakers but the goals, issues are the same (it’s just a matter of scale):

– Large sites like ILM (Industrial Light and Motion), Pixar, Disney, Warner Bros., etc. that get solutions from firms like Harmonic and Quantum but also have the luxury of an IT (information technology) team that sweats the small things and keeps the content safe from outsiders
– Medium-sized organizations that has a team of creatives/techies like One River Media that can build out their storage solutions or tap the expertise of firms like Symply when they want to for special, really tricky/insane projects
– The Indie filmmaker with 1-3 people like Catania Group, e-llusion media, and the 10s of thousands of people around the globe who rely on organizations like Advantage Video Systems and manufacturers for field/studio and workstation solutions

Several bloodhounds got lost trying to find the session rooms but the stubborn folks (engineers, solution designers and marketers, along with some great filmmakers) did get some good insights on how shooters use flash and other storage stuff as well as what they’d like/need and why.

Film Facts

Filmmakers are pushing image resolution and frame rates higher and can easily end up using 100-300TB of storage. Multi-camera and VR (virtual reality) video can push storage requirements toward the BP point

Endless Stream – M&E isn’t just the movie theater or TV anymore. The range of viewing options puts pressure on filmmakers to produce content in a growing range of formats and sizes to be clearly watched on every possible screen. — **Endless Stream** – M&E isn’t just the movie theater or TV anymore. The range of viewing options puts pressure on filmmakers to produce content in a growing range of formats and sizes to be clearly watched on every possible screen.

Nearly all of today’s digital cameras use flash for content capture (about 15 percent of the filmmaker’s total project storage budget); and even the highest-capacity (expensive) cards are constantly being swapped out with the data immediately loaded onto a computer and then backed up to an SSD or HD field RAID unit

As if that wasn’t bad enough; today’s filmmaker has to serve up high-quality, sometimes immersive, content that not only looks great on SD/HD/UHD screens, it has to stream flawlessly across the Internet (OTT – over the top) to smartphones, tablets, computers, devices.

That isn’t easy because even the youngest filmmakers on the panels had touches of grey.

Most filmmakers start each project with fresh media simply because they can’t afford the risk of losing bits/bytes, frames. The cost of redoing a shoot (if even possible) is a whole lot more than the storage media

Growing Content – With shooters, producers pushing 4K, HDR and VR, the demand for high- performance, highly reliable storage for production and post production is becoming a major issue for filmmakers. — **Growing Content –** With shooters, producers pushing 4K, HDR and VR, the demand for high- performance, highly reliable storage for production and post production is becoming a major issue for filmmakers.

Nearly 80 percent of the total M&E storage budget is used in production/post-production where raw capacity and pretty good speeds are good enough.

The cost of pay-as-you-go cloud storage is appealing for mid to small filmmakers but they prefer to have their storage next to their workstations.

Global Work – Whether it’s a tentpole production or documentary, film production usually isn’t all done in one location. Filmmakers use the best talents, the best specialists regardless of their location so production/post work is often done globally. — **Global Work –** Whether it’s a tentpole production or documentary, film production usually isn’t all done in one location. Filmmakers use the best talents, the best specialists regardless of their location so production/post work is often done globally.

Working through the cloud with RAW video is impossible (pushing 50-100 TB of data can take days) so proxy (highly compressed, very low res video) is shared with specialty work groups. Large content producers do nearly all of their work in the private or public cloud but mid to small filmmakers use it primarily for backup of the backup of the backup but still…

Flash people make very compelling arguments for using their memory devices for … everything. However, if every filmmaker switched to flash-based storage, they could probably only meet 10-15 percent of the M&E industry’s requirements.

Of course, all they have to do is make more right?

A new fab costs about $2B and takes two years to get up and running. According to Coughlin Associates, there are 19 of these projects underway this year and next, so don’t expect prices to drop too quickly–especially since car, home, IT and sensor everywhere folks are planning to get their share of the production capacity.

Filmmakers like what flash-based solutions offer – speed and rugged/reliable storage, especially in the field and on the set. Sure, price is important; but when time is money, things like performance, dependability/reliability and capacity outweigh cost.

Work Horse – All of the new opportunities for filmmakers are placing tremendous pressure on the individual’s/organization’s computers and workstations to ingest RAW content from a range of sources and process it quickly and accurately to maintain tight production schedules. The standard computer is just the starting point. — **Work Horse** – All of the new opportunities for filmmakers are placing tremendous pressure on the individual’s/organization’s computers and workstations to ingest RAW content from a range of sources and process it quickly and accurately to maintain tight production schedules. The standard computer is just the starting point.

While IDC and Gartner like to point out that PC sales are flat to lower, that hasn’t stopped people like HP from delivering some powerful and expandable workstations and laptops that even died-in-the-wool Mac users are slowly beginning to use in their workflow. As an added incentive, Blackmagic DaVinci, Adobe, Avid, Autodesk and other production tool producers make their software available for both platforms.

Team Effort – Whether it’s a small or large production team, content is generally centralized and specialists work on key aspects of the film to finish the project on budget and on time. Speed and saving every frame, every pixel is vital. — **Team Effort** – Whether it’s a small or large production team, content is generally centralized and specialists work on key aspects of the film to finish the project on budget and on time. Speed and saving every frame, every pixel is vital.

Having software available for both platforms does solve part of the problem.

However, network issues like the need for high throughput, high capacity and low latency are major bottlenecks.

The hybrid and tiered flash solutions just might be the answer.

To illustrate where storage goes, Marco Solorio, of OneRiver Media, took the audience through the production of a recent racing documentary, 10/10ths, doing all the work in-house.

Work Load – Marco Solorio, OneRiver Media, sets up one of the cameras he used in the production of the firm’s documentary, “10/10ths” RAW content, work prints, audio, special effects, insurance, production prints and preserved copy is all digital today and requires masses of storage. — **Work Load** – Marco Solorio, OneRiver Media, sets up one of the cameras he used in the production of the firm’s documentary, “10/10ths” RAW content, work prints, audio, special effects, insurance, production prints and preserved copy is all digital today and requires masses of storage.

I won’t bore you with all the gory details, but let’s just say:

– Acquisition formats – 12-bit cDNG RAW, ProRes HQ, MPEG4
– Resolution formats – 4K, 2.5K, 1080HD
– Off-the-shelf media included 480, 215GB SSD; 128, 256GB CFast2, 64, 128GB SD; 64, 128GB compact flash
– Total assets – 10TB
– 32TB custom RAID for local workstation
– 20TB archive RAID onsite backup plus 20TB archive RAID offsite backup plus 8TB archive bare HDs backup plus 11TB cloud archive backup

He brought tears of joy to the storage folks’ eyes.

Panelists Tim Standing, developer of SoftRAID; and Chris Bross, of DriveSaver; emphasized points the audience probably knew deep down; i.e., that RAID is good for data protection but it is no replacement for backup and SSDs do fail.

The Go-To Media –Flash memory products provide most of the features filmmakers need – speed, performance, data integrity, rugged – but like any storage media, they can fail and usually at the worst possible time. — **The Go-To Media** –Flash memory products provide most of the features filmmakers need – speed, performance, data integrity, rugged – but like any storage media, they can fail and usually at the worst possible time.

For HDs, products like SoftRAID will thoroughly check every HD and provide the user with a complete health report. It will even do some predictive analysis on when failures might occur (great if you’re in the middle of a project so you can replace the drive at your convenience instead of in a panic).

However, the two emphasized that flash-based devices will fail and that they fail differently in unique and complex ways. One minute you’re storing/reading great 4K content and BAM! suddenly zip, zero, nada.

Dang!

Bross offered some DIY (do-it-yourself) steps to possibly recover your content:

– Try different card readers, they aren’t all the same
– Call media support; they may provide software/service assistance
– Try free/inexpensive recovery tools available online
– SSD failures are complex to diagnose so only use vendor-specific tools
– Be careful … don’t make the problem worse
– When you’ve run outta’ options, go to professional recovery doctors

Bross and Standing said that they and others are working to determine symptoms that can indicate flash and SSD failure is imminent, but it’s still a work in progress.

Video Classification

For video/audio storytellers, storage isn’t just nice; it holds their reputation, their creative efforts and their future so it was especially interesting that Chis Haeffner, of OWC (Other World Computing), didn’t provide the usual product-centric presentation but suggested the video industry needed better collaboration from all involved and video storage classifications/standardization.

Tailored Standards –Flash storage (and other options) do have product standards, but filmmakers have critical requirements for their unique applications. It is still up to the filmmaker to choose the media he/she uses from a specific manufacturer/supplier based largely on past experience and recommendations. — **Tailored Standards** –Flash storage (and other options) do have product standards, but filmmakers have critical requirements for their unique applications. It is still up to the filmmaker to choose the media he/she uses from a specific manufacturer/supplier based largely on past experience and recommendations.

Every flavor of storage technology has a set of standards but he suggested the industry was at a point where broad collaboration was necessary with unifying compatibility and qualification standards that were established by camera, workstation/notebook, workflow software, storage producers and studio/independent filmmakers.

The whole idea was to make it easier for:
– Storage industry to test, qualify, and market storage
– Video industry to clearly communicate storage needs
– Customers to get what they need

It was a helluva’ an idea and had people from every segment nodding in agreement because they all know that video storytelling is nothing but data … data with a soul!

Filmmakers understood the power of that statement and Hank Pym’s comment, “It’s not about saving our world. It’s about saving THEIRS.”