The Refreshing View of Immersive Content
Developing, delivering, and using a new technology like VR isn’t easy; but gawd, is it fun.
At IBC, we’ll see – and hear from – people who can see it, almost touch it. And we’ll hear a ton of naysayers.
They’ve forgotten that:
-Silent movies lasted from 1895 – 1936
-B&W TV started in 1928
-The first U.S. color transmission was in 1954 (Tournament of Roses) but B&W reigned for another dozen years
Not a lot of color TV sets sold back then.
Why? Lack of content, high cost of production and lack of viewing screens.
The M&E industry has a lot of challenges, issues, opportunities on its plate but the Amsterdam event (Sept 14-19) is focused on helping the industry be prepared to prosper in the next 50 years.
And if that isn’t enough, there’s the SMPTE (Society of Motion Picture and Television Engineers) conference a month later in Hollywood.
Very simply, we can deliver the immersive experiences visionaries and early adopters envision. The technologies – HW/SW – will continue to evolve and the early creators are shooting/producing content people want and expect.
But it will not happen overnight because we’re changing just about everything.
Today, we’re transitioning people from the flat screen they view (regardless of the size) and someone else is dictating the content storyline and its flow to content that enables viewers to explore the general story but can change it by looking left, right, up and down.
One analyst questioned how you monetize the new video approach; citing that presently, the audience is small, production costs are high, theatres are leery, there’s no “star power”/actor identification and maybe, just maybe, it’s a fad.
The transition from B&W to color was important but it didn’t challenge the industry – nothing changed for shooter, producer, director, production, audio or viewer but … everything was in color.
The challenge for VR today is to achieve instant success.
VR is just in its first consumer generation with the classic chicken and egg scenario; and it’s something that producers with enviable track records and independent filmmakers are investing in to change.
They – and early adopters – are just beginning to discover what works and what doesn’t.
Doug Liman, who produced Bourn Identity and Edge of Tomorrow was tired of the industry’s old rules and stepped outside the filmmaker guidelines to produced Invisible, an episodic series, with Melissa Wallack and Lewis Smithingham of 30 Ninjas.
It’s difficult to step out of the industry comfort zone where you have a proven set of steps to deliver a product the audience can judge based on the script, actors and production work.
Fortunately, an increasing number of filmmakers are willing to take chances to let people experience the artform.
Basic Tool
At IBC, there will be a dizzying array of professional VR shooting rigs in the halls from superb palm-sized units like Sphericam to ultra-large 360 units.
The quick-to-market units have been 360 devices that are a series of cameras hidden inside a mystery housing. Others will be designed from the inside out as true VR cameras that capture it all, including above and below.
Some will be outta’ sight expensive (so you rent the rig for your project); others will be consumer grade for a few hundred-thousand dollars.
However, as every shooter knows; it’s not the rig, it’s the glass that makes the camera and the lenses are often the key aspect every filmmaker will look for.
The great thing about all of the rigs is that they capture a lot of content – 10X RAW video, compared to a conventional 4K/8K camera and in the right hands, the solutions (and labor-intensive production work) can deliver exhilarating experiences.
There is a growing number of VR projects that are being shot, produced and enjoyed by people who want more than sedentary entertainment.
Indie and studio VR projects like John Wickes Chronicles, The Martian, Meat and Sand, Hello Mars, The Protectors, Fear Thy Neighbors and Gone are giving viewers exposure to the new video medium.
As early experiences show, VR won’t replace 4K/HDR productions but should be viewed as a complement … not a competitor.
As for the failure prophets, VR content is a break from the past mass theatrical events.
It requires a new approach to the storyline and technical execution we’re only beginning to be comfortable with.
Andrew Shulkind, one of the leading cinematographers in the immersive space and developer of the Headcase Cinema Camera, noted, “Storage and bandwidth in the immersive space are factors that we have to really take seriously.”
For gaming, he noted that there is a trend to approximate frame rates that match the headset refresh that works best for CG (computer graphics) environments. But for live-action cinematic content, the motion blur from lower frame rates adds a more suitable softness.
Otherwise, it feels like a soap opera.
Frame rate has significant storage implications; 60fps requires twice the storage space of footage shot at 30fps. The Headcase Cinema Camera, one of the highest-resolution capture devices available, captures 92GB/min @ 30p and 184GB/min @ 60p.
Shulkind should know the challenges because he has shot with every format available (and many being developed) and consults with firms like Google and Canon.
He continued, explaining, “Oftentimes I need to have my DIT (digital imaging technician) not only manage, store, and backup what we’re shooting but also transcode the footage to a more compatible format for post. This is especially true when shooting RED or CDX (Codex’s proprietary uncompressed imaging format).
Shulkind emphasized, “The priority when capturing in 360, but really with all digital capture,” he added, “is always about maintaining as raw an image as possible with the least amount of compression so we have as much range to work with downstream when we color it.”
The content from the lenses must be stitched together and edited so no matter where viewers look, they can experience the action/environment just as though they were there. And the sound must come in a “natural” fashion.
When it’s done right, the results are great. When it isn’t … not so much!
The process of stitching the 360-degree content has made tremendous moves forward. There’s software such as with Sphericam that automatically does most of the work and with excellent results.
But it’s all the other work that makes people excited about their story, their experience.
“Much of the earliest work wasn’t scripted VR,” said Andrew MacDonald, creative director at CreamVR. “It was more spectacle-driven to show off the unique aspects of the medium. Making scripted VR is proving to be much more challenging than standard 2D film scripts.”
One must strongly “suggest” where the viewer should be looking, the point of interest.
MacDonald and others in the emerging field find the best results are achieved by thoroughly mapping out what the points of interest should be, where they are in the sphere and planning the next cut around where the viewer will be looking.
If you get everything right, it flows like a great personally involved event.
If you get it wrong, the viewer gets lost very quickly
Many professionals who have done multiple VR projects say the work is more difficult but also more fun because they are creating a world the viewer can explore that is unique to them.
The key is to let them search the frame to find and follow/develop the story.
As a result, most of the VR efforts aren’t a story but multiple stories, depending on where the viewer takes it so they view the project several times and every time it’s different.
Rush to the Table
While many in Hollywood and other content production and delivery areas whisper and shout that VR is 3D déjà vu all over again, hundreds of companies are heavily investing to nurture the technique so it becomes a mainstream entertainment option.
Content is in its infancy but that hasn’t slowed companies large and small from rushing to deliver all the peripheral products the shooter, producer and viewer will need.
As VR climbs Gartner’s hype cycle chart, stories of established and newly funded firms appear as IBC and SMPTE help the industry get behind the new VR experience – equipment, software, production studios and viewing options.
The most visible and recognized VR tool is the HMD which, unless you’re a gamer, makes you look a little ridiculous; but they will change. They could end up being like lieutenant commander La Forge’s Star Trek headset or even less conspicuous contact lenses; but better solutions will emerge.
Right now, the hardware and software experience for people in the industry is fragmented and the professionals who have embraced the content experience production are experimenting, inventing, modifying as they move to deliver new viewer experiences.
Their focus is on producing more relevant, impactful, and memorable content that resonates with people.
Conan in the Round
One of the first to meld VR with conventional TV was late-night comedian/political commentator Conan O’Brien.
At this year’s Comic-Con in San Diego, the organization brought back Conan360 to give people the feeling of really being one of the participants, cast.
To deliver a seamless, near real-time 2D/VR experience, 30 Ninjas again brought in a complete post production operation that worked alongside the show’s regular crew.
The show was shot using four OZO camera with the content sent to the on-site production team. The team live-stitched, live-switched the data to deliver a seamless broadcast experience for HMD viewers.
Using a combination of production, custom technology and unique, precision workflow; 30 Ninjas was able to deliver a color-graded, denoised, graphics-enhanced 360 feed simultaneously to cable TV, VR headsets and the web.
Not for the faint-of-heart or novice, Smithingham and his crew used leading-edge Dell 7910 computers and racks of high-performance OWC SSD (solid state drives) to produce finished content.
The Conan360 content had less than one-second latency and zero dropped frames across all cameras during the five days of continuous production and streaming.
The VR footage was then incorporated live into the TV show and highlight video was uploaded within an hour of each evening’s wrap. All of the Conan360 Live shows are also streamed on Facebook, YouTube, and team coco.
“Efforts like this show how really compelling VR is,” Smithingham said, “and how immersive VR can be as well as how it can work to enhance and enrich 2D content.”
Discover Your World
Taking the more “conventional” route, Discovery Communications has been an early adopter of VR as five- to ten-minute films that are ancillary content to the flat TV series. The episodic segments include Fear Thy Neighbor and the most recent Wild Things with Dominic Monaghan for the Travel Channel.
Using unique technology, methodology and workflow, CreamVR transports the HMD wearer to Indonesia to visit the color, sights and sounds of the area.
In the Wild Things segments, viewers get to get up close and personal with king cobras, outsized spiders and other creatures that call the area home.
“Discovery set the early standard for enriched added content for shows and special features,” MacDonald commented, “but we’re seeing more and more networks and studios seeing the value of immersive content that really draws the viewer into the environment to enrich the viewing experience.”
And it works.
Those who experience good–really good VR, find it very compelling.
VR is more than just a line-of-sight experience. It’s unique and very satisfying when it’s created properly. People can experience new environments and breathtaking activities as well as interesting and entertaining events/shows.
The technologies and production techniques are continuing to evolve and improve.
The challenge, as always, for the M&E industry is how quickly the uniqueness of the new storytelling art form can be monetized.
That requires something that is in very short supply in the industry and consumer have very little of … patience.
But the people working on the technology and developing/distributing the content as well as those executives attending IBC and SMPTE’s Annual Technical Conference are confident Dr. Jekyll was right when he said, “She will not stop until she has remade our world into her own.”
We’ll take a look at how folks are going to view it … later.
# # #