Sometimes timing is everything.
And in our latest research to “prove” that VR filmmaking is real, SIGGRAPH couldn’t have come at a better time.
For adult kids, SIGGRAPH is great any time because you get to see some awesome special effects and film work as well as get an idea of what tomorrow will look like.
Of course, for folks like our grumpy friend – we’ll call him Ted – it’s all about the game stuff.
When it comes to VR, he knows why the stuff hasn’t taken off, “People just can’t get comfortable wearing a headset for extended periods of time and no one likes how solitary you are when you’re viewing a VR film. The key to a good movie or TV show is sharing.”
And he made that comment about the commonly voiced issue while staring at his iPhone, oblivious to anything around us on the show floor!
Even before Facebook bought Oculus for about $3B in 2014, folks like Digi-Capital were forecasting heady growth and sales numbers.
Not that Ted played many VR games, but we reminded him that the early games sucked!
They were just good to decent video games that developers slapped VR technology into using one of the two leading VR game engines – Unity or Unreal.
People got queasy.
The same was true of some of the earliest VR film efforts, because it was a new filmmaking medium.
Sci-Fi stuff like Brainstorm, The Lawnmower Man, Matrix and Avatar whetted the market’s appetite for great VR and VCs/market analysts drank their own Kool-Aid.
Consumer industry folks rushed out expensive (yes, Ted, clunky) HMDs (head-mounted displays) and waited for stuff to be delivered.
But it was an idea still in its infancy and all realities (virtual, augmented, mixed, cross) were thrown in the same pot.
Indie filmmakers have been testing, refining, producing results for years.
Hollywood has been selectively using it for years.
Want to draw a crowd to the theater? Do a VR trailer, put it on social media and people will click in – 180, 360, immersive – doesn’t matter.
They’re paving the way.
We saw The Martian VR Experience a few years ago at the SMPTE conference and it was freakin’ awesome.
We worked, struggled, sweat right along Mark Walberg…had to see the movie!
With a lot riding on each release, Lionsgate released posted their Shatter Reality trailer to accompany Shatter Reality.
What kind of action?
They let viewers be a part of every cringe-worthy piece of action you wanted to experience with him.
Of course, since it was “only a movie,” you didn’t get to experience his broken ankle, which set the production schedule back … a little.
It’s OK Tom, we’ll pass.
Granted, VR hasn’t taken off like “those folks” predicted.
There weren’t a lot of economic, off-the-shelf VR cameras in the early days; so smart, adventurous filmmakers developed their own solutions.
Stitching, image rendering and post production solutions were still works in progress, people who could “see” the potential and were anxious to get their content produced for the anticipated audience.
Most computers were woefully underpowered to handle the brutal processing load–even with the best GPU cards available at the time.
The VR filmmakers at SIGGRAPH couldn’t wait to get their hands on the new 8th generation Nvidia Turing-based GPU. Enabling light ray tracing, it’s going to help shooters produce killer film realism.
Yes 4K, 8K showrunners, filmmakers and post people drooled when it was announced; but their mouths went a little dry when they heard the cost and that they might need a new computer.
But imagine the time Indies will save in production–and time is … money!
And with some of the SIGGRAPH demos, it was hard to see if you were in the virtual or real world.
Most of the early projects combined the narrative forward momentum of things people knew worked with films with an immersive exploration of VR.
The results were often the worst of both mediums.
Immersive pioneer Andrew Shulkind has been there since the beginning.
Besides doing much of the seminal experimentation with wide angle lenses, stabilized camera movement, storage for high frame rate capture and stereo conversion at the highest level, he was one of the first to diligently bring the best of “worked before” in this burgeoning new medium.
But it took that kind of experimentation with camera arrays, steadicams and a million different versions of framing, composition, and editing to get broadcast quality.
Andy Cochrane, an equally experienced interactive and immersive creator, added that 360-degree VR content is obviously more immersive than standard 2D films.
“We followed many of the same rules,” Andrew MacDonald, creative director at CREAM, added. “Some of the work reflects the fact that people often integrate interactive VR elements into their cinema experience rather than expand on it.”
“But the most recent work that Shulkind, friends at CREAM, 30 Ninjas and other VR pro teams are releasing are true VR films with a beginning, middle and end,” Cochrane added. “It’s a creative challenge when the viewer can direct the content flow; but by using the right audio and visual guides at the right times, you can help them through the story.”
“Game producers had all of the same problems,” Lewis Smithingham, 30Ninjas, observed. “Their development engines were great, but they were having trouble helping the viewer through the game. The result was a disappointing gaming experience. Many of them came to VR filmmakers to help develop way points for the game player to get to the next level or go off on their own to experience immersive options.
“We helped them deliver the scripted and unscripted game play that gamers really wanted, and it also helped improve film work viewers are experiencing now,” he added.
Developing improved and enhanced films required hours of stitching the content, removing rendering glitches and refining VR content. Helping viewers navigate through the new virtual space was time-consuming, hard on the eyes and hard on the stomach.
The stuff VR shooters are delivering today is darn good and the more folks experience it, the faster it will grow.
360 is Real, Good
Nick Bicanic, RVLVR, was one of the first filmmakers to visit Cuba when the travel ban was lifted and wanted to do more than just give viewers a visual tour; he wanted to help people experience the country, the people, the culture.
He spent three weeks in Cuba capturing the essence and excitement of the cities, countryside, people and their pastimes.
“We gave them a solid taste of Cuba that wasn’t possible with a flat 4K film,” he emphasized. “Viewers shared Cubans love of baseball, quiet checkers matches, color and music of all types, including a special Rolling Stones concert.”
Sometimes, timing is everything and he was able to capture the excitement of the legendary group and the interaction of Cuba’s love of music.
“It was a great way to mix the old and new of Cuba,” he noted.
MacDonald and the rest of the Toronto-based Cream VR team have been instrumental in helping Hulu produce a solid library of VR shows including their most recent, A Curious Mind featuring Dominic Monaghan.
A Curious Mind evolved over several years, when 360 and VR were just in their infancy and was similar to the style of Carl Sagan’s Cosmos: A Space Odyssey; but gives viewers a closer, more personal experience.
“The strength and weakness of the immersive content is that the viewer is ‘involved,’ MacDonald said. “Creatively, we needed to control the viewing to tell the complete story.
“The user can physically participate in the story telling and shape the experience simply by looking away from the POI (point of interest),” he explained. “Our challenge was to guide him or her so Dominic and the viewer both arrive at the conclusion at the same time.”
In A Curious Mind, Dominic and the viewer make the journey together; manipulating time and space, pulling apart atoms and exploring/experiencing science as never before.
Now that 360 video is becoming established and “understood,” all three Andrews are working on next-phase projects – volumetric image capture technology.
Originally developed for Microsoft’s platform HoloLens, it uses multiple cameras to map a target location from every angle which is then combined with 3D imaging techniques to create accurate representations of depth.
Using volumetric or 4D video (4D = 3D space + x,y,z + use time as the 4th dimension) technologies, the filmmakers capture the shape and form of human performances and place them in immersive experiences.
“This marks a new paradigm in how an audience engages with content,” Shulkind explained. “With 360˚ video you are locked to a fixed position. Capturing assets volumetrically restores the filmmaker’s ability to creatively block action around the viewer to make their own decisions. It marks a paradigm shift in how audiences will engage with content in the future.”
Instead of capturing reality based on a specific perspective, storytellers capture the entire scene with infinite camera positions and infinite camera angles.
Volumetric is captured from the outside in, so people really have a true, no-holds-barred three-dimensional viewing experience.
Each of the VR filmmakers we talked with are working on volumetric projects scheduled for release late this year.
Cochrane noted, “We’re building ways to let you go anywhere you want and do anything you want within our stories.
He added, “There’s a lot of work still to be done, but it’s already getting quite magical.”
Those who had a chance to try it at SIGGRAPH said it is well worth the immersive isolation.
Of course, they also knew they would need Nvidia’s next-generation GPU to process the huge volumes of data that are generated.
To ensure that the industry will reach the analysts forecasts of – pick your favorite number, clusters of companies intent on making big money off filmmakers and viewers have been busy holding consortium meetings to develop “open” standards for VR/AR/MR.
What we seldom see in these groups though is direct involvement of filmmakers who must take the gear and software to really memorable stuff.
Still, we’re looking forward to getting an update on the Consumer Technology Association (CTA) standards at IBC as well as the Video Electronics Standards Association (VESA) standards recommendations, The Khronos Group Open XR update and the Global Virtual Reality Association’s (GVRA) best practices.
There are probably other “standards groups” at work we don’t even know about – yet.
Then, we want to see what comes out of October’s Society of Motion Picture and Television Engineers (SMPTE) conference.
Maybe they’re working on things like file formats, compression, distribution and IP delivery.
Trying to calm the hungry horde, Cochrane recently told a concerned and frustrated pro-VR audience, “Just (expletive) calm down. This is a marathon, not a sprint.
“We’re going to change humanity in uncountable ways,” he continued. “We’ve accomplished a lot in a few short years and we’ve still got a lot of work to do.
“It’s going to be a heckova’ run but we can do it,” he said.
# # #