Facebook’s new Surround 360 video cameras let you move around inside live-action scenes

The freedom of VR with the fidelity of real life

Photo: Facebook

Facebook today announced the second generation of its Surround 360 video cameradesign, and this time the company is serious about helping potential customers purchase it as an actual product. The Surround 360, which Facebook unveiled last year as an open-source spec guide for others to build off of, has been upgraded as both a larger, more capable unit and a smaller, more portable version.

Facebook is calling the big model the x24, because it now has a 24-camera array arranged in an orb instead of the 17 cameras the original flying saucer-shaped Surround 360 called for. The small model is the x6, with just six cameras but in a far more manageable package. Instead of just releasing the design schematics for these cameras online, as it did last year, Facebook is now teaming up with a select group of hardware partners to manufacture and sell finished products later this year. It’s unclear if these products will be Facebook-branded in any way, but the company is still stressing that it has no plans to sell the cameras directly.

 Photo: Facebook

As for the upgrades, the x24 and x6 aren’t simply just refined versions of last year’s hardware. They also now capture 8K-quality scenes with what’s known as six degrees of freedom (6DOF), which means your body can move forward, backward, up, down, left, and right so long as your wearing a VR headset with positional tracking like the Oculus Rift. In other words, this is the same kind of freedom of motion high-quality VR allows, but this time with live-action shots. It’s done using a mix of hardware and software that captures a better understanding of the depth of objects in a scene, and replicates perspectives that the camera never captured originally.

“We capture and then we can estimate depth,” says Brian Cabral, an engineering director at Facebook who leads Surround 360 development. “We actually compute for every pixel where it is in the scene.” Once the pixel has a location, he says, viewers can view it from any perspective as it if were part of a real-life scene. After the depth estimation process, which Facebook handles with its own custom software in the cloud, the video then can be edited with any number of standard post-processing toolsets from Adobe, Foundry, and others.

This 6DOF tech has existed in the past, usually reserved for extremely high-end Hollywood special effects and editing tools, yet other companies in the camera and VR space are working on bringing costs down and making content more accessible. Camera maker Lytro is perhaps the best known, with a rig of its own that uses light field technology to capture the geometry of the light in a live-action 360-degree scene. It’s a different approach than Facebook’s, which relies more on computer vision, but it achieves a similar effect.

Facebook, however, thinks its hardware and software is cohesive and easier to use than existing tech. In a demo shot with its own Surround 360 x24 prototype, I was able to observe the differences between a static, 360-degree video and one with the new freedom of motion. The difference is immediate. In the static video shot at the California Academy of Sciences, I could only look around as if my surroundings were a static painting stretched into a sphere around me.

In the updated version, however, I could move my shoulders, lean in close to objects, and even walk around in a small sphere-shaped zone. The effect this has on the viewer is a substantial jump in the level of immersion, making one feel as if they’re viewing a real-life moment play out live instead of simply existing inside of a looped recording. It’s effectively the freedom of VR with the fidelity of real life.

Facebook’s x24 camera creates a depth mask for every object in a scene. That’s what allows these CGI butterflies to interact with the light and objects in this scene as if they were real.
Photo: Facebook

Facebook says this will have big implications for the 360-degree format, giving developers the ability to create more engaging videos. You’ll even be able to edit live-action captures with CGI imagery like adding butterflies to an outdoor scene or even changing the background from cloudy to sun or from daytime to nighttime, all thanks to the light and depth data.

Plus, the content can be captured once and reformatted for different platforms, Cabral says. So you can shoot a video designed for the Rift headset and have it work on the Gear VR or even on a smartphone screen accessing it through the Facebook mobile app. With each step down, you lose features like head and body tracking and the six degrees of freedom. But the fidelity of the compressed image remains high-quality because it was natively captured in 8K.

Like the original camera, both of the new Surround 360 models were developed at Facebook’s on-site hardware lab, Area 404, under the supervision of Cabral, who has extensive computer graphics and imaging expertise. Facebook hired Cabral from Nvidia for his engineering and computer vision chops, and he’s been leading the hardware charge at the company’s camera division as a way to solve the difficult “chicken and the egg” problem with 360-degree video. Facebook has the platform to serve this kind of video to billions of users — and the VR headset company to sell those users pricey hardware. Yet not a lot of this content exists yet. Creators also don’t want to put the time, money, and effort toward creating it if they think users won’t ever experience it.

That’s precisely why Facebook has invested resources into building its series of 360-degree video cameras, both to show creators what the tech is capable of and to help kickstart the immersive video boom Facebook thinks might be the future. “What we’re trying to do with VR in general is bring people up the immersion curve,” says Mike Schroepfer, Facebook’s chief technology officer. “The end vision is as close as you can to feeling like your’e actually there. The gold standard is real life.”

 Photo: Facebook

Part of that effort is meeting VR halfway with live-action captures of real-life moments that, over time, incorporate the benefits of virtual worlds. “We didn’t want to be in the camera business. We wanted creative people to be able to make this content,” Schroepfer says. FAcebook hopes to get the cameras to market some time this year, but it wouldn’t disclose pricing. (The original Surround 360 was estimated to cost about $30,000 at using the company’s exact schematics.)

Still, for the 360-degree video market, Cabral says a camera of this quality — plus a cheaper, portable version you could use more easily in tighter indoor spaces — will help push the medium forward. Ultimately, Facebook wants “to establish a creative norm,” Cabral says. “There’s a whole new language you have to shoot. We can take for granted we can shoot in 2D. The reason we want high-end creators to use this is for them to teach the rest of us.”


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s