[Interview] How Software Engineers Made the Gear 360 into a ‘More Immersive and Expanded Viewing Experience’

“We expect the demand for VR and AR video to grow continuously, and I believe 360-degree cameras will develop further,” says Seungkyoon Shin, a principal engineer on Applicative Product Software R&D Team, Mobile business.

360-degree videos are all the rage on social media, which means that peoples are now welcoming the arrival of an exciting range of 360-degree cameras. One of them is Samsung Gear 360, whose unique, spherical design has won much positive feedback from the market. However, as four of the device’s software engineers explain, the Gear 360 is more than just a pretty face – practical and fun, it is built to perform.

From left to right: Principal Engineers Taewon Eom and Seungkyoon Shin, Senior Engineer Hyunock Yim and Principal Engineer Jinpyo Gwak

Easy to Use and Distinctive

More than just a new technology, the Gear 360 is offering people a new way to create and share their memories, in a format that’s more immersive than ever. “When I let my friends watch a 360-degree video with the Gear VR, they all looked excited and amazed,” says Principal Engineer Taewon Eom. “The widespread use of 360-degree cameras like the Gear 360 will give more people a much more vivid and expanded viewing experience.”

While the popularity of 360-degree videos is rapidly growing, existing equipment has usually been designed for professionals, making it too elaborate for many people. Often those videos required multiple cameras and tools to shoot and edit. But the Gear 360 is an easy-to-use, all-in-one device that lets everyone make photos and videos in this exciting format.

“The editing process [for existing equipment] is also complicated for ordinary people,” notes Principle Engineer Jinpyo Gwak. “However, with the Gear 360, users can shoot and edit UHD-grade videos using a single device. Also users can use amusing camera features such as Time-lapse and Looping Video for special occasions.”

Shin adds that, in addition to creating 360-degree content, the Gear 360 also makes it easier than ever for people to share. “Anyone who uses a 360-degree camera will find it attractive,” he says. “The more active the 360-degree content sharing platform is, the more people will want to create their own content.”

Stitching Together a New Viewing Experience

The engineers say that one of their main focuses during the software development process was properly implementing the Gear 360’s two lenses. Each one captures a 195-degree field, and to make a complete 360-degree image or video, the team needed to find a way to stitch together the fields in a smooth and natural manner.

Gwak says the team considered both quality and speed while developing the stitching feature. He notes, “Obviously, the more time you can spend stitching, the better the video quality becomes. What we needed to do was find an optimal way to reduce stitching time so that users could use the stitching function with ease.”

“Since a 360-degree video includes subjects shot at both short and long distances, all such subjects need to be stitched together. We developed an algorithm that quickly detects and connects subjects that need to be stitched together on the same line,” he added.

After performing a number of tests, the team created a database for natural stitching. Gwak says, “We took photos of the same subject from different distances. If we came across anything that looked strange in any given situation, we modified the subject and took a whole batch of photos again. In the past, I mostly took photos while developing new camera devices. However, while developing the Gear 360, I was the photographer as well as the subject. It was a new and very different experience for me!”

Software Made-Easy for 360 Video Creation

In order to enable people to get the full benefits of the Gear 360 as simply as possible, the software team created two software programs: the Gear 360 Manager app for smartphone and Gear 360 ActionDirector for PC.

Senior Engineer Hyunock Yim says that for existing devices, editing 360-degree videos required the use of several tools. “The Gear 360 is the only such device that enables users to shoot with a real-time view, stitch and trim their videos on their smartphones, and to edit what they recorded on PCs using a single tool,” Yim explains.

They had to decide whether to only make a viewer function for smartphones and PCs, or if they also wanted to add editing functions at the beginning. Eom says, “After we decided which features to add, we made sure that the Gear 360 Manager app was compatible with Galaxy smartphones, including the Galaxy S7, Galaxy S6 and Galaxy Note5, in terms of reliability. We needed to ensure that the Gear 360 Manager app worked in harmony with a range of smartphone apps such as Gallery, Camera and Samsung Gear, both in terms of look and feel.”

While developing the Gear 360 ActionDirector, adopting the concept of 360-degree video for the PC also provided a unique challenge. “Because the Gear 360’s videos are not flat, we had to choose a viewing mode whereby users could watch the videos they had made,” says Yim. “We decided to adopt a spherical viewing mode so that users could enjoy videos in 360 degrees – from up, down, right and left.”