Cloud-based 360° Video Playout

The Cloud-based 360° Video Playout allows content providers and broadcasters to provide an innovative video experience on traditional TV screens. Viewers can experience video content with freely selectable views on their primary video viewing device – the large TV screen. The capability of view analytics allows content producers and advertisers to get specific information about what viewers are interested in and which part of the scene they are watching, allowing detailed feedback about viewers' interests. The viewer can control the experience of the video playback and can interact with dynamic content overlays (e.g. for interactive advertisement).

The Cloud-based 360° Video Playout for HbbTV consists of the following components:

• 360° Video Pre-processing:
  The Cloud Rendering Server operates on the source 360° video and calculates the different FOVs depending on the configuration that is also passed as input. A FOV is defined using the four parameters (φ, θ, Ah, Av)
• Storage and Streaming:
  The Cloud Streaming Server offers a REST API to control the individual view and playback state for each client. The output FOVs and manifest of pre-processing step need to be available on a streaming server that is accessible from the CDN provider
• 360° Video Player Client:
  The Cloud Streaming solution enables Smart TVs and other streaming devices to provide 360° Video experience through usual video playback of the output from the cloud streaming server. Users can control their individual view and playback state via remote control. The player constructs the final video stream to display from the FOV videos. There is no need to process the received video content before playback. The client platform needs only to provide an API that allows applications to send video segments to the video decoder.
• 360° Video Control Mobile App:
  The mobile control app acts as a smart remote control for the 360° Video TV Player. It offers an easy-to-use UI to control the video playback and the angle of current view using touch, gestures or device orientation.

As 360° Video is getting more and more popular and commercially relevant, the experience should be possible on all kinds of screens, including the TV screen. However, 360° video material in high (24K and more to achive a UHD field of view) resolution requires very high bitrate (> 200 –  300 Mbit/s) and high processing load for the view projection and rendering. The 360° Video Playout allows the experience of 360° videos on TV screens as well as on any other low capability devices by performing the rendering of the individual field of view at the server-side in a cloud based infrastructure. Only the selected high-resolution view is then streamed to the playback device, which reduces the bitrate dramatically. The viewer can interact with dynamically inserted interactive objects which enrich the video experience further. The solution for example also allows advertisers to dynamically include personalized advertisements (ad-insertion).

• Support of 24K 360° videos with a 4K Field of View
  
• Processing and delivering of live 360° video input in real time
  
• Cross-Platform Support: HbbTV, Smart TV, Android TV, Chromecast, Apple TV, Desktop and Mobile Browsers and many others
• Navigable while just using a standard remote control 
• Cloud-based video transforming engine
• Efficient bandwidth usage while delivering best possible quality due to the preprocessing approach
• DASH support allows adjusting the playback quality to the bitrate, especially important for large UHD 360° videos
• 360° video playback can be synchronised over multiple TVs and companion devices
• Support for interaction in 360° videos to allow users to control the experience (video playback but also to activate additional content dynamically)
• Picture-in-Picture (PiP) support for showing additional content within a 360° experience and switching between videos in an immersive library style
• PiP allows dynamic ad insertion as well as combining multiple live streams into one experience 
  
• Support for following predefined viewing paths and switching between them 
• Editorial Tool to enrich 360° videos with interaction
• Analytics framework to determine popular views, hot spots, regions of user interest
  

A 360° video experience in 24K on multiple screens

The Cloud-based 360° Video Playout allows  to watch high quality 360° videos even on devices with constrained capabilities such as HbbTV TVs. The delivery of this huge source video material consumes a large bandwidth and results in a considerable processing load for performing the geometrical view projection and rendering. For example, a bandwidth of more than 300Mbit/s is required to stream a 24K 360° video and it takes few seconds to render a single 4K FOV frame from the source 24K equirectangular frame which makes a smooth playback impossible - even on modern devices with high processing and graphical computing capabilities. Our solution addresses both aspects and reduces the required bandwidth and processing resources by rendering the field of view from the source 360° video in advance and streaming only this to the client. By doing so, many features which are applicable to classical videos can be also applied to the FOV videos for example Field of View Adaptive Streaming using DASH and MSE, Efficient Delivery over existing CDNs and DRM Support over EME.

Features and Assets:

• Experience highly synchronized 360° videos shared on multiple TVs and companion devices
• Cross-Platform Support: HbbTV, Smart TV, Android TV, Chromecast, Apple TV, Desktop and Mobile Browsers and many others.
• DASH support allows to adjust the playback quality to your bandwidth, especially important for large UHD 360° videos.
• Support of 24K 360° videos with a 4K Field of View.
• Add interaction to 360° videos and allow users to control the experience. Try our Editorial Tool and easily enrich your videos with interaction.
• Unlike other 360° video playouts our system saves bandwidth while delivering best possible quality due to the preprocessing approach.