Fraunhofer Institute for Open Communication Systems
Fraunhofer Institute for Open Communication Systems

FAMIUM Metaverse Framework

Metaverse VR virtual reality game playing, man and woman play metaverse virtual digital technology game control with VR goggle
© istock / Thinkhubstudio

The FAMIUM Metaverse Framework offers a powerful solution for rendering and streaming compute-intensive 3D applications and Metaverse experiences on nearly any device. Even devices with limited graphical processing capabilities, like TV devices, can access high-quality Metaverse experiences with ease. This is made possible by offloading the compute-intensive 3D graphics processing to remote servers that run on-premise, on the edge of a 5G network, or in the cloud. The rendered views are then streamed to end devices for display.

One of the critical components of the FAMIUM Metaverse Framework is the 1ClickMetaverse Testbed, which enables testing and evaluation of Metaverse applications developed using popular 3D graphics engines like Unity, Unreal Engine, and NVIDIA Omniverse. This testing is done under varying network conditions (Ethernet, WiFi, LTE, 5G), communication protocols (WebRTC, WebTransport), video encoding (VP8, VP9, H264, H265, AV1), deployment scenarios (on-premise, edge cloud, central cloud), and number of connected users, among others. The experiences are packaged as Docker containers and rendered in headless mode on servers with varying computation resources and hardware-accelerated graphical processing. This abstraction from the underlying compute and storage platforms allows for the support of additional 3D graphics engines in the future.

Furthermore, our testbed includes a comprehensive metric reporting and analytics tool powered by FAMIUM SAND that collects performance metrics throughout the entire rendering and streaming pipeline. These metrics are presented to the user through a user-friendly web-based visualization frontend. Our platform ensures reliable performance and seamless user experiences, providing an ideal solution for consumer, enterprise, and industrial Metaverse applications.

FAMIUM Metaverse Framework Architecture - Metaverse remote rendering architecture 1click
© Fraunhofer FOKUS
FAMIUM Metaverse Framework Architecture
FAMIUM Metaverse Demo: Chrome, Safari, iOS, Mac, Android
© Fraunhofer FOKUS

The FAMIUM Metaverse Framework consists of multiple components, each working together to deliver high-quality Metaverse experiences. The components are described below and depicted in the diagram above:

  • Web Server: Hosts all web components of the FAMIUM Metaverse Framework, including HTML, JavaScript, and CSS. This component is provided as a Docker image.
  • Signalling & Coordination Server: Manages rendering sessions and connections between clients and rendering instances (running on-premise, on the edge, or in the cloud). This component is also provided as a Docker image.
  • WebRTC STUN/TURN Server: Enables the establishment of WebRTC communication between web clients and rendering instances. This component is provided as a Docker image.
  • SAND Metrics Server: Collects and monitors metrics throughout the entire rendering and streaming pipeline. The collected metrics can be visualized in the SAND dashboard in any web browser.
  • Metaverse Unity Renderer: Renders and streams 3D and Metaverse experiences developed in Unity to clients in real-time. Each 3D/Metaverse experience is provided as a Docker image.
  • Metaverse Unreal Engine Renderer: Renders and streams 3D and Metaverse experiences developed in Unreal Engine to clients in real-time. Each 3D/Metaverse experience is also provided as a Docker image.
  • Web Client: Displays the video output of 3D/Metaverse experiences rendered remotely via any of the Metaverse Renderers (Unity, Unreal Engine) in web browsers on desktop, mobile, or TV devices. It captures and sends user inputs (remote control, touch, gyroscope, mouse, keyboard) to the corresponding rendering instance. This component is provided as a JavaScript library.