FVV Live is a real-time, low-latency, end-to-end free viewpoint video system including capture, transmission, synthesis on an edge server, and visualisation and control on a mobile terminal.
The following video shows an application where the advanced features of FVV Live are used (free viewpoint, user control, real-time, low-latency, ...) while explaining the technical developments:
For more info, contact us:
SYSTEM DESCRIPTION
The following video presents the laboratory initial setup (cameras on rig, not on individual tripods) as well as the low-latency real-time performance:
CAPTURE
- Consumer-grade stereo cameras over standard USB connections, no genlock required.
- Software synchronization procedure over a shared clock source distributed using PTP (IEEE 1588-2002).
- Real-time depth estimation and segmentation to enable layered synthesis and bandwidth savings.

TRANSMISSION
- Real-time encoding all (necessary) camera streams, both color and depth, RTP transmission.
- Depth information means structure, but video lossy codecs are designed for natural images and the HVS: lossless coding.
- Extended precision to deliver depth data for the synthesis: adaptation of 4:2:0 video structures to get 12 bits per pixel.

SYNTHESIS AND VISUALIZATION
- Real-time synthesis, too much data from all cameras, therefore only the cameras nearest to the virtual viewpoint are used.
- Very complex task, and also unreliable depth data. Soft transitions mix contributions from closest reference cameras.
- Dense off-line background model with Retinex+AKAZE+SfM+MVS.
- Layered synthesis to integrate reliable off-line background model with on-line foreground to combat noisy on-line depth estimation in the background.
- Cell phone controls the position of the virtual camera.


For more info, contact us:







