A mixed reality telepresence system for
collaborative space operation

Fairchild, AJ; Campion, SP; Garcia Jimenez, AS; Wolf, R; Fernando, TP; Roberts, DJ

Abstract

This paper presents a Mixed Reality system that results from the integration of a telepresence system and an application to improve collaborative space exploration. The system combines free viewpoint video with immersive projection technology to support non-verbal communication, including eye gaze, inter-personal distance and facial expression. Importantly, these can be interpreted together as people move around the simulation, maintaining natural social distance. The application is a simulation of Mars, within which the collaborators must come to agreement over, for example, where the Rover should land and go.
The first contribution is the creation of a Mixed Reality system supporting contextualization of non-verbal communication. Tw technological contributions are prototyping a technique to subtract a person from a background that may contain physical objects and/or moving images, and a light weight texturing method for multi-view rendering which provides balance in terms of visual and temporal quality. A practical contribution is the demonstration of pragmatic approaches to sharing space between display systems of distinct levels of immersion. A research tool contribution is a system that allows comparison of conventional authored and video based reconstructed avatars, within an environment that encourages exploration and social interaction. Aspects of system quality, including the communication of facial expression and end-to-end latency are reported.

Citation

collaborative space operation. IEEE Transactions on Circuits and Systems for Video Technology, 27(4), 814-827. https://doi.org/10.1109/TCSVT.2016.2580425

• Journal Article Type: Article
• Acceptance Date: Jun 3, 2016
• Online Publication Date: Jun 13, 2016
• Publication Date: Apr 1, 2017
• Deposit Date: Jun 17, 2016
• Publicly Available Date: Jul 21, 2016
• Journal: Transactions on Circuits and Systems for Video Technology
• Print ISSN: 1051-8215
• Electronic ISSN: 1558-2205
• Publisher: Institute of Electrical and Electronics Engineers
• Volume: 27
• Issue: 4
• Pages: 814-827
• DOI: https://doi.org/10.1109/TCSVT.2016.2580425
• Publisher URL: http://dx.doi.org/10.1109/TCSVT.2016.2580425
• Related Public URLs: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=76
• Additional Information: Funders : EU fp7;Engineering and Physical Sciences Research Council (EPSRC); Projects : CROSDRIVE; Grant Number: 607117