With you – an experimental end-to-end telepresence system using video-based reconstruction

Roberts, DJ; O'Hare, JJ; Fairchild, AJ; Campion, SP; Moore, C; Aspin, R; Duckworth, TW; Gasparello, P; Tecchia, F

Abstract

We introduce withyou, our telepresence research platform. A systematic explanation of the theory brings together the linked nature of non-verbal communication and how it is influenced by technology. This leads to functional requirements for telepresence, in terms of the balance of visual, spatial and temporal qualities. The first end-to-end description of withyou describes all major processes and the display and capture environment. This includes two approaches to reconstructing the human form in 3D, from live video. An unprecedented characterization of our approach is given in terms of the above qualities, and influences of approach. This leads to non-functional requirements in terms of number and place of cameras and the avoidance of a resultant bottlekneck. Proposals are given for improved distribution of processes across networks, computers, and multi-core CPU and GPU. Simple conservative estimation shows that both approaches should meet our requirements. One is implemented and shown to meet minimum and come close to desirable requirements.

Citation

Roberts, D., O'Hare, J., Fairchild, A., Campion, S., Moore, C., Aspin, R., …Tecchia, F. (2015). With you – an experimental end-to-end telepresence system using video-based reconstruction. IEEE Journal of Selected Topics in Signal Processing, 9(3), 562-574. https://doi.org/10.1109/JSTSP.2015.2402635

• Journal Article Type: Article
• Online Publication Date: Feb 11, 2015
• Publication Date: Apr 1, 2015
• Deposit Date: Feb 17, 2015
• Publicly Available Date: Apr 5, 2016
• Journal: Selected Topics in Signal Processing
• Print ISSN: 1932-4553
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 9
• Issue: 3
• Pages: 562-574
• DOI: https://doi.org/10.1109/JSTSP.2015.2402635
• Publisher URL: http://dx.doi.org/10.1109/JSTSP.2015.2402635
• Related Public URLs: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=4200690
• Additional Information: Funders : Engineering and Physical Sciences Research Council (EPSRC); Projects : Eye Catching: Supporting tele-communicational eye-gaze in Collaborative Virtual Environments; Grant Number: EP/E010032/1