M3-Apps: Enabling User-centric Modelling of Immersiveness in Multimodal Multimedia Applications

Multimodal, immersive systems are the latest development within the field of multimedia. These systems emulate the senses by means of omnidirectional visuals, 360° sound, motion tracking and touch simulation to create a feeling of presence in the virtual environment. They have the potential to substitute physical interactions in application domains such as training (Industry 4.0), or e-health (telesurgery). However, the current COVID19 pandemic has shown that they are not ready, as they do not fulfill the requirements in terms of immersiveness. In this research project, I argue that immersiveness can be devised from measuring four aspects, namely: presence (i.e. the feeling of being 'in' the environment), cybersickness, network-related Quality of Experience and the usability of the application. Therefore, this project proposes a user-centric approach on the assessment and modelling of immersiveness. Two dimensions are put forward: (1) Subjective and objective assessment of presence, cybersickness, usability and QoE and (2) Real-time modelling of immersiveness. Furthermore, a proof-of-concept is designed including two use cases. As such, this work will significantly advance the state of the art on immersive systems and multimedia in general.