User Interface Design for Information Acquisition in Virtual Reality

Abstract

Virtual Reality (VR) technology has been developing rapidly over the past several years, enabling various online learning scenarios, such as VR exhibitions and museums, where visitors can remotely experience digital artefacts and exhibits without having to physically visit the physical site. In physical exhibitions, visitors are accustomed to a specific pattern of information acquisition and learning, which is reading multimedia User Interfaces (UIs) (e.g., text, images) on labels or posters in close proximity of an exhibit. In VR scenarios, however, there is unlimited displaying area, where UIs do not have to be fixed on physical surfaces and can appear in any places in the space, in anytime time. Therefore, given the large design space, UIs in VR learning scenarios possess a variety of design options with few spatial and temporal constraints. It is challenging and time-consuming to manually determine the geometric and time attributes of these UIs. In this work, I propose a set of sequence-to-sequence (seq2seq) models towards UIs' time attribute, a set of design paradigms for UIs' content attribute, and an automatic layout method for UIs' geometric attribute. Specifically, I first propose three seq2seq models to predict a future time sequence of the user’s visual behaviour from past records. Then, I present a set of design paradigms for VR UIs. I develop an optimisation model incorporating UI layout guidelines and an optimisation algorithm, considering design criteria of ergonomics, functionality, and aesthetics. Lastly, I develop a design system that takes a set of 3D exhibits and UI information as input and generates corresponding UI layout recommendations. This work considers five design objectives for learning experience in VR, including adaptiveness, sense of immersion, acquisition efficiency, readability, and perceived usability. The ultimate goal of this work is to develop intelligent user interfaces in VR, which automatically adapts themselves according to the environment input, designer requirement, and user status.