Human-Centred ASL Learning: Effects of Interface Modality, Game Mechanics, and AI-Based Diagnostic Guidance

Abstract

This dissertation presents an exploration of the integration of cutting-edge technologies to enhance American Sign Language (ASL) learning and to improve the efficacy of mediated human-human textual conversations. Emphasizing user experience (UX) design, it comprises a series of empirical studies that collectively demonstrate how user-defined interactions, gamification in camera-based environment can significantly augment traditional methods of learning.

This research investigates the development and evaluation of a user-defined hand gesture interface for learning static ASL. The findings indicate that custom gestures substantially increase user motivation and engagement, contributing valuable insights into the role of personalised interfaces in language acquisition. This is followed by a series of experiments focusing on the implementation of an innovative VR camera-based, gamified learning environment. These studies offer a critical comparison between 2D and 3D VR systems in the context of ASL education, revealing that while 3D environments enhance user engagement and satisfaction, they do not exhibit a significant advantage over 2D environments in terms of learning outcomes. This suggests that engagement and effectiveness in learning are not always linearly related.

Extending the scope of immersive virtual environments, the dissertation introduces a novel approach to ASL learning through immersive VR, specifically targeting numeric ASL. The approach is evidenced to elicit higher user engagement in comparison to conventional web-based methods, offering evidence that immersive learning environments can provide more satisfying and efficacious educational experiences.
Collectively, this dissertation contributes to the fields of educational technology and human-computer interaction. It offers empirical evidence supporting the integration of user-defined interactions and gamification in enhancing learning processes. This work provides interesting insights into the design of user-centric educational tools, highlighting the transformative potential of these technologies in revamping traditional methodologies and bolstering user engagement and educational experiences.