Kim Jin Ryong (University of Texas at Dallas) / Multimodal Interfaces for Immersive Virtual Reality
Abstract: Humans perceive the world through multiple modalities, including the basic senses of sight, hearing, smell, taste, and touch. For example, a person in a coffee shop can see nearby people, hear the ambient noise in that setting, smell the coffee in their cup, and feel its warmth while holding it. These modalities work together to provide them with a rich and reliable sense of their surroundings. A multimodal human-computer interface is a user interface that offers different types of sensory stimuli at the same time (e.g., visual, auditory, haptic, olfactory, etc.). Multimodal interfaces are necessary to support multi-sensory experiences, which can benefit usability and user experience. They can lead to more “natural” interaction with digital environments, providing multiple types of sensory information similar to what we perceive when interacting with physical environments. It is challenging to create high-quality multimodal interfaces: This requires an in-depth understanding of human perception and device output capabilities and how these can be combined to make interactions with a convincing multi-sensory experience. In this talk, I will discuss how I have designed and engineered interactive multimodal interfaces and how they could impact the user experience by delivering multi-sensory experiences in an immersive virtual reality.
Biosketch: I am an Assistant Professor in the Department of Computer Science and a Director of the Multimodal Interaction Lab (MI Lab) at the University of Texas at Dallas. My research strives to create novel haptics interactions to tackle technical and human factors' challenges to amplify human satisfaction through enriched user experience. My research goal is to establish an interdisciplinary research collaboration program to solve real-world challenges in haptics interaction and address issues in human perception and multimodalities in immersive environments. I received my Ph.D. in Electrical and Computer Engineering (Advisor: Dr. Hong Z. Tan) and an M.S. in Computer Science, both from Purdue University. I also received M.S. and B.S. in Electrical and Computer Engineering from Hanyang University, Seoul, Korea. Prior to the current position, I was a Staff Researcher at Alibaba Group in Sunnyvale, California, and a Senior Researcher at the Electronics and Telecommunications Research Institute (ETRI). I also worked at Microsoft Research Asia (Beijing, China), Samsung Advanced Institute of Technology (Yongin, Korea), and NHK Science and Technology Research Laboratories (Tokyo, Japan) as a research intern.