UMD GAMMA Group Awarded Best Paper and an Honorable Mention at IEEE VR 2021

The UMD Geometric Algorithms for Modeling, Motion and Animation (GAMMA) group led by Distinguished University Professors Dinesh Manocha (ECE/CS/UMIACS/ISR/MRC) and Ming C. Lin (CS/UMIACS/Adjunct ECE) recently won the “Best Paper” award at the prestigious IEEE VR 2021 conference for their novel work on developing a transformer-based learning method that can simulate emotive human gestures from natural language text inputs.

The paper titled “Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Emotive Agents,” was authored by graduate students Uttaran Bhattacharya, Nicholas Rewkowski, Abhishek Banerjee, Pooja Guhan, Assistant Research Professor Aniket Bera, and Paul Chrisman Iribe Professor Dinesh Manocha.

The awarded work focuses on improving the efficiency of virtual agents used in areas like virtual and augmented reality (VR/AR), online learning, and virtual social interactions to create highly realistic agents endowed with social and emotional intelligence. The authors described a novel approach that combined ideas from machine learning, affective computing, and biomechanics to automatically generate emotionally expressive gestures that result in natural interactions of virtual agents with the human users. Besides, AR/VR such agents are increasingly used in digital and virtual platforms for everyday communication and interactions. The team plans to further the research by combining the Text2Gestures with varying facial expressions corresponding to different emotions and vary the emotional tone in voices.

A video overview of the research work can be found here.

The GAMMA group also received an honorable mention for their work on creating a more immersive and smooth virtual experience of locomotion for users in VR/AR environments. Overall, locomotion in these environments remains a difficult problem, as the users often wish to explore large, virtual environments while located in much smaller physical environments in the real world.

The paper that received honorable mention is titled “ARC: Alignment-based Redirection Controller for Redirected Walking in Complex Environments,” and is authored by graduate student Niall Williams, Aniket Bera, and Dinesh Manocha. This paper describes a novel redirected walking algorithm based on alignment that allows the user to explore large and complex virtual environments using natural walking, while minimizing the number of collisions with obstacles in the physical environment.

According to the researchers, Redirected Walking is a technique that allows users to explore large virtual environments while constrained to smaller real-world environments by manipulating the mapping between the user’s physical and virtual movements. The alignment-based redirection controller (ARC) steers the user such that their proximity to obstacles in the physical environment matches the proximity to obstacles in the virtual environment as closely as possible, which minimizes the chance of collisions with physical obstacles.The group intends to extend Alignment-based Redirection Controller (ARC) to dynamic scenes with moving obstacles and multiple users for future work.

A video overview of the project can be found here.

In addition to this the GAMMA group also presented a paper on interactive sound simulation, “Learning Acoustic Scattering Fields for Dynamic Interactive Sound Propagation,”authored by graduate students Zhenyu Tang, Hsien-Yu Meng and Dinesh Manocha. The research work presents a novel hybrid sound propagation method that uses machine learning methods for fast and accurate computation of acoustic fields and utilizes them to generate plausible sound effects.

The UMD GAMMA group was established in 2018 and since then the members have presented 15 papers at the prestigious IEEE VR annual conference.

One of the leading groups in the virtual reality research community, GAMMA’s research has been supported by Adobe Research, Army Research Office, the Dr. Barry L. Mersky Professorship, Capital One Professorship, the Elizabeth Iribe Chair for Innovation, Facebook Reality Labs Research, Google Research, Intel Corporation, the National Institute of Health, the National Science Foundation, and the Paul Chrisman Iribe Professorship.