Open Medical Gesture: An Open-Source Experiment in Naturalistic Physical Interactions for Mixed and Virtual Reality Simulations

Abstract

Mixed Reality (MR) and Virtual Reality (VR) simulations are hampered by requirements for hand controllers or attempts to perseverate in use of two-dimensional computer interface paradigms from the 1980s. From our efforts to produce more naturalistic interactions for combat medic training for the military, USC has developed an open-source toolkit that enables direct hand controlled responsive interactions that is sensor independent and can function with depth sensing cameras, webcams or sensory gloves. Natural approaches we have examined include the ability to manipulate virtual smart objects in a similar manner to how they are used in the real world. From this research and review of current literature, we have discerned several best approaches for hand based human computer interactions which provide intuitive, responsive, useful, and low frustration experiences for VR users.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 2022
Accession Number
AD1183497

Entities

People

  • Chinmay Chinara
  • Thomas B. Talbot

Organizations

  • University of Southern California

Tags

Communities of Interest

  • Autonomy
  • Biomedical
  • Sensors

DTIC Thesaurus Topics

  • Applied Computer Science
  • Artificial Intelligence
  • Cognitive Systems Engineering
  • Computational Science
  • Computer Vision
  • Detectors
  • Eye Tracking
  • Human Systems Integration
  • Human-Computer Interaction
  • Human-Machine Interaction
  • Machine Learning
  • Mixed Reality
  • Object Recognition
  • Optical Detectors
  • Recognition
  • Simulations
  • User Interface
  • Virtual Reality

Fields of Study

  • Computer science

Readers

  • Agent-Based Social Robotics and Mobile-Assisted Learning in Virtual Environments.
  • Robotics and Automation.