Closed Captioning available in video.
From 00:22 to 00:30 seconds, the video shows a profile of a person wearing the Microsoft Hololens and facing the kinect sensor, moving their arms about in space, reaching. From 00:30 to 00:56 seconds, the video shows the kinect tracking data, represented by spherical estimations of body joint locations. The person is reaching and performing squats. From 00:56 to 1:20, the video shows the first-person view from the Hololens video feed. Here the visual effects representing movement targets are seen composited into the real world with the user’s hands.
This video demonstrates ways in which full body gesture sensors like the Microsoft Kinect can be used with augmented reality headsets like the Microsoft Hololens to produce unique therapeutic experiences for home and the clinic. These applications can provide clear feedback to the user in real-time, using multimodal sensory inputs (stereoscopic sound and vision). Practitioners can also interface with the system in order to observe idiosyncrasies of a patient’s movement and suggest more effective exercises.
This application was developed as a therapeutic experience to elicit patient and practitioner perspectives of AR for rehabilitative therapy, in-clinic and in-home. Findings will be presented at the 2019 annual meeting of the Human Factors and Ergonomics Society (HFES) conference in Seattle, WA.
Key Findings:
- Patients with a history of chronic condition or disability reported less satisfaction and faith in exercise regimens prescribed by a therapist.
- The less confident a practitioner is in their patient’s ability to perform their exercises, the more they feel AR systems could benefit those patients.
- The more confident a practitioner is in their own ability to identify postural adjustments and communicate with their patient, the less they felt AR could contribute to their professional development.
- Practitioners with prior in-clinic VR experiences expressed concerns about patient safety in VR due to vestibular destabilization. Passthrough AR headsets may mitigate this effect.
- Patients may underreport their difficulties with finding adequate space and time to do their exercises to their therapists. The connection between environmental barriers and patient motivation warrants further exploration.