Extended reality (XR), encompassing virtual (VR) and augmented reality (AR), incorporates objects and environments from both the physical and digital or virtual world. In the context of medicine, XR presents unique opportunities for improving patient outcomes by introducing novel methods for patients and medical practitioners to interact with each other and with environments including digital elements, such as medical data.
Advancements enabled by XR have already demonstrated success in medical education and are actively being used in surgery1-2, diagnostics3, therapy2, and training2. While these applications present new opportunities, the adoption, impact, and longevity of XR applications in medicine will be facilitated with the development of evaluation techniques, quantitative assessment of the effectiveness of new diagnostic and treatment approaches, and a clear understanding of the value added by XR over current clinical practice standards. With a multitude of XR devices and applications in medicine rapidly emerging, related assessment methodologies are lagging and currently lack community consensus. Assessment challenges are pervasive in XR devices across human factors, display performance, and sensor reliability and contribute to user and patient safety risks, such as image registration or tool tracking errors in surgery.
We are proposing a public workshop focused on discussing evaluation needs, gaps, and approaches for medical XR. The overarching vision is to advance the evaluation of medical XR devices and applications, thus accelerating the development of medical uses of XR benefiting patients while advocating for safety and efficacy. Implementing this vision requires identifying evaluation needs and approaches as well as addressing critical gaps that impede device development, innovation, and use in medical applications.
The workshop will be successful only with wide participation of relevant stakeholders across the development and end user space, including hardware and software developers, medical professionals, academics, government researchers, and patients. The public workshop will provide a pre-competitive space to discuss open questions in current research, challenges in development process, the implementation in the workflow of medical procedures, and to articulate unmet patient needs.
A consensus on assessment methods for XR medical applications will serve the medical community by accelerating application development, increasing the predictability of regulatory processes, and establishing effective use cases for medical professionals. Understanding the interplay between assessment needs and evaluation techniques will help illuminate the specific barriers impeding development and adoption, such as uncertainty about clinical and bench validation methods. During this workshop, we propose to (1) discuss evaluation, assessment approaches, and challenges for XR applications among stakeholders in industry, medicine, academia, and government, (2) foster consensus on assessment methods for medical applications, and (3) raise awareness in the larger medical and XR device communities of significant assessment issues and best practices for device characterization and patient safety. We encourage interested parties to attend the Medical eXtended Reality (MXR) public workshop to be held on 3/5/2020 at the FDA White Oak campus (Silver Spring, MD). In addition, those interested in actively contributing to the organization and shaping the agenda of the workshop may contact the authors through the following website ().
- J. W. Yoon, et al., “Augmented reality for the surgeon: systematic review,” The Int. J. Med. Robotics Comput. Assist. Surg. 14, e1914 (2018).
- T. M. Peters, et al., Mixed and augmented reality in medicine 16 (2018).
- D. Duncan, et al., “Using virtual reality to improve performance and user experience in manual correction of MRI segmentation errors by non-experts,” J. Digital Imaging pp. 1–8 (2018).
- R. S. Draper, J. Penczek, R. Varshneya, and P. A. Boynton, “72-2: Standardizing fundamental criteria for near eye display optical measurements: Determining eye point position,” in SID Symposium Digest of Technical Papers, vol. 49, pp. 961–964 (2018).
- J. Penczek, P. A. Boynton, F. M. Meyer, E. L. Heft, R. L. Austin, T. A. Lianza, L. V. Leibfried, and L. W. Gacy, “65-1: Distinguished paper: Photometric and colorimetric measurements of near-eye displays,” in SID Symposium Digest of Technical Papers, vol. 48, pp. 950–953. 27 (2017).
- L. Zhang and M. J. Murdoch, “Color matching criteria in augmented reality,” in Color and Imaging Conference, vol. 2018, Society for Imaging Science and Technology, pp. 102–109 (2018).
- H. Suenaga, H. H. Tran, H. Liao, K. Masamune, T. Dohi, K. Hoshi, and T. Takato, “Vision-based markerless registration using stereo vision and an augmented reality surgical navigation system: a pilot study,” BMC medical imaging 15, 51 (2015).
- M. Zhu, F. Liu, G. Chai, J. J. Pan, T. Jiang, L. Lin, Y. Xin, Y. Zhang, and Q. Li, “A novel augmented reality system for displaying inferior alveolar nerve bundles in maxillofacial surgery,” Sci. Reports 7, 42365 (2017).
- G. Riva, B. K. Wiederhold, and F. Mantovani. "Neuroscience of virtual reality: from virtual exposure to embodied medicine." Cyberpsychology, Behavior, and Social Networking 22.1, 82-96 (2019).
- J. J. LaViola Jr, "A discussion of cybersickness in virtual environments." ACM SIGCHI Bulletin 32.1, 47-56 (2000).
This post was authored by Ryan Beams (OSEL/CDRH/FDA), Heather Benz (OSEL/CDRH/FDA), Kim Kontson (OSEL/CDRH/FDA), Andrea Kim (OSEL/CDRH/FDA), Rafael Grossmann (Singularity University), John Penczek (NIST), Ed Margerrison (OSEL/CDRH/FDA), and Aldo Badano (OSEL/CDRH/FDA).