The field of augmented reality/virtual reality (AR/VR) is the subject of nearly 40 FDA marketing authorizations in the past few years, but this is still a relatively unexplored technological area. The FDA’s medical extended reality (MXR) program was established in 2021 to address some of the deficits in methods for validating AR/VR products, including the potential for these products to induce cybersickness. In this two-part blog, we’ll first look at the issues related to developing a validated AR/VR product, and then we’ll examine how developers might solve some of these problems before filing a finished application with the FDA.
The FDA maintains a list of AR/VR products, which totals 39 authorizations since 2015, although 20 of these came through in the past two calendar years. The orthopedic devices space is well represented on this list with 15 applications for virtual reality surgical systems, although radiology is a close second at 14 applications. These two product areas account for nearly 75% of these applications, a ratio that may change over time as digital mental health products begin to establish a foothold in the AR/VR space.
Most AR/VR Applications Filed Under 510(k) Program
One of the critical points to be drawn from this list is that only two of these 39 applications are de novo applications, a process that can take two months longer than a 510(k) just in terms of FDA review time. The preponderance of 510(k)s on the list demonstrates that the path to market for these systems need not be excessively burdensome in terms of FDA review time, even if there may be significant demands in terms of clinical studies and human factors engineering (HFE). We would also point out that the time needed to complete an application under the de novo program is likely to be considerably longer as well.
The FDA held a webinar in November that sorted through a number of considerations, such as the operation of an AR/VR head-mounted display (HMD). The requirements for an HMD for a surgeon’s use are somewhat different than when that headset is intended for patient use, but in both scenarios, the design of the stereoscopic display must account for several key characteristics, including but not limited to:
- Spatial resolution;
- Optical aberrations; and
- Rendering artifacts.
Because of the close proximity of these displays to the human eye, the FDA recommends that image latency, visual noise, contrast, and brightness also be managed. The field of view and the angular resolution are both critical for the performance of the HMD, and the need for an adequate field of view and high resolution can be difficult to accommodate in such a confined space. An HMD cannot be bulky and uncomfortable to wear as this would present an intolerable source of distraction during surgical procedures, and the FDA also observed that a display resolution of 8k may be required, no easy feat in such a small display space.
An HMD often must be able to track where the user is looking, which is particularly critical for a unit used in a surgery environment. Consequently, the software must manage latency with regard to eye tracking and other parameters of use. The FDA webinar stated that the agency’s MXR program is geared toward developing bench testing methods and HFE approaches that can be useful in validating the design of these systems.
Among the technical challenges associated with AR/VR systems is how to control for the difference in the way a radiology image presents in ambient lighting on a conventional display versus how that image appears in an HMD. The image might present somewhat differently on an HMD with some bleed-through of ambient light as well, a question that was addressed last year in an article in the Journal of Digital Imaging. The FDA’s concern is that the intrusion of ambient light onto the HMD presentation may wash out a feature of interest, which could lead to a false negative finding related to a lesion that is suggestive of disease.
One key measure for HMDs is the contrast across the field of view (FOV), which is often adequate for gross features, but could be a problem for smaller features in a medical imaging application. Image non-uniformity can be a problem when it comes to variance in brightness across the image, which may detract from the value of AR/VR when it comes to surgical planning.
Ideally, a surgical application of this kind of technology might present the real world as a background with a virtual image overlaid on that background. The brightness of the ambient lighting can make this approach difficult, with color contrast an associated issue. The FDA presenters who spoke during the webinar used the example of the OpenSight 510(k) by Novarad Corp. (K172418), a system that offers 2D, 3D, and 4D representations of the surgical field. The OpenSight system and others with a similar intended use can greatly minimize the invasiveness of the surgery, thus speeding healing times and reducing time in the operating room.
Glare is another potential problem when bright and dark areas are adjacent to each other, which can degrade from image quality, particularly at the periphery of the FOV. Spatial resolution is yet another problem that shows itself at various lateral points across the FOV, an issue that can migrate temporally into a spatiotemporal display problem that presents itself when the user of the HMD tracks a moving object.
Small Differences Between Users Equal Potentially Big Problems
Among the usability challenges is the fit of the device on the user, which includes differences in head sizes, but perhaps less obvious is the inter-user difference in interpupillary distance. The range of interpupillary distance is subject to greater variation than might be commonly understood, and a failure of the HDM design to account for this variance can create significant image quality problems, particularly blurring. Interpupillary distance for pediatric users is of course smaller than for adults, which suggests that a single design for patient use will have to offer a large degree of adjustability if it is to serve a wide range of users/patients.
In addition to these developmental difficulties, there are some hurdles in the area of clinical trials, including:
- Determining the appropriate outcome measures;
- How to construct a sham procedure;
- Patient blinding procedures; and
- The effect of whether the AR/VR instrument will serve as an adjunct rather than as a replacement for the standard of care on clinical trial design.
One of the fascinating aspects of AR/VR is that a product in this category may straddle multiple regulatory domains, such as HFE, software as a medical device (SaMD), and in some instances, artificial intelligence. Rather than delve into each of these questions here, we’ll direct our readers to previous content, such as a blog we posted in January on HFE. One of the key issues surrounding HFE is whether a mechanism in the interface actives a function in a manner that users will find counterintuitive. This consideration is certain to be especially salient for FDA reviewers when it comes to AR/VR for surgical purposes.
In part 2 of this two-part series, we’ll look at several successful AR/VR applications, the associated clinical trials, and the standards that are available to demonstrate device performance.
