The FDA’s draft guidance for predetermined change control plans (PCCPs) is months away from conversion into a final guidance, but that hasn’t stopped software developers from creating products with a PCCP component for the U.S. market. Thanks to pending legislation in the European Union (EU), the PCCP concept may soon be operational in the EU market as well, which would constitute a massive step forward in regulatory harmonization for this class of software products.
The FDA approved the de novo application by Caption Health on Feb. 24, 2023, for a machine learning (ML) product that processes ultrasound images to calculate left ventricular ejection fraction (LVEF). The result of this analysis can be used only to assist the cardiologist or other healthcare professional (HCP) in evaluating the patient’s overall cardiological health, but the more interesting part of this decision summary is the description of the generic device type. The description is “a radiological machine-learning-based quantitative imaging software with predetermined change control plan.”
No PCCP, No Predicate
From this, we take away the message that any device that uses this Caption Health product as a predicate must also incorporate a PCCP. The description states that the planned modifications are part of the product design, which creates a need for greater rigor in ensuring that design history files are appropriately populated with the required information. Since this is a relatively novel area of medical device software, there may be some ambiguity with regard to the agency’s expectations about the level of detail in design history files.
The FDA received the application from Caption Health on Sept. 28, 2022, which means that the fourth medical device user fee agreement, or MDUFA IV, governed the standard turnaround time for the application. MDUFA IV called upon the FDA to make a determination on de novos in 150 days, which is approximately the amount of time the agency needed for this application. The decision summary referred to unspecified deficiencies in the initial filing, which seem to have been addressed quickly. The mere fact that the agency and the applicant needed time to come to agreement about the content of the application is hardly surprising, given the novelty of PCCPs.
Training Sets Should Include Range of Ultrasound Systems
While the FDA has not yet updated the regulation with Part 892.2055, the de novo summary offers a few insights into the requirements for this type of application. In addition to hazard analysis, the application’s section on software design verification/validation should consist of a description of:
- The image postprocessing algorithm, including a description of each major component or block, the algorithm’s inputs and outputs, and the limitations of the algorithm;
- Training data sets, including subsets for characteristics such as patient demographics and clinically relevant confounders; and
- The performance testing protocols that were used to demonstrate that the algorithm performs as intended.
For any applications that use the Caption Health de novo as a predicate, the performance testing protocols should include recognized objective measurements such as sensitivity, specificity, and predictive value. Performance testing should also include an adequate number of cases from cohorts to confirm that the algorithm performs reliably for that cohort and for the ultrasound imaging equipment used by the HCP.
Patents and Trade Secrets May Diminish Value of Predicate
One of the primary concerns on the FDA’s part regarding the Caption Health de novo was the potential for the HCP to misunderstand the changes to input criteria, output performance, or other aspects of design changes as they are implemented under the PCCP. Another key point of emphasis is that risk management activities include an assessment of the risks of the planned modifications and the associated mitigations, but the FDA also emphasized that the planned changes must be described in the device master record.
One important question that cannot be answered by an FDA guidance is the extent to which intellectual property protection considerations will come into play. It has been argued that patents and trade secrets are often an issue in connection with software applications to the FDA, limiting the disclosure the agency can make about the predicate and thus limiting the utility of that predicate. Even when the intellectual property is treated as a trade secret rather than as a patented technology, the developer of the predicate device may press the FDA to avoid disclosure of information that may be crucial to establishing substantial equivalence.
While the FDA may seem to have forged ahead of its counterparts where the PCCP concept is concerned, the EU may soon catch up upon passage of the Artificial Intelligence Act (AI Act). The AI Act states that it would be appropriate for an AI system to undergo a new conformity assessment when changes to the algorithm affect the compliance status of the product. However, the existing text of the legislation (which is not yet in its final form) also states that it is necessary to provide a regulatory framework such that any changes to the algorithm and its performance – so long as these are described in the premarket dossier and are assessed by the notified body – should not constitute a substantial modification, thus relieving the need for a new regulatory filing.
Question of FDA’s Legislative Authorities is Resurrected
While the FDA’s draft guidance for PCCPs might seem to suggest the agency has all the statutory authorities it requires for ML, the conversation about those authorities may not yet be at an end. Scott Gottlieb and Lauren Silvis posed several related questions in a recent journal article, which stated that the FDA’s current regulatory toolkit requires enhancement, which may require legislative help from Congress. Silvis and Gottlieb wrote that a regulatory framework that is tailored for AI and ML would consider how these algorithms might influence clinical decision-making, but would give due consideration to the HCP’s exercise of clinical judgment. We note that the FDA’s final guidance for clinical decision support (CDS) software is the subject of a citizen’s petition that raises a question of whether the final guidance interferes in a physician’s practice of medicine, including when the CDS incorporates an ML algorithm.
Whatever one thinks of the arguments advanced in this editorial, it is important to bear in mind that Gottlieb’s status as a former FDA commissioner means that his views can influence policy discussions in Washington. This is especially the case because the FDA’s Jeff Shuren has said on at least one occasion that the agency requires new statutory authorities to properly regulate AI and ML. Pay close attention to this policy space, as it seems destined to evolve far more quickly than any regulatory question that preceded it.
