Previously, we highlighted some of the issues with CSV, which is still governed by an FDA guidance issued in 2002. One of the critical points to bear in mind is that the CSA draft is intended only to supersede Section 6 of the 2002 guidance, not to replace the 2002 guidance in its entirety. The earlier guidance otherwise remains untouched, but Section 6 occupies only four full pages and part of a fifth in the 2002 guidance, and thus the level of detail is expanded upon greatly in this new 25-page draft.
As a reminder, the scope of this CSA draft is limited to computer systems that are used in production systems and/or for management of the site’s quality system. The draft is not intended to provide a specific framework for validation and verification, but to provide a risk-based approach for determining how exhaustively those activities should be conducted. The document proposes a two-tier risk framework for CSA, the high process risk and not-high process risk categories, the latter of which encompasses moderate-risk and low-risk software and software functions.
Part 11 Compliance Questions Answered
One of the differences between these two documents is that Section 6 of the 2002 guidance suggests that commercial off-the-shelf (COTS) software might not be appropriate in some circumstances. This guidance states that an inability to audit the vendor’s design and development methodologies used to form COTS software, which could be handled by a third party, might suggest a need to go with an alternative software product. While black-box testing may suffice to overcome such considerations under the terms of the 2022 draft, the CSA draft makes no mention of any need to reject COTS for which the manufacturer is unable to obtain information on the vendor’s software design and development activities.
Another area in which the CSA draft varies from the 2002 guidance is in the discussion of compliance with Part 11, which addresses electronic records and electronic signatures in all regulated manufacturing environments. The 2002 guidance states that extensive testing may be required for a manufacturing site-wide electronic record and electronic signature system while the FDA noted in the CSA draft that manufacturers have communicated in recent years that they are not clear on whether Part 11 applies to its data systems. In response, the CSA draft states that Part 11 enforcement discretion is in place for validation of computer systems that are used to create, modify and maintain electronic records. A quality management system document promulgated to satisfy the requirements of Part 820 would typically qualify as an electronic record, and thus these validation activities do not require electronic signature tracking per Part 11.
While there is some overlap between these two documents, much of the CSA draft constitutes a complete departure from Section 6 of the 2002 guidance. Section B of the CSA draft states that a risk-based analysis of software should consider failures that are reasonably foreseeable rather than just those failures that are likely. This section goes on to state that high process risk is associated with software for which a failure may migrate to an elevated risk associated with use of the manufactured medical device. A failure of software that falls into the not-high risk software category would be unlikely to translate into a significant increase in device failure or malfunction rates.
Among the functions that would place software into the category of high risk are:
- Maintenance of manufacturing process parameters (such temperature, pressure and humidity) when those parameters have an impact on device safety or quality;
- Analysis of a manufacturing process or a product to determine acceptability without the need for human input; and
- Automated surveillance, trending or tracking of data that has been identified by the manufacturer as essential to the quality and/or safety of the device.
In contrast, the software functions that would fall into the category of not-high risk include:
- The collection and recording of manufacturing process data for monitoring/review purposes when those purposes have no direct impact on production/process performance;
- Compliance with corrective and preventive action (CAPA) requirements and for logging and tracking complaints; and
- Data management and software that provides an alert for exceptions to an established process.
The CSA draft states that the FDA does not see ISO 14971 as applicable to this risk framework because of the non-probabilistic nature of risk associated with software used for these applications. This non-probabilistic approach to failure is also seen in the most recent draft guidance for premarket considerations for device cybersecurity, but Section 6 of the 2002 guidance makes no reference to 14971, although the ISO standard is mentioned elsewhere in the 2002 guidance.
A manufacturer may select unscripted testing for any computer systems and functions it believes do not fall into the high process risk category, which may include one or more of:
- Ad hoc testing, which does not necessarily require extensive documentation;
- Error-guessing, which requires some knowledge and experience regarding past software failures or a general familiarity with failure modes; and
- Exploratory testing, which again relies on experience and knowledge, and which may seek to evaluate unanticipated user behaviors and accidental use situations that could lead to software failure.
Scripted testing for high process risk software can be more robust and/or formal, but the CSA draft makes a few key concessions, such as the use of purchasing controls to establish a baseline for validation of purchased software. In some instances, the vendor’s validation work might suffice for some lower-risk software features, but a manufacturer’s production and process controls that are functionally adjacent to the function of the software in question can also be leveraged to help ensure the reliable performance of the software.
The net effect of this CSA draft is to relieve the manufacturer/developer of a range of validation and verification activities for a manufacturing site’s computer systems, but the manufacturer must do more up-front work to do to determine which functions qualify for a lesser regulatory burden. While there are sure to be a few close calls as to what constitutes a high process risk software function – a debate that is unlikely to occur until an FDA inspection is underway – this draft opens an entirely new conversation in what has been a very difficult and exhausting dialogue between the regulator and the regulated for two decades.
