Predetermined Change Control Plans for Adaptive AI Devices: A Regulatory Requirement with Significant Risk Exposure
In healthcare and other regulated industries, organisations deploying adaptive AI devices must now ensure predetermined change control plans are in place to mitigate the risk of uncontrolled updates causing system failures or compromising patient care.
The development of adaptive AI devices enables these systems to learn from data and adapt to new situations as they arise. However, this adaptability also increases the complexity of managing changes to the device, making it crucial that organisations establish predetermined change control plans before implementing these systems in their operations.
For buyers of AI, particularly those in healthcare, insurance, and financial services operating in Canada and the UK, this development poses significant risks. If not managed properly, adaptive AI devices can malfunction due to uncontrolled updates, leading to system failures or compromised patient care. This risk exposure arises from three key areas: capability risk (does the technology do what was claimed), governance risk (can the decision be defended when boards or regulators ask), and liability risk (what the organisation is accountable for when the AI errs).
To mitigate these risks, prudent organisations should take immediate action by reviewing their current change control processes and ensuring they align with regulatory requirements. This may involve implementing new procedures to manage changes to adaptive AI devices, such as conducting thorough impact assessments before making updates and obtaining approval from relevant stakeholders.
Straven & Co helps organisations like yours address the challenges of validating AI claims by examining AI decisions against your operations, people, governance and legal exposure, and vendor claims, delivering a plain verdict: proceed, proceed with conditions, or do not.