OSFI Names Indeterminable Input-Output Relationships as a Core Model Risk: Protect Your Organisation from an Unexamined AI Decision
The Office of the Superintendent of Financial Institutions (OSFI) has identified indeterminate input-output relationships as a significant risk associated with artificial intelligence (AI) models in regulated industries. For organisations that buy or deploy AI, this means their potential for failure is high if they do not carefully evaluate these relationships before committing to a solution.
In essence, OSFI’s warning concerns the technical heart of a validation engagement: ensuring an AI model performs as claimed without compromising your organisation’s governance obligations, legal exposure, and patient/customer trust. A confident but wrong answer can have severe consequences in industries like healthcare, insurance, and financial services, where human lives or substantial assets are at stake.
OSFI highlights that organisations struggle to address this risk due to internal teams being too close to the decision-making process and a lack of independent means to test AI claims. To mitigate these challenges, prudent organisations must take immediate action to safeguard themselves against unexamined AI decisions.
Straven & Co helps organisations evaluate potential AI solutions by independently testing their real-world applications, governance frameworks, and legal exposure before deployment. The company’s expertise lies in examining the decision-making processes of complex AI systems against an organisation’s operations, people, and regulatory requirements.