Sunday, 16 September 2018

Monitoring the Risk and Business Impact of AI-Based Solutions



AI-based solutions can shape how financial services businesses make money, whether the business model is the same or not. For an existing financial services business, the motivations may vary and range from efficiency to expanding the business. There would be project risk as with any development, but leaving that important consideration aside, it is worth bearing in mind that AI-based solutions would also impact the risk profile of the business. This may or may not be the original intention, but it becomes more likely. The key implication is that implementing an AI-based solution would require a radically different risk oversight approach by the business.

Standard computer algorithms which are not AI-based canand dosolve complex problems. The main feature of such algorithms is that the problem is somehow defined and an algorithm developed to solve it which will produce the same answer as long as the same inputs are provided. So a credit-scoring mechanism calibrated to capture a certain type of client gives you just that.

The answers offered by an AI-based system may change over time. New data is used to reassess the underlying relationships and recalibrate the relationship between the target variable and the potential explanatory variables. This “learning” can also happen in a standard programme when there is a process of recalibration. The difference is that in the case of AI, learning would happen on a real-time basisthat’s the essence of AI.

Alternatively, with AI a target variable may not have been defined. That’s not as unusual as it might sound. For example, algorithms assessing a loan or credit card underwriting may fall in this category because there is no single rule to predict a borrower’s likelihood of repayment. New data can lead to a certain recalibration or can be used to identify new relationships between certain data. For example, over time an AI-based system might identify that outstanding debt is a better predictor of the likelihood of borrower repayment than repayment history and penalise someone with a relatively good track record of timely repayments.

The first type of AI-based solution is called “supervised machine learning” and the second one “un-supervised machine learning”. The key difference is the extent of autonomy that goes with the learning.

Consider the potential impact on conduct risk of AI-based tools. One of the expectations from Treating Customers Fairly (TCF) with respect to product governance is that they are designed to meet the needs of identified consumer groups and are targeted accordingly. This requires a clear business strategy, including identification of the target market through a combination of qualitative and quantitative research and oversight of the business to ensure that it is aligned with initial expectations of customers and business generated. Take the example of automated investment services covered in a recent FCA review. These providers would rely on some type of AI-based solution, whether supervised or unsupervised machine learning. The possibility of capturing different customers or the advice generated being different from what was envisaged cannot be ruled out. The challenge is how to put in place a monitoring approach which ensures that outcomes and risks which arise are consistent with the expectations in the business plan.

Something similar can apply from the perspective of credit risk, impacting the quality of the portfolio and performance. Suppose you have been targeting retail customers with a specific risk rating for a credit card business. If you roll out an AI-based solution to enhance the efficiency of product underwriting, you would need to have in place mechanisms to ensure that the credit quality of the portfolio is consistent with your expectationsor else change those expectations. Both options are fine. You may want to keep your target credit rating constant and seek more volume, or perhaps you see AI-based solutions as a more robust tool to support decision making and, in a controlled manner, can relax your target rating. Regardless of your choice, you would need to put in place a credit risk monitoring approach that is suited to the new AI-based solutions, as well as ensure that the business understands the portfolio implications of “learning” that is at the core of an AI-based solution system.

The salient point to take away is that the roll-out plan of AI-based tools may focus on the launch. However, the greatest challenge may well be the need to provide for the ongoing and timely monitoring of the AI-based tools and their integration in business governance and risk management, which I will cover in the next post.