AI Explainability 360 - Demo

A Data Scientist wants to understand:

What is the overall logic of the model in making decisions?
Is the logic reasonable, so that we can deploy the model with confidence?

Using Directly Interpretable Models for Global Understanding

In evaluating an AI model for deployment, the Data Scientist would ideally like to understand the behavior of the model as a whole, not just on specific loan applicants. For example, they may have to present the model to: 1) managers for review before deployment, 2) a lending expert to compare the model to the expert's knowledge, or 3) a regulator to check for compliance.

Directly interpretable models can provide such global understanding. We demonstrate this using the Generalized Linear Rule Model (GLRM) algorithm in the AI Explainability 360 Toolkit, which provides an enhanced level of explainability. The algorithm produces models that are weighted combinations of rules (e.g., debt < $10,000, number of accounts > 10), which give insight into loan repayment predictability. The algorithm also has the option of combining rules with linear terms, which we will use here and explain below. The model output predicts the probability of repaying on time (Y=1).

Rules and Linear Terms Learned from the FICO Dataset

Below on the left are the rules and linear terms that GLRM has learned for three of the features from the FICO dataset. You can find the rules for the full set of features in this Python notebook. For each predictor variable, a linear term was learned that represents the general trend of how the prediction score (computed as log-odds, higher score indicates higher probability to repay the loan) changes as the variable changes. At the same time, several rules were derived regarding each variable to capture the non-linear changes to the prediction score at some specific values.

To make it easier for data scientists to see the overall effect of the rules related to a variable, the AIX360 toolkit provides a function to create summary plots for a variable. Three such plots are shown below on the right for the rules listed on the left. One can easily see from these plots whether the variable is positively or negatively influencing the prediction score (by the slope of the lines), and the location and size of the non-linear changes indicated by the rules related to that variable (by the vertical lines that occur at the rule thresholds). The bars below each graph summarize the relation of the variable to the prediction score, with the length of the bar indicating the significance of the variable to the prediction, and the color of the bar indicating the direction of correlation (green indicates positive, and red indicates negative).

The list of rules and linear terms and the associated visualizations give the Data Scientist a global view into how the model predicts repayment probability. The Data Scientist can verify that the features identified as important to the prediction and the directions and sizes of their contributions all make sense to someone knowledgeable about lending.

In the examples below, the Data Scientist can see that ExternalRiskEstimate is positively associated with a person's likelihood to repay the loan, and this likelihood gets additional boosts when ExternalRiskEstimate is greater than 69, 72, and 75. The Data Scientist can also see that NetFractionRevolvingBurden is negatively associated with a person's repayment likelihood, whereas MSinceMostRecentDelq does not affect the repayment likelihood in general except for a change at 21 months.

The Data Scientist can also see that ExternalRiskEstimate has a larger impact on repayment likelihood than MSinceMostRecentDelq because the lines span a larger range (from -1 to 1 for ExternalRiskEstimate, and from -0.5 to 0 for MSinceMostRecentDelq) and the green bar below each graph is longer for ExternalRiskEstimate.

Click on the info icon on each plot to learn more about what the features mean and how they contribute.