Marginal effects are a regression-based process, used for many years as an analytical tool. The Consumer Financial Protection Bureau (CFPB) has been and continues to use marginal effects in its fair lending exams. Aside from the fact that regulators utilize marginal effects in their exams, financial institutions may choose to use marginal effects in their fair lending analyses for three additional reasons:
- lowering review costs
- reducing fair lending risk
- enhancing analytical versatility
In the Fall 2015 edition of the CFPB’s Supervisory Highlights, they discussed two measures of risk that they may use in their credit decision fair lending exams: (1) odds ratios and (2) marginal effects. While odds ratios are likely familiar to anyone who has worked in fair lending, marginal effects may be a relatively unfamiliar concept. Since the CFPB article in 2015, there have been other articles that mention the CFPB’s use of marginal effects, but none discuss the concept in detail.
Here we’ll explain the concept of marginal effects in lay terms, discuss why the CFPB uses marginal effects in addition to odds ratios, and describe how financial institutions that choose to incorporate marginal effects in their fair
lending analyses can reduce file review costs and fair lending risks. This article begins with a brief review of odds ratios, introduces marginal effects, discusses how they are different from odds ratios, and how these differences can benefit your institution.
Odds Ratios
An odds ratio is the ratio of a particular outcome occurring versus not occurring for one group versus another. Regulators use odds ratios as one way to quantify fair lending risk. For example, a denial rate odds ratio of 2.0 indicates that an applicant group’s declined-to-approved ratio was twice the declined-to-approved ratio of their comparator group. However, there are disadvantages to using odds ratios, including the two described below, which are mentioned by the CFPB in their 2015 Supervisory Highlights article:
- Regulators’ experience is that “…the magnitudes of odds ratios can be difficult to interpret.”1 In particular, some people confuse odds ratios for differences in denial probabilities. However, as their name suggests, odds ratios are ratios and consider denial probabilities, as well as approval probabilities. As stated by the CFPB, “An odds ratio in the context of an underwriting analysis is the ratio of the odds of a loan application denial—that is, ‘the probability of being denied’ divided by ‘the probability of not being denied (that is, approved)’—for the test group and the control group, respectively.”2
- Odds ratios are not unique. An odds ratio of 2.0 could be the result of the target group having a denied rate of 66% and the control group having a denied rate of 50% or the target group denied rate being 80% and the control group having a denied rate of 66%.3
Despite these shortcomings, odds ratios remain a part of the regulator fair lending arsenal due in a large part to their familiarity in the industry. Another benefit of odds ratios is their ability to analyze small datasets. Unlike regression analyses, which recommend at least 30 target group applications, 30 control group applications, and 100 combined applications to produce meaningful results, odds ratios requirements are less strenuous, and their results are acceptable if there are 5 target group applicants and 25 control group applicants. Nevertheless, to overcome the odds ratio limitations and legacy status, regulators may utilize an additional fair lending risk measurement called marginal effects.
Marginal Effects
In credit decision analyses, marginal effects are a measure of the difference in denial probabilities. For example, suppose the denial probability of a Hispanic applicant was determined to be 66%, but if that same applicant was White, and his/her denial probability was found to be 50%, the marginal effect for the Hispanic applicant would be 16% (66% – 50% = 16%). This result means the Hispanic applicant is 16% more likely to be denied. In summary, marginal effects as used in underwriting fair lending analyses express the absolute change in the denial probability associated with a prohibited basis group applicant versus the denial probability of an applicant who is not a member of a prohibited basis group.
So why do the regulators use marginal effects in their analyses, and why should compliance officers include them in the analyses they perform for their institutions? One reason why regulators use marginal effects in their credit decision analyses, unlike odds ratios, is that marginal effects results actually show the difference in denial probability between two groups or applications. Despite some individuals believing, or at least articulating, that odds ratios do the same, they do not. As discussed above, an odds ratio is the ratio of two comparator groups’ denied-to-approved loan ratios and do not suggest which group is more LIKELY to be denied. Marginal effects as used in credit decision analyses express the change in the denial probability associated with an applicant identifying with a prohibited basis group versus a nonprohibited basis group. This difference between denial probabilities allows us to determine if a particular applicant, or group of applicants, would be more or less LIKELY to be denied if their prohibited basis group status was reversed.
Using marginal effects allows regulators and analysts to better prioritize their exams/file reviews. Instead of organizing file reviews based on the statistical significance of prohibited basis groups, marginal effects allow analysts to determine which applicants across all applicant groups should be reviewed first via the largest marginal effect values, which indicate the applicants with the greatest fair lending risk. Note that while certain applicant groups (prohibited basis or non-prohibited basis) may not have produced harmful, statistically significant results in a regression analysis, individual applicants from these groups may possess large marginal effect values and warrant a file review. Analysts may wish to incorporate a file review strategy similar to that of regulators, which involves reviewing applicants with the largest marginal effect value in statistically significant prohibited basis groups first and then continuing to review applicants in descending order of their marginal effects values.
Another benefit of marginal effects is their analytical versatility. Whereas odds ratios strictly calculate the ratio between two comparator groups without considering any of the applicants’ risk factors, marginals effects not only consider applicant risk factors (e.g. credit score, debt-to-income ratio, loan-to-value ratio, etc.), but also allow an analyst to determine the impact a change in any risk factor has on the denial probability. For example, suppose denial rates for Whites and Hispanics are 50% and 66%, respectively. While an odds ratio might tell you that Hispanics denied-to-approved loan probability was twice that of Whites, a marginal effects analysis would consider the risk factors for Hispanics and Whites, determine the probability of denial for both applicant groups, and lastly calculate the difference between the two probabilities to determine the marginal effect value. Similarly, if an analyst wanted to know the denial rate effects of an applicant changing from the institution’s “A” credit tier to its “A+” credit tier, it could be accomplished in a similar fashion where the probability of denial is found for “A” and “A+” credit tier applicants and the difference in probabilities would provide the marginal effect value. Since a marginal effects process can examine the influence of any factor on a credit decision (applicant group, census tract demographic, credit score, branch, region, mortgage product), any one of these can be used as the “focus variable” to determine the variable’s impact on the denial probability.
Therefore, marginal effects can be used in an array of analyses like sensitivity analyses and intra and interbank comparisons. Marginal effects analysis provides banks and regulators additional forms of analysis that do not exist with odds ratios.
Conclusion
While odds ratios are a more familiar concept, they are often misinterpreted and only provide one type of analysis. The marginal effects concept on the other hand, produces results that are simple to interpret (X – Y = change in denial probability), and allow for multiple forms of fair lending analysis not readily available using odds ratios. As a result, using marginal effects will better clarify an institution’s fair lending risk. Additionally, because marginal effects produce output at the applicant level as well as the applicant group level, institutions can reduce costs by focusing their reviews on those applicants with the largest marginal effect values.
So, as someone working in fair lending, which indicator of fair lending risk should you use—odds ratios or marginal effects? If you ask your regulator you may get the familiar answer: “It depends.” Our interpretation of that answer, based on some regulator conversations, is that you should perform both methods during your fair lending analysis and understand what the odds ratio and marginal effects analyses say about your institution’s treatment of applicants. Perhaps perform the odds ratio analysis as an initial assessment then perform a marginal effects analysis as an in-depth assessment. The marginal effect results can also be used to prioritize the applicants for manual file review by moving from the largest to smallest marginal effect values.
Operationally, utilization of marginal effects analyses provides an opportunity for banks to reduce fair lending risk and lower file review costs. Furthermore, marginal effects evaluations provide bankers with a more extensive understanding of potential disparate treatment and how their institutions are treating applicants.
ENDNOTES
- Bureau of Consumer Financial Protection. Supervisory Highlights, issue 9, Fall 2015, pp. 29. Bureau of Consumer Financial Protection, https://files.consumerfinance.gov/f/201510_cfpb_supervisory-highlights.pdf .
- Bureau of Consumer Financial Protection. Supervisory Highlights, issue 9, Fall 2015, pp. 29. Bureau of Consumer Financial Protection, https://files.consumerfinance.gov/f/201510_cfpb_supervisory-highlights.pdf .
- Odds Ratio 1: (0.66/0.33) / (0.5/0.5) = 2, Odds Ratio 2: (0.8/0.2) / (0.66/0.33) = 2. Both Odds Ratio 1 and Odds Ratio 2 have an odds ratio of 2.0, but have different denial ratios.
To learn more about marginal effects and how Preiss&Associates, LLC. can help your institution make marginal effects a part of its fair lending program, contact Rick Preiss (rpreiss@preissco.com), Phil Preiss (ppreiss@preissco.com), or give us a call at (847) 295-6881.
For a downloadable PDF of this article, click here.
