What happens if right after you finalize your financial models and get ready to make critical pricing or balance sheet decisions, your head of risk modeling leaves your institution with little notice, and to your dismay, little documentation? While your former model developer was brilliant, your institution is now left with a range of financial models that may or may not be accurate or effective.
“While this scenario may seem rare or unlikely, it isn’t a scenario you want to face,” says Ray Wong, Vice President, Analytics at MountainView Financial Solutions, a Situs Company.
But identifying unlikely circumstances or hidden risks is exactly what Wong’s job entails. In fact, Wong spends much of his time combing through and analyzing modeling assumptions, data, processes, documentation and governance to protect businesses from the thing that tends to get the best of them every time: blind spots.
“When reviewing a model, it’s not always that a model is incorrect,” states Wong as he shows a graph of model validation results. “It is just that specific aspects of a model, model process or modeling data may be inaccurate, or details — such as product contractual characteristics — may be overlooked. And unfortunately, the outcome of a flawed model can, at first glance, appear picture-perfect.”
The graphs below reinforce Wong’s point. Graph 1 shows the result of a prepayment back-test of an institution’s Mortgage Servicing Rights (MSR) portfolio. The forecasted prepayment speeds (the solid line) appears somewhat in line with actual prepayment results (the dotted line). The forecast appears directionally correct; as actual Conditional Prepayment Rates (CPRs) decreases, forecasted CPRs decrease too. The forecasted versus actual CPRs are tracked over a 12-month period. Based on this graph, the forecasted monthly prepayments of this portfolio are overstated by approximately 1.6% CPR.
However, the model validation revealed that the 5/1 ARMs in the Institution’s MSR portfolio was modeled with forecasted CPRs generated by observing the prepayments of the total MSR portfolio. Because the MSR portfolio was heavily dominated by 30-year fixed-rate mortgages, applying this prepayment forecast to the 5/1 ARMs resulted in a prepayment back-test that was out of line with actual 5/1 ARM prepayments. Based on the below graphs, the actual prepayments of 5/1 ARMs are significantly different versus forecast.
In further discussing the importance of model validation, Wong points out that bank and credit union regulators require independent validation of financial models. However, from Wong’s point of view, regulation should not be the primary reason for an institution to obtain a third-party validation. An organization should want to ensure its models and model processes behave as expected since they inform significant business and balance sheet decisions. And, says Wong, the same is true for an institution’s trusted resources. If the brilliant model developer ghosts the institution, it will spend precious days trying to develop and document new models from scratch. A validation can highlight the larger blind spots as well as the more nuanced.
Wong warns, “Risk exists in the people, process and technology.”
If your institution requires a financial model validation, please reach out to firstname.lastname@example.org