model risk management - elliott davis · model risk management ... are set in line with the ank’s...

Model Risk Management

© Elliott Davis Decosimo, PLLC

Michael Koupal, CPA Senior Manager

Melody Reed, CRCM, CFSA Manager

This material was used by Elliott Davis Decosimo during an oral presentation; it is not a complete record of the discussion. This presentation is for informational purposes and does not contain or convey specific advice. It should not be used or relied upon in regard to any particular situation or circumstances without first consulting the appropriate advisor. No part of the presentation may be circulated, quoted, or reproduced for distribution without prior written approval from Elliott Davis Decosimo.


Disclaimer

Per the OCC Supervisory Guidance on Model Risk Management (OCC 2011-12)

A model consists of three components:

• Information input (which delivers assumptions and data to the model)

• Processing (which transforms inputs into estimates)

• Reporting (which translates estimates into useful business information)


Model Definition

Model Risk occurs for two primary reasons:

1. The model may have fundamental errors and may produce inaccurate outputs Errors can occur at any point from design through implementation.

2. The model may be used incorrectly or inappropriately. Even a fundamentally sound model producing accurate outputs consistent with the design objective of the model may exhibit high model risk if it is misapplied or misused.


Model Risk

• Model risk increases with greater model complexity, higher uncertainty about inputs and assumptions, broader use, and larger potential impact.

• Even with skilled modeling and robust validation, model risk cannot be eliminated, so other tools should be used to manage model risk effectively.

• establishing limits on model use

• monitoring model performance

• adjusting or revising models over time

• supplementing model results with other analysis and information


Model Risk (Continued)

• Model validation is the set of processes and activities intended to verify that models are performing as expected, in line with their design objectives and business uses. Effective validation helps ensure that models are sound. It also identifies potential limitations and assumptions, and assesses their possible impact.

• All model components, including input, processing, and reporting, should be subject to validation; this applies equally to models developed in-house and to those purchased from or developed by vendors or consultants.


Model Validation

• Validation involves a degree of independence from model development and use. Generally, validation should be done by people who are not responsible for development or use and do not have a stake in whether a model is determined to be valid.

• Staff doing validation should have the requisite knowledge, skills, and expertise. A high level of technical expertise may be needed because of the complexity of many models, both in structure and in application. These staff also should have a significant degree of familiarity with the line of business.


Who should complete the validation?

• The range and rigor of validation activities conducted prior to first use of a model should be in line with the potential risk presented by use of the model.

• Validation activities should continue on an ongoing basis after a model goes into use, to track known model limitations and to identify any new ones.

• Validation is an important check on model use during periods of benign economic and financial conditions, when estimates of risk and potential loss can become overly optimistic, and when the data at hand may not fully reflect more stressed conditions.

• Banks should conduct a periodic review—at least annually but more frequently if warranted—of each model to determine whether it is working as intended and if the existing validation activities are sufficient. Such a determination could simply affirm previous validation work, suggest updates to previous validation activities, or call for additional validation activities.


How Detailed? How Often?

• An effective validation framework should include three core elements:

- Evaluation of conceptual soundness, including developmental evidence (Quality of model design and construction)

- Ongoing monitoring, including process verification and benchmarking

- Outcomes analysis, including back-testing


Key Elements of Comprehensive Validation

• Validation should ensure judgement exercised in model design and construction is well informed, carefully considered, and consistent with published research and sound industry practice

• How is this handled when model is outsourced?

• Can sensitivity analysis help evaluate?

- Sensitivity analysis – measuring the impact inputs have on model outputs


Quality of design and construction

Michael Koupal, CPA

Email: [email protected]

Phone: 704.808.5211

Website: www.elliottdavis.com

Melody Reed, CRCM, CFSA

Email: [email protected]

Phone: 919.987.2776


Elliott Davis Decosimo ranks among the top 50 CPA firms in the U.S. With seventeen offices across seven states, the firm provides clients across a wide range of industries with smart, customized solutions. Elliott Davis Decosimo is an independent firm associated with Moore Stephens International Limited, one of the world's largest CPA firm associations with resources in every major market around the globe. For more information, please visit elliottdavis.com.


http://www.elliottdavis.com/


BSA Models: Understanding and Maximizing the Backbone of Your AML Program


Melody Reed, CRCM, CFSA Manager

• Purpose of the Model – Why do I need this?

• Key Areas of the Model

• Tuning – Did I Buy a Car?

•Understanding the Importance of the Model


Objectives/ Areas to Cover

Manual Reports = Ineffective and Inefficient Monitoring

Regulators Make Us It’s the Right Thing To Do


The “Why”


Automated Monitoring

Before After

•Who has to understand how the model works? Isn’t that an IT thing?

•No – BSA Officers should understand the functionality of the model, including details about how the model is scoring and flagging transactions for additional review.


Understanding the Model

“Alert”

• YellowHammer BSA = Worklist Item

• Banker’s Toolbox BAM = Report Item

• Fiserv’s FCRM = Alert


Definition of Terms

• Customer Information

- Customer Due Diligence

- Customer Risk Scoring

- Ongoing Due Diligence


Key Model Areas

• Suspicious Activity Monitoring

- Unusual Transactions

- Out of Pattern Behavior

- Transactions in High Risk Areas


Key Model Areas (Continued)

Too Much Static or “Noise”


Tuning – Like a Radio?

reduce the noise to get a clearer picture


Tuning

Tighten the net…

… and loosen it.


Tuning and Optimization


T&O (Continued)

• Evaluate Coverage Based on Triggering Events

– Manual SAR Referrals

– Regulation Changes

– Bank M&A Activity

– New Products

– Enforcement Actions

• Evaluate Thresholds

– Dollar Amounts

– Transaction Frequency

• Challenge Customer Risk Scores

– New Markets

– High Risk Areas

•Model Validation

- OCC 2000-16: Model Validation

•Model Risk Management

- OCC 2011-12 and SR Letter 11-7: Model Risk Management


Making Sure It Works

• Reviewing system parameters, settings, security and validating that the system is working

•Assessing the setup of the model to ensure appropriate coverage in terms of customer risk and transactional risk

• Reviewing parameters and thresholds to verify they are set in line with the Bank’s size and BSA risk profile

Elliott Davis Decosimo Approach


• Independent Review

•Defined Responsibility

•Model Documentation

•Ongoing Validation

•Audit Oversight

Keys to Model Governance


Melody Reed, CRCM, CFSA Email: [email protected]

Phone: 919.987.2776




mailto:[email protected]


Interest Rate Risk and Management



• Interest Rate Risk

- What is IRR?

- Current Regulatory Focus

- Internal Control System

- Independent Review and Validation


Overview

• Banks are in the business of managing IRR

- Repricing Risk: timing differences between coupon changes or cash flows of assets and liabilities

- Yield Curve Risk: non-parallel changes in yield curve

- Option Risk: cash flows change with embedded options (prepayment/extension, call options, runoff)

- Basis Risk: different indices with same maturity move at different pace


What is IRR?

•Margin pressure is hindering meaningful earnings recovery

• Increases in long-term asset exposure to support yield coupled with surge in non-maturity deposits

• Fear of substantial deposit runoff (surge deposits and parked funds)

• Examiner focus on assumptions, sensitivity analysis, internal controls/validation


Current Regulatory Focus

• Board established system of internal controls

- Corporate governance

- Compliance with policies and procedures

- Comprehensive measurement system


Internal Control System

• Roles, responsibilities, and authority

•Adequate segregation of duties

• Inputs and measurements are accurate and complete

• Policy compliance

• Independent review and validation

•Management response and follow-up

• Size, nature, and complexity of institution should be incorporated in evaluating all aspects


Effective Control Structure

• Review/Test

- Lines of authority

- Segregation of duties

- Corrective actions

- Compliance with risk limits

• Ensure staff compliance with procedures


Adequacy and Compliance of Control System

•Data Integrity

- Is data accurate, complete, and useful?

- Source of data

•Data Input Controls

- Automatic vs. Manual input

- Reconciliation and review process

• Test Data Inputs

- Balance sheet

- Budgets/forecasts

- Assumptions


Data Inputs

• Reasonableness

- Can compare to historical and current data

- If using peer or national data, should still determine if reasonable for your institution

- Should be based on expectations, not just budget

•Documentation

- Understandable format and includes all assumptions

- Basis for balance sheet predictions

- Conclusions and strategies developed based on identified risks


Assumptions

• Sensitivity analysis

- Which factors are most important? (Stress Testing)

• Sufficiency of modeled scenarios

- Reasonable range of rate changes and models

• Board approval and understanding


Assumptions

• Internal Models

- Significant amount of time required for validation process.

- Includes validation of model mechanics and mathematics.

• External Models

- Vendors normally provide validation results. Management should review and assess at least annually.


Validation

• Compare Modeled vs. Actual Results

- Who should complete?

- Annually or quarterly?

- Should include rate vs volume variance

- Detailed enough to determine accuracy

•Were assumptions accurate?

- If not, has management identified changes for future modeling?

• Identify causes of differences


Backtesting

• Report to Board/Audit Committee

- Testing details

- Findings summary

- Key assumptions

- Management’s responses


Reporting

• FIL-52-96 - Joint Agency Policy Statement on Interest Rate Risk

- http://www.fdic.gov/news/news/financial/1996/fil9652.html

• FIL-2-2010 - Financial Institution Management of Interest Rate Risk


• FIL-2-2012 - Interest Rate Risk Management: Frequently Asked Questions


• FIL-46-2013 - Managing Sensitivity to Market Risk in a Challenging Interest Rate Environment

- https://www.fdic.gov/news/news/financial/2013/fil13046.html


Guidance

http://www.fdic.gov/news/news/financial/1996/fil9652.html



https://www.fdic.gov/news/news/financial/2013/fil13046.html

Allowance for Loan and Lease Loss Validations



This material was used by Elliott Davis Decosimo during an oral presentation; it is not a complete record of the discussion. This presentation is for informational purposes and does not contain or convey specific advice. It should not be used or relied upon in regard to any particular situation or circumstances without first consulting the appropriate advisor. No part of the presentation may be circulated, quoted, or reproduced for distribution without prior written approval from Elliott Davis Decosimo.


Title

• ASC 450 (FAS 5) General Reserve

- Historical Loss Factors

- Environmental Factors

• (Qualitative adjustments to historical loss rates)

- What if a migration analysis is used?

• ASC 310-10-35 (FAS 114) Specific Reserve

- Impaired Loans/Troubled Debt Restructures (“TDRs”)


Overview

• Historical Loss Factors - Calculated using historical charge-offs by loan

pool over a designated time period

• Test loss factors by :

- (a) Recalculating the historical loss factor

• Subject a sample of current year charge offs to proper approval and recording

• Reconcile charge-offs / recoveries to allowance roll forward and GL

• Verify loss history is properly applied against average loan balances of proper pool

• Verify impaired loans are excluded from the outstanding average loan balances in the application of the loss factors in order to avoid layering


General Reserve - ASC 450 (FAS 5)

• Historical Loss Factors

• Test loss factors by :

- (b) Evaluating the appropriateness ofthe historical loss period

• Usually between 8 quarters and 15 quarters is reasonable historical loss period


General Reserve - ASC 450 (FAS 5)

•What happens if the Bank is using a migration analysis instead of a historical analysis for their general reserve?

- Need to gain an understanding of the migration to determine if the methodology is reasonable

- Recalculate at least a sample to determine if the system is working properly (trace/vouch)

- Great use of Excel

- If using third party system, see if they’ve already done a certification/validation on the model so you only have to focus on the input and outputs.


Migration Analysis

• Environmental Factors

• Trends in delinquencies and impaired loans

• Levels and trends in recoveries and charge-offs

• Trends in volume and terms of loans

• Experience and ability of lending management and relevant staff

• National and local economic trends and conditions

• Credit concentrations

• Supportable/documented

• Reasonable

Environmental Factors


• Impaired Loan

- Impairment occurs when it is probable that the entity will be unable to collect all amounts due according to the contractual terms of the receivable

- All amounts due according to the contractual terms means that both the contractual interest payments and the contractual principal payments will be collected as scheduled according to the receivable's contractual terms. Need not consider an insignificant delay or insignificant shortfall in amount of payments.


Specific Reserves ASC 310-10-35 (FAS 114)

•Methods for calculating impairment

- Impairment calculated based on fair value of collateral

• For collateral dependent loans

- Impairment calculated based on present of expected future cash flows

• For non collateral dependent loans

- Fair Value (rarely used) © Elliott Davis Decosimo, PLLC


• Collateral dependency

- Repayment is expected to be provided solely by the underlying collateral

• Should adjust for selling costs (taxes, repairs, agents, etc.)

- Repayments from proceeds of sale of collateral

- Cash flows from continued operation of collateral

• Apartment building, shopping mall

• Cash flows are derived solely from the property’s rental income



• Sale of underlying collateral

• It is important to: - Evaluate the professional qualifications of the appraiser

- Consider the date and age of the appraisal

- Obtain an understanding of the appraiser’s relationship to the client and consider the objectivity of the appraiser

- Obtain an understanding of the methods and assumptions used by the appraiser

- Make appropriate tests of data provided to the appraiser, including the legal description of the property and any other assumptions such as expected cash flows



• Impairment based on PV of expected cash flows

• Examine evidence that supports management’s expected cash flows. (For example, evidence might include borrower financial statements and income tax returns.)

• Consider contradictory evidence that suggests that management’s cash flow expectations are unreasonable.

- Contractual payment terms required under modified loan are not necessarily the best estimate of expected future cash flows




• Compare the discount rate used in the cash flow calculation

to the loans original effective rate (not modified rate)

• Test the clerical accuracy of the cash flow calculation

• Consider default and prepayment assumptions

• Environmental factors – industry, geographical, economic, political



• Impairment based on PV of expected cash flows - Cash flows from other available sources (including guarantors)

must be more than nominal to conclude a loan is not collateral dependent

- E.g., cash flows generated by operation of a business or other source outside of lender’s security interest in the collateral

- Balloon payments

- Significant uncertainty may exist regarding the borrower’s ability to refinance/pay the loan off at maturity when contractual balloon payments are required

- Acceptable approach is to utilize the fair value of collateral (less costs to sell) as expected future cash flows at maturity

• Unless balloon payment amount is less than FV of collateral; in those cases, use the balloon payment




- In general, GAAP does not allow impairment calculations to run beyond the contractual term

- Exceptions

• E.g., automatic renewal at maturity

• Regulators are open to possibility of adjusting the impairment measurement by a qualitative factor if quantifiable/objectively measured

- More accurately approximate the risk and economics of the relationship between the institution and borrower

- Key point is to have a well-documented workout plan if measuring impairment beyond contractual term



• Troubled Debt Restructuring • Loan modified by granting a concession on

- Rate

- Term/maturity extension

- Payment amount

- Interest or principal forgiveness

• For economic or legal reasons related to the borrowers financial difficulties

- Should be considered impaired and accounted for as an impaired loan in accordance with ASC 310-40



• Troubled Debt Restructuring • It is ok to pool, non-collateral dependent, insignificant loans?

• How would the reserve be calculated?

- Search for TDRs

- Options for removing TDRs ASC 310-20 (FAS 91)



Michael Koupal, CPA Email: [email protected]

Phone: 704.808.5211





model risk management - elliott davis · model risk management ... are set in line with the ank’s...

Documents