August TBS PMO L&P Team Meeting

Model Governance – Transamerica ApproachDarin Zimmerman

Director, Actuarial Integrity

Actuarial Club of the Southwest Spring Meeting

June 15 & 16, 2017

2

INTRODUCTION

3

Valuing Controls is a paradigm shift from the Actuarial Wild West

► We are financial engineers at heart and engineers are always trying to improve efficiency

• Faster

• Cheaper

• Less Waste

► Controls are engineered inefficiency

• Slower

• More costly

• Redundant

Engineering 101: What is the engineer’s primary concern?

Why are we here today?

We have do this to prevent

costly mistakes!

Safety

Agenda

Introduction

Phase 1 – Model Validation & Change Management

Phase 2 – Actuarial Integrity & In Control (current state)

Phase 3 – It’s TIME to Modernize! (future state)

5

PHASE 1: MODEL VALIDATION & CHANGE MANAGEMENT

6

Started with Model Validation

Created a model Inventory, then risk-rated our models into low, medium, and high

► Inforce size and financial volatility

► Model complexity

► Product Novelty (Whole life has 75 years of data, nursing home rider on annuities maybe 10)

Validated (First round used consultants)

► Methodology

► Assumptions

► Model documentation

► SOX control procedures

Then discussed “findings” and documented SLIME (slide 13)

Base-lining (“KC0”) & Validation is first-line function

► We relied on second-line

Phase 1: Model Validation

7

Then we created the Change Management Office (“CMO”):

(has since been renamed the Model Governance Team)

Design and execute a controlled model change management process (workflow)

Oversee that change process was followed (the component verification control)

Build secure repository for models

► A stop-gap on way to true production environment

► Will include production, test, and development regions

► IT controlled and administered

► Models not run in this environment, but secure file cabinet with versioning capabilities

Maintain Model Governance Systems

Phase 1: Creation of the CMO

8

Phase 1 Change Management Workflow: High-level View

9

The last slide wasn’t our first iteration

► We started with a paper form

► The last section was peer review

► This was the consulting firm view

► Didn’t work for us

Four critical elements for a successful peer review

1. Subject Matter Expert (SME understands the work)

2. Independence from the work (Can’t review your own work)

3. Independence of the reviewer (Can’t review your boss’s work)

4. Time / Capacity to actually perform the review (easily overlooked, but critical)

Phase 1: A Few Words on Peer Review

10

Phase 1 CMO Systems (Change Management Workflow)

CMO - Actuary

Review technical actuarial

requirements

Reviewer approve?

MO or Developer

Create tech specs from requirements

documentation

MO or Developer

Complete model changes/coding/

unit testing/ docu

Peer of MO or Developer

Review coding and unit testing

Reviewer approve?

CollaborationCMO, MO,

Developer, IT

Develop UAT plan

MO or Developer

Workflow Form Communication

Model TesterExecute UAT

Reviewer approve?

Model Owner

Present results to governance & Corp

Risk for approval

CMO - Actuary

Final review for proper governance and documentation

Reviewer approve?

CMO - Operations

Promote model to production; update SLIME & change log

No

Yes

Model Supervisor

Acceptance of UAT results

Governance

Accepts and reflect in meeting minutes

Transamerica Management

Approves Model change

Model Owner

Submit TBS Model Change Request

Form

No

Yes

No

Yes

No

Yes

Collaboration MO, SBU, IT, AMERC

BusinessRequirements In

itia

tio

nD

eve

lop

me

nt

Test

ing

Go

ve

rna

nce

Model Owner

Acceptance of UAT ResultsCMO – Actuary

Execute CVC

11

Phase 1 Question: “Does this need a ticket?”

Data

Logic

Assumptions

Routine Production

Set up

Non-routine Change

Management

Model Start Date, Policy Inventory, Asset inventory, Bridge Scenario, Sensitivity Multipliers, RBC Factors

IT Changes to Admin Systems & Record Layout

Economic Scenario FilesUpdating Expense & Mortality Tables

Expenses, Mortality, Morbidity, Surrender, Static & Dynamic Lapse, Competitor Rate, Crediting Strategy, Reinvestment Strategy, Long-term Mean Reversion

Product Data File Setup, User Defined Function Coding, Database Coding, Changes to Spreadsheet Formulas Changes to Macros and VBA FormulasChanges to Table Names & Dimensions

DAC Spreadsheets

SOX Controls: Change Management or Production?

12

1. Risk Model Catalog (RMC)

2. SVN – Secure Repository

3. Change Management Workflow

RMC comprised of:

► The Model Inventory

► The SLIME log

► The Assumption and Methodology Repository

The Model Inventory Captures Essential Model information

► Model Name

► Risk Rating (based on complexity, novelty, and materiality)

► Model Owner and Model Supervisor

► Commercial projection system used

► Model Groupings

Phase 1: Model Governance Systems

13

SLIME Log

► Simplifications

► Limitations (in the modeling platform)

► SOX control Issues (as in execution process)

► Maintenance (update assumption, new regulations)

► Errors

• Actuarial Error: Does model do what it was intended to do?

• Accounting Error: Is the number materially correct?

SLIME is an unfortunate acronym: It is not a list of problems to be fixed: it is a list of opportunities to improve model precision; most are not cost justified

Phase 1 Model Governance Systems

14

The Assumptions and Methodology Repository

► Not just an inventory, but future state vision is a source for populating models at runtime to ensure we are using the properly authorized assumption set for a particular run date.

Three Basic Elements

1. Assumption definition (name, type, model, etc.)

2. Review information (multiple histories)

3. Implementation (multiple histories)

Phase 1 Model Governance Systems

15

We had to be precise with our language► Management wanted a “secure environment”

► In the short-term we could only deliver a “secure repository”

► Significant IT involvement to achieve the former

Currently we are creating an SVN region for our experience studies (Technically Phase II)

► Technically, they meet our definition of model, but were scoped out because of resource constraints and that wasn’t really the objective of Phase I

Phase 1 Model Governance Systems (SVN – Secure Repository)

16

Phase 1 Model Governance Systems (SVN – Secure Repository)

17

Ensure no unintended changes were introduced – compare the current production model with the previous production model

► Use Beyond Compare to verify each file

• Supposed to change and did (GOOD)

• Not Supposed to change and didn’t (GOOD)

• Changed when wasn’t supposed to (BAD)

• Didn’t change when it was supposed to (BAD)

► Compare Spreadsheets with Excel 13

► Compare other MS files with Beyond Compare

► Use other platform tools for Axis, ALFA, MoSes, ArcVal, Poly

► Maintain database of reports for routine and Non-routine

Phase 1 Model Governance Systems Component Verification Control

18

PHASE 2: ACTUARIAL INTEGRITY & IN CONTROL (CURRENT STATE)

19

Coincided with Transamerica Functional Reorganization► Actuarial Functional area still has 3 Valuation Legs and 3 Pricing Legs (based on product line)

► Appointed Actuary Team is centralized

► Actuarial Integrity Team is centralized (Corporate was removed, but if any group is Corporate Actuarial, we are it)

► All of the above report to one Chief Actuary of Transamerica

Actuarial Integrity is responsible for:► Actuarial Model Integrity (1st line KC0 function)

► Model Governance (formerly CMO); with additional independent Actuarial Review of assumptions and methodologies

► Experience Analytics (Experience Studies, Predictive Analytics, Assumption Setting, Controls)

► Assumption and Methodology Governance

Phase 2 – Creation of Actuarial Integrity Area

20

Transamerica reviewed Model Governance controls in 2016

► Enhanced controls associated with:

• Assumption Approvals (AMR)

– Introduce manual controls to reconcile governance to workflow

• Audit Trails for Unintended Changes

– Introduce procedures to track file migration from directories to SVN

• Introduce Enhanced Reporting to Promote Ownership of Model Governance Responsibilities

– Changes in SLIME

– Model Inventory

– Looking to assign Model Liaison

– Documentation is KEY!

» Being able to follow the study, recommendation, how you set the assumption, rationale for expert judgment, etc.

» TA has started to look at technical writers, so far with mixed results

Phase 2 – In Control

21

Phase 2 – Committee Governance Approval Structure

Supposedly Congress only does two things: Nothing and Over-react

22

PHASE 3: IT’S TIME TO MODERNIZE! (FUTURE STATE)

23

When it comes to our

finance and actuarial

processes and

platforms….

It’s TIME to modernize

24

Transamerica’s

Initiative to

Modernize our

Enterprise

T

I

M

E

Initiative to

Modernize our

Enterprise

25 25

Phase 3: Modernization: ATOM and One-MDLC

Actuarial Target Operating Model (“ATOM”) and One-Model Development Life Cycle (“One-MDLC”) are two of nine projects in the

Modernization program, focused on future state actuarial systems and target operating model, and development of a Model

Development Life Cycle that is consistent across TA

ATOM –Outcomes

• Decision-making framework

• List and description of business capabilities

• Tool inventory categorized by product, process and

calculation framework

• High level architecture supporting actuarial work

• Future state operating model

• Timetable for platform and tool review

One-MDLC –Outcomes

• Model and assumption governance framework

• Creation of an IT controlled production environment

• System selection for governance workflow and inventory

management

• Rollout effort plan/roadmap for other processes

• Assumptions store design and build

• Expanded reporting capabilities

26 26

Phase 3: Process, Controls & Governance – key pieces

The conceptual target operating model is founded on using a repeatable process to leverage key pieces for the different product lines

Actuarial Governance

Process

Realign roles and

responsibilities of actuarial

processes to the right ability

Change Management

Mold change

management process

around the new

environments

Systems Management

Continuous review and

management of actuarial

systems and tools

.

Controlled

Environments

Development, testing,

production, and post-

production in a

controlled setting

27

Phase 3: Framework