updated - report writing using the pcir--handouts · of the mmpi-2-rf, the university of minnesota...

2016 ABPP Annua l Confe rence & Workshops Chicago, IL

Thursday , May 12 , 2016

Evidence-Based Report Writing Using the MMPI-2-RF Police Candidate Interpretive Report

Presented by

David M. Corey, PhD, ABPP

David M. Corey, Ph.D., ABPP May 12, 2016

Chicago 1

Evidence-Based Report Writing Using the MMPI-2-RF PCIR

7th Annual ABPP Conference & WorkshopsChicago, Illinois

May 12, 2016

David M. Corey, Ph.D., ABPPCorey & Stewart, Consulting Psychologists

5285 SW Meadows Road, Suite 311Lake Oswego, Oregon 97035

[email protected]

Copyright Notice

May 12, 2016 2

Copyright © 2016 David M. Corey

All Rights Reserved

David M. Corey, Ph.D., ABPP

Disclosure

I receive research funds from the publisher of the MMPI-2-RF, the University of Minnesota Press. As coauthor (with YossefBen-Porath) of the MMPI-2-RF Police Candidate Interpretive Report (PCIR), I receive royalties on the sale of the report.

3May 12, 2016 David M. Corey, Ph.D., ABPP

Slide Changes All additions and revisions to the

slides are noted with a starburst in the upper right corner

Revised handouts will be made available to attendees following the workshop

May 12, 2016 4David M. Corey, Ph.D., ABPP


Chicago 2

Focus of Workshop Optimizing efficiency, defensibility,

transparency, and relevance in preparation of written reports of preemployment psychological evaluations of law enforcement candidates


Agenda Focused on 5 principle litigation contexts

in police and public 42 USC 21 § 1983 Grievance arbitrations Civil suits alleging discrimination (ADA and

Title VII) Civil service and U.S. Merit Systems

Protection Board appeals Writs of mandamus


Features of the PPE Evaluation criteria are framed as the

absence of problems likely to impair performance and resilience, not on strengths or qualities (select-in)

Assessment of strengths or qualities is only for the purpose of assessing protective or mitigating factors


Appelbaum, K. L. (2010)

“The quality of our reports is often the most tangible and visible measure of

our professionalism.”



Chicago 3

Otto, R. K. (2009)

“The only time the expert is in complete control of the information that is and is not presented is when writing the report. As a result, it is when writing a report that one can clearly meet ethical obligations.”


DeMier, R. L. (2016)

“The report stands alone as the enduring and tangible evidence of the

results sof the evaluation.”


Report Has Multiple Audiences Hiring agency Candidate Second-opinion psychologist or other

expert reviewer Plaintiff’s counsel Civil service commission or appeal board Court of ultimate jurisdiction Licensing board/ethics committee


Functions of the Report Reach a suitability determination Document it Explain the rationale for it Describe the evidentiary support for all

opinions Dissuade challenges Repel criticism Create a testimonial record



Chicago 4

Grisso’s (2008) Principles1. Let the evaluation criteria drive,

guide and limit the content of the report

2. Report what is necessary. Do notreport what is not necessary. Not necessary if unrelated to the criteria Justify inclusion and exclusion on the

basis of relevance


Grisso’s (2008) Principles3. Sequence and describe information

in a way that is useful and understandable to the reader

4. Separate facts from inferences and opinions. Inferences and opinions reflect what is

suggested by the facts Inferences and opinions should be clearly

linked to the facts supporting themMay 12, 2016 15David M. Corey, Ph.D., ABPP

Grisso’s (2008) Principles5. Explanations involve showing why

you think what you think Connect the dots Be transparent

6. Explain your rejection of other possible opinions and conclusions Explaining rejected ideas is an essential

component of transparency Essential for future testimony



Chicago 5

IACP-PPSS Guideline 11 Minimum report content: “clear determination of the applicant’s

psychological suitability for employment based upon an analysis of all psychological assessment materials”

“expressly linked” to the “skills, behaviors, attributes, and other personal characteristics associated with effective and counterproductive job performance”


IACP-PPSS Guideline 11 use clinical diagnoses or psychiatric labels

only if “relevant to the examiner’s conclusions, necessary for the hiring agency to make a hiring decision, and/or required by law”

“is focused on the individual applicant’s ability to safely and effective perform the essential functions of the position under consideration”

“clearly state” the period of report validity


No Single Report Format

Format/content informed by: State mandate Requirements of the hiring agency

IACP-PPSS Guidelines 11.1: Any agency-specific restrictions or

other requirements relevant to the format or content should be determined in advance


Standard (Minimum) Components Name of candidate Birthdate Date of evaluation Position Hiring agency Name, degree and

license number of psychologist

Methodology

Suitability determination or rating

Summary of findings supporting the determination or rating

Period of report validity



Chicago 6

Elective Components Method of

identification Non-name

identifier (SSN or other unique code)

Description of screening dimensions

Screening dimension ratings

Cautionary caveats and declarations

Summary or details of self-reported history


Cautionary Caveats: Dr. Mark Zelig ”This psychological assessment is to

evaluate this candidate sole for the above position. This is a summary and does not included all the information I learned. If the background check is discrepant with the history contained herein, please advise me and I will amend my conclusion accordingly.”


Cautionary Caveats: Dr. Mark Zelig “This report is confidential. Its

distribution should be limited to only those who are directly involved in the hiring decision. This evaluation should be filed where access is limited to only the chief administrator of your department.”


Cautionary Caveats: Dr. Mark Zelig “Compliance with the Americans with Disabilities

Act (ADA) and Genetic Information Nondiscrimination Act of 2009 (GINA): The candidate received a written conditional offer of employment that indicated he would be considered for employment upon passing the psychological evaluation. In compliance with GINA, the evaluator did not inquire nor record any information about psychopathology or other illness in biological or adopted relatives. Furthermore, the candidate was instructed to refrain from providing such information.”



Chicago 7

Declarations: Dr. Mark Zelig “The steps taken to complete this

evaluation included:1. “Before the candidate commenced testing, he

executed a general informed consent. A supplemental disclosure advised him that all of our interactions would be electronically recorded. The candidate also completed the social history portion of the Biographical Data Information Form (PDIF).”


Declarations: Dr. Mark Zelig2. “Once I received the above documents, this

candidate took the paper-and-pencil tests and other questionnaires at my officer under the supervision of a proctor in accordance with protocol. I reviewed these materials prior to conducted the interview.”


Declarations: Dr. Mark Zelig3. “At the beginning of the interview, I reviewed the

informed consent documents described above. The candidate was further instructed that this entire process was voluntary and he could stop the process at any time or decline to answer a particular interview question if he felt it was inappropriate. Given the option of declining to answer a question, I told the candidate if he chose to answer a question, I expected an honest response. I also reminded him that our interactions were being electronically recorded. Once the candidate demonstrated an understanding of the conditions noted in the informed consents, I proceeded with the interview.”


Restricted vs. UnrestrictedReporting of Personal History “There is no reason in law or policy why

an employer should be allowed access to detailed family or medical histories of its employees, or to the intricacies of its employees’ mental processes, except with the individual’s freely given consent to the particular disclosure or some other substantial justification.” Pettus v. Cole, 1996



Chicago 8

Restricted vs. UnrestrictedReporting of History An employer is “not entitled to disclosure

of … the intimate and irrelevant details of [an individual’s] home life.” McGreal v. Ostrov, 2004


GINA, Title II Prohibits the acquisition and use of

genetic information in hiring and employment

Genetic information includes: information about the manifestation of disease or

disorder in family members, including children, siblings, parents, half-siblings, nieces, nephews, grandparents, great grandparents, great-great grandparents, aunts, uncles, great aunts and uncles, first cousins, and children of first cousins


Minimum Narrative Content Ratings of “unqualified,” “unsuitable,” or

“not recommended” require justification, transparently linking data/findings with conclusions, and conclusions to criteria

Ratings of “qualified,” “suitable,” or “recommended,” in the context of substantial negative test data or personal history, require reconciliation


A Data Integration Model Spilberg & Corey (2014)

1. Assess protocol validity2. Assess substantive scale findings3. Assess background findings4. Assess interview findings5. Identify relevant risk findings6. Refine the assessment process



Chicago 9

Step 1:Assess Protocol Validity Validity requires reliability The greatest threat to reliability of an

individual’s test results is: Lack of thoroughness and attention in

answering the items (i.e., unanswered items and response inconsistency)

Excessive underreporting


Underreporting Some underreporting is expected

Absence of underreporting is uncommon and a possible red flag

Impact of underreporting is asymmetrical: Non-elevated substantive scale scores cannot be

interpreted as indicating the absence of problems assessed by those scales

Elevated substantive scales can be interpreted, but may underestimate problems


Step 2:Assess Substantive Scales Identify and interpret relevant

findings, using general adult and police candidate norms Assess for significance against standard

norms (no adjustment to cutoff scores) Assess for comparison group significance Assess for convergent, divergent, and

complementary findings


Interpreting Subclinical Scores Never infer meaning without adequate

evidence to support the inference Never infer psychopathology from a

subclinical score even if it is substantially deviant from the comparison group mean



Chicago 10

Step 3:Assess Background Findings Evaluate personal history information

from all sources to determine: If any information meets agency standards

for disqualification How the information is convergent with,

divergent from, and complementary to relevant test findings and interpretations


Step 4: Assess Interview Findings Assess how the interview findings and

clinical observations converge with, diverge from, or complement test and background findings


Step 5: Reach a Risk Determinationa. Considering all risk-related findings from all sources,

what evidence-based inferences can be drawn from them? Eliminate those that do not map onto the selection

criteriab. What divergent findings mitigate these inferences?

Eliminate those inferences that are outweighed by divergent findings of sufficient relevance, validity and reliability

c. Are any surviving risk-related inferences of sufficient relevance and quality to warrant the candidate’s disqualification?


Mitigating Negative Data

Wernimont & Campbell (1968) Past performance is relevant in predicting

future performance only if in similar contexts

In the absence of samples from a similar context, validated signs (tests) are superior to dissimilar samples



Chicago 11

Step 6: Refine the Assessment Obtain feedback from the hiring

agency to determine the status of hired candidates and make adjustments to the prediction strategy, as necessary


PCIR Features Standard MMPI-2-RF profile with

expanded Police Officer Candidate comparison group


Expanded Police Candidate Comparison Group

North American RegionGender

TotalMale Female

Pacific 136 136 272

West 381 381 762

Midwest 131 131 262

South 224 224 448

Northeast US and Canada

165 165 330

Total 1,037 1,037 2,074

PCIR Features Transparency

All interpreted scores are explicitly identified

The source(s) for every interpretive statement are annotated

Intended to provide users with economic efficiency

Provides comparison group base rates for interpreted scores



Chicago 12

PCIR Features Organizes job-relevant behavioral

correlates by California POST-like dimensions

Guidance for interpreting deviant underreporting scale scores Lists items with potentially verifiable

historical content to facilitate scrutiny

Item-level information, incorporating comparison group base rates


User’s Guide for the PCIR Complete description of

the structure and rationale for the report

Comprehensive list of references

Summaries of the key empirical studies underlying the report

Detailed instructions for producing reports via Q-Local and Q-global


Case Example:

Mr. C

Moderate Elevations Profile


Case Example:

Ms. D

Underreporting Profile



Chicago 13

Historically Verified Deception In addition to presenting herself as very well-

adjusted and having an implausibly virtuous character reported by only 1.1% of police officer candidates, some of Ms. D’s responses are contradicted by other facts revealed in the interview and background investigation. The candidate’s responses to psychological testing preclude reliance on them as evidence of the absence of problems they measure and predict.


Demonstration Case:

Mr. G


Case Material Background Agency-provided background

investigation report Self-reported (PsyQ, interview)

Testing MMPI-2-RF (PCIR) CPI-434 (PPSSR)

Clinical interview



Chicago 14

Integration Model Applied1. Assess protocol validity

a. No indications of over- or under-reporting

b. L-r=42T Lowering L-r is associated primarily with

increases in behavioral-externalizing domain (BXD, RC4, JCP, DISC-r)

c. K-r=66T



Chicago 15

Integration Model Applied2.Assess substantive scale findings

a. BXD=60Tb. RC4=68Tc. JCP=70Td. SUB=55Te. AGG=56Tf. DISC-r=69Tg. RC3=34T



Clinical Findings: RC4/JCP



Chicago 16

Diagnostic Considerations


Comparison Group Findings BXD: Only 4.4% of comparison group

members convey this or a greater level of behavioral difficulties.

DISC-r: Only 2.1% of comparison group members give evidence of this or a greater level of disconstraint.

RC4: Only 0.9% of comparison group members demonstrate this or a greater level of antisocial behavior.


Comparison Group Findings JCP: Only 3.8% of comparison group

members give evidence of this or a larger number of juvenile conduct problems.

SUB: Only 9.9% of comparison group members convey this or a greater level of misusing substances.


Comparison Group Findings AGG: Only 5.1% of comparison group

members convey this or a greater level of inappropriately aggressive behavior.



Chicago 17

Job-Relevant Correlates Possible job-relevant problems in multiple

domains: Emotional Control and Stress Tolerance Routine Task Performance Feedback Acceptance Social Competence and Teamwork Integrity Conscientiousness and Dependability Substance Use Impulse Control

May 12, 2016 65David M. Corey, Ph.D., ABPP May 12, 2016

May 12, 2016

Normal Personality TestingCalifornia Psychological Inventory-434 Community (general adult) norms Two highest: Do (Dominance)=70T Em (Empathy)=69T

Two lowest: So (Socialization)=49T Ac (Achievement via Compliance)=54T

May 12, 2016May 12, 2016 68David M. Corey, Ph.D., ABPP


Chicago 18

Normal Personality TestingCalifornia Psychological Inventory-434 Applicant Norms Two highest: Sa (Self-Acceptance)=68T Em (Empathy)=63T

Two lowest: So (Socialization)=28T Ac (Achievement via Compliance)=29T


Integration Model Applied3.Assess background findings

a. Up to age 19, rule-abiding, prosocialb. Age 19-25 Beginning in his freshman year in college,

while away from home for the first time, he was arrested for underage DUI on the college campus.


Background Findings (cont’d) A year later, he was arrested on a charge

of interfering with police officers during a political protest event at the college. (The record associated with this arrest was subsequently expunged.)


Background Findings (cont’d) He worked as a sales associate for two

retail clothing stores to help pay for college, but he was fired from both positions for “no-call” and “no-show” absences that, according to Mr. G’s statement to the background investigator, were a result of hangovers from late night drinking and partying.

At the age of 23, he dropped out of college without completing his degree.



Chicago 19

Background Findings (cont’d)c. Age 25-current At the age of 25, he stopped drinking

alcohol. Collateral sources confirm his abstinence

from alcohol use since age 25.


Background Findings (cont’d) He wrote in his Statement of Personal

History, “My drinking in years past has historically been a contributing factor in many of the poor decisions I made. Unfortunately, it took me more time than it should have to realize that. Once I understood what alcohol was doing to me, I never took another sip.”


Background Findings (cont’d) Returned to college at the age of 29 and

completed it within the past year. In addition to attending college and

working as a security officer, Mr. G volunteered at two police agencies, providing mentorship to at-risk teens in juvenile diversion programs.


Background Findings (cont’d) Background investigator’s summary:

“[Mr. G] appears to be an overall responsible, honest, reliable, hardworking and trustworthy person. He was responsive and cooperative during the background investigation. He made lots of mistakes and was irresponsible when he was young, but he appears to have learned from his mistakes and has worked hard to improve his life by volunteering to help other young people avoid the same mistakes he made.”



Chicago 20

Integration Model Applied4.Assess interview findings

a. Claimed participation in weekly AA meetings since he quit drinking six years ago.

b. Evidenced strong personal insight, self-reflection, and acceptance of personal accountability.

c. No discrepancies between self-report and background information were identified.


Interview Findings (continued)d. Described his motivation for becoming a

police officer as stemming from his commitment to sobriety: “Being of service to others helps me combat my narcissism, and narcissism will lead me to think I can drink again. I’m happiest when I’m thinking of others.”

e. No indications of naïveté were found in the interview.


Interview Findings (continued)f. Persuasively expressed remorse and

accountability for his past irresponsibility (e.g., displayed a strong, non-self-referential understanding of the ways that his past misconduct posed harm to others).


Integration Model Applied5. Identify relevant risk findings

a. Identify evidence-based inferences1) PCIR Job-Relevant Correlates (BXD, RC4,

JCP, SUB, AGG, DISC-r)2) CPI-434 (So, Ac)3) Problematic behavioral history (ages 19-25)



Chicago 21

Integration Model Applied5. Identify relevant risk findings

a. Identify evidence-based inferences1) PCIR Job-Relevant Correlates (BXD, RC4,

JCP, SUB, AGG, DISC-r)2) CPI-434 (So, Ac)3) Problematic behavioral history (ages 19-25)


Relevant Risk Findings (cont’d)b. Identify divergent findings of sufficient reliability

and quality to mitigate the evidence-based risk related findings1) Strong evidence of prosocial behavior in the past 8

years2) No evidence of contemporary negative behavior in

the past 8 years (i.e., no support for contemporary behavior consistent with problematic constructs)

3) Behavioral-internalizing scale elevations derive almost entirely from behavior 8 to 12 years prior (i.e., static risk)



Chicago 22

Integration Model Appliedc. Determine if surviving risk findings warrant

disqualification

May 12, 2016May 12, 2016 85David M. Corey, Ph.D., ABPP 86

Do Not Report Test Items Verbatim

Violates test security Violates copyright law If accessed by candidate, creates a

more sophisticated test taker and undermines reliability


Sample Reports Mr. N Mr. Z


updated - report writing using the pcir--handouts · of the mmpi-2-rf, the university of minnesota...

Documents