updated - report writing using the pcir--handouts · of the mmpi-2-rf, the university of minnesota...
Post on 20-Feb-2019
225 Views
Preview:
TRANSCRIPT
2016 ABPP Annua l Confe rence & Workshops Chicago, IL
Thursday , May 12 , 2016
Evidence-Based Report Writing Using the MMPI-2-RF Police Candidate Interpretive Report
Presented by
David M. Corey, PhD, ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 1
Evidence-Based Report Writing Using the MMPI-2-RF PCIR
7th Annual ABPP Conference & WorkshopsChicago, Illinois
May 12, 2016
David M. Corey, Ph.D., ABPPCorey & Stewart, Consulting Psychologists
5285 SW Meadows Road, Suite 311Lake Oswego, Oregon 97035
503-620-8050dmc@corey-stewart.com
Copyright Notice
May 12, 2016 2
Copyright © 2016 David M. Corey
All Rights Reserved
David M. Corey, Ph.D., ABPP
Disclosure
I receive research funds from the publisher of the MMPI-2-RF, the University of Minnesota Press. As coauthor (with YossefBen-Porath) of the MMPI-2-RF Police Candidate Interpretive Report (PCIR), I receive royalties on the sale of the report.
3May 12, 2016 David M. Corey, Ph.D., ABPP
Slide Changes All additions and revisions to the
slides are noted with a starburst in the upper right corner
Revised handouts will be made available to attendees following the workshop
May 12, 2016 4David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 2
Focus of Workshop Optimizing efficiency, defensibility,
transparency, and relevance in preparation of written reports of preemployment psychological evaluations of law enforcement candidates
5May 12, 2016 David M. Corey, Ph.D., ABPP
Agenda Focused on 5 principle litigation contexts
in police and public 42 USC 21 § 1983 Grievance arbitrations Civil suits alleging discrimination (ADA and
Title VII) Civil service and U.S. Merit Systems
Protection Board appeals Writs of mandamus
6May 12, 2016 David M. Corey, Ph.D., ABPP
Features of the PPE Evaluation criteria are framed as the
absence of problems likely to impair performance and resilience, not on strengths or qualities (select-in)
Assessment of strengths or qualities is only for the purpose of assessing protective or mitigating factors
May 12, 2016 7David M. Corey, Ph.D., ABPP
Appelbaum, K. L. (2010)
“The quality of our reports is often the most tangible and visible measure of
our professionalism.”
May 12, 2016 8David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 3
Otto, R. K. (2009)
“The only time the expert is in complete control of the information that is and is not presented is when writing the report. As a result, it is when writing a report that one can clearly meet ethical obligations.”
May 12, 2016 9David M. Corey, Ph.D., ABPP
DeMier, R. L. (2016)
“The report stands alone as the enduring and tangible evidence of the
results sof the evaluation.”
May 12, 2016 10David M. Corey, Ph.D., ABPP
Report Has Multiple Audiences Hiring agency Candidate Second-opinion psychologist or other
expert reviewer Plaintiff’s counsel Civil service commission or appeal board Court of ultimate jurisdiction Licensing board/ethics committee
May 12, 2016 11David M. Corey, Ph.D., ABPP
Functions of the Report Reach a suitability determination Document it Explain the rationale for it Describe the evidentiary support for all
opinions Dissuade challenges Repel criticism Create a testimonial record
May 12, 2016 12David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 4
Grisso’s (2008) Principles1. Let the evaluation criteria drive,
guide and limit the content of the report
2. Report what is necessary. Do notreport what is not necessary. Not necessary if unrelated to the criteria Justify inclusion and exclusion on the
basis of relevance
May 12, 2016 14David M. Corey, Ph.D., ABPP
Grisso’s (2008) Principles3. Sequence and describe information
in a way that is useful and understandable to the reader
4. Separate facts from inferences and opinions. Inferences and opinions reflect what is
suggested by the facts Inferences and opinions should be clearly
linked to the facts supporting themMay 12, 2016 15David M. Corey, Ph.D., ABPP
Grisso’s (2008) Principles5. Explanations involve showing why
you think what you think Connect the dots Be transparent
6. Explain your rejection of other possible opinions and conclusions Explaining rejected ideas is an essential
component of transparency Essential for future testimony
May 12, 2016 16David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 5
IACP-PPSS Guideline 11 Minimum report content: “clear determination of the applicant’s
psychological suitability for employment based upon an analysis of all psychological assessment materials”
“expressly linked” to the “skills, behaviors, attributes, and other personal characteristics associated with effective and counterproductive job performance”
May 12, 2016 17David M. Corey, Ph.D., ABPP
IACP-PPSS Guideline 11 use clinical diagnoses or psychiatric labels
only if “relevant to the examiner’s conclusions, necessary for the hiring agency to make a hiring decision, and/or required by law”
“is focused on the individual applicant’s ability to safely and effective perform the essential functions of the position under consideration”
“clearly state” the period of report validity
May 12, 2016 18David M. Corey, Ph.D., ABPP
No Single Report Format
Format/content informed by: State mandate Requirements of the hiring agency
IACP-PPSS Guidelines 11.1: Any agency-specific restrictions or
other requirements relevant to the format or content should be determined in advance
May 12, 2016 19David M. Corey, Ph.D., ABPP
Standard (Minimum) Components Name of candidate Birthdate Date of evaluation Position Hiring agency Name, degree and
license number of psychologist
Methodology
Suitability determination or rating
Summary of findings supporting the determination or rating
Period of report validity
May 12, 2016 20David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 6
Elective Components Method of
identification Non-name
identifier (SSN or other unique code)
Description of screening dimensions
Screening dimension ratings
Cautionary caveats and declarations
Summary or details of self-reported history
May 12, 2016 21David M. Corey, Ph.D., ABPP
Cautionary Caveats: Dr. Mark Zelig ”This psychological assessment is to
evaluate this candidate sole for the above position. This is a summary and does not included all the information I learned. If the background check is discrepant with the history contained herein, please advise me and I will amend my conclusion accordingly.”
May 12, 2016 22David M. Corey, Ph.D., ABPP
Cautionary Caveats: Dr. Mark Zelig “This report is confidential. Its
distribution should be limited to only those who are directly involved in the hiring decision. This evaluation should be filed where access is limited to only the chief administrator of your department.”
May 12, 2016 23David M. Corey, Ph.D., ABPP
Cautionary Caveats: Dr. Mark Zelig “Compliance with the Americans with Disabilities
Act (ADA) and Genetic Information Nondiscrimination Act of 2009 (GINA): The candidate received a written conditional offer of employment that indicated he would be considered for employment upon passing the psychological evaluation. In compliance with GINA, the evaluator did not inquire nor record any information about psychopathology or other illness in biological or adopted relatives. Furthermore, the candidate was instructed to refrain from providing such information.”
May 12, 2016 24David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 7
Declarations: Dr. Mark Zelig “The steps taken to complete this
evaluation included:1. “Before the candidate commenced testing, he
executed a general informed consent. A supplemental disclosure advised him that all of our interactions would be electronically recorded. The candidate also completed the social history portion of the Biographical Data Information Form (PDIF).”
May 12, 2016 25David M. Corey, Ph.D., ABPP
Declarations: Dr. Mark Zelig2. “Once I received the above documents, this
candidate took the paper-and-pencil tests and other questionnaires at my officer under the supervision of a proctor in accordance with protocol. I reviewed these materials prior to conducted the interview.”
May 12, 2016 26David M. Corey, Ph.D., ABPP
Declarations: Dr. Mark Zelig3. “At the beginning of the interview, I reviewed the
informed consent documents described above. The candidate was further instructed that this entire process was voluntary and he could stop the process at any time or decline to answer a particular interview question if he felt it was inappropriate. Given the option of declining to answer a question, I told the candidate if he chose to answer a question, I expected an honest response. I also reminded him that our interactions were being electronically recorded. Once the candidate demonstrated an understanding of the conditions noted in the informed consents, I proceeded with the interview.”
May 12, 2016 27David M. Corey, Ph.D., ABPP
Restricted vs. UnrestrictedReporting of Personal History “There is no reason in law or policy why
an employer should be allowed access to detailed family or medical histories of its employees, or to the intricacies of its employees’ mental processes, except with the individual’s freely given consent to the particular disclosure or some other substantial justification.” Pettus v. Cole, 1996
May 12, 2016 28David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 8
Restricted vs. UnrestrictedReporting of History An employer is “not entitled to disclosure
of … the intimate and irrelevant details of [an individual’s] home life.” McGreal v. Ostrov, 2004
May 12, 2016 29David M. Corey, Ph.D., ABPP
GINA, Title II Prohibits the acquisition and use of
genetic information in hiring and employment
Genetic information includes: information about the manifestation of disease or
disorder in family members, including children, siblings, parents, half-siblings, nieces, nephews, grandparents, great grandparents, great-great grandparents, aunts, uncles, great aunts and uncles, first cousins, and children of first cousins
May 12, 2016 30David M. Corey, Ph.D., ABPP
Minimum Narrative Content Ratings of “unqualified,” “unsuitable,” or
“not recommended” require justification, transparently linking data/findings with conclusions, and conclusions to criteria
Ratings of “qualified,” “suitable,” or “recommended,” in the context of substantial negative test data or personal history, require reconciliation
May 12, 2016 31David M. Corey, Ph.D., ABPP
A Data Integration Model Spilberg & Corey (2014)
1. Assess protocol validity2. Assess substantive scale findings3. Assess background findings4. Assess interview findings5. Identify relevant risk findings6. Refine the assessment process
May 12, 2016 32David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 9
Step 1:Assess Protocol Validity Validity requires reliability The greatest threat to reliability of an
individual’s test results is: Lack of thoroughness and attention in
answering the items (i.e., unanswered items and response inconsistency)
Excessive underreporting
May 12, 2016 33David M. Corey, Ph.D., ABPP
Underreporting Some underreporting is expected
Absence of underreporting is uncommon and a possible red flag
Impact of underreporting is asymmetrical: Non-elevated substantive scale scores cannot be
interpreted as indicating the absence of problems assessed by those scales
Elevated substantive scales can be interpreted, but may underestimate problems
May 12, 2016 34David M. Corey, Ph.D., ABPP
Step 2:Assess Substantive Scales Identify and interpret relevant
findings, using general adult and police candidate norms Assess for significance against standard
norms (no adjustment to cutoff scores) Assess for comparison group significance Assess for convergent, divergent, and
complementary findings
May 12, 2016 35David M. Corey, Ph.D., ABPP
Interpreting Subclinical Scores Never infer meaning without adequate
evidence to support the inference Never infer psychopathology from a
subclinical score even if it is substantially deviant from the comparison group mean
May 12, 2016 36David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 10
Step 3:Assess Background Findings Evaluate personal history information
from all sources to determine: If any information meets agency standards
for disqualification How the information is convergent with,
divergent from, and complementary to relevant test findings and interpretations
May 12, 2016 37David M. Corey, Ph.D., ABPP
Step 4: Assess Interview Findings Assess how the interview findings and
clinical observations converge with, diverge from, or complement test and background findings
May 12, 2016 38David M. Corey, Ph.D., ABPP
Step 5: Reach a Risk Determinationa. Considering all risk-related findings from all sources,
what evidence-based inferences can be drawn from them? Eliminate those that do not map onto the selection
criteriab. What divergent findings mitigate these inferences?
Eliminate those inferences that are outweighed by divergent findings of sufficient relevance, validity and reliability
c. Are any surviving risk-related inferences of sufficient relevance and quality to warrant the candidate’s disqualification?
May 12, 2016 39David M. Corey, Ph.D., ABPP
Mitigating Negative Data
Wernimont & Campbell (1968) Past performance is relevant in predicting
future performance only if in similar contexts
In the absence of samples from a similar context, validated signs (tests) are superior to dissimilar samples
May 12, 2016 40David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 11
Step 6: Refine the Assessment Obtain feedback from the hiring
agency to determine the status of hired candidates and make adjustments to the prediction strategy, as necessary
May 12, 2016 41David M. Corey, Ph.D., ABPP
PCIR Features Standard MMPI-2-RF profile with
expanded Police Officer Candidate comparison group
May 12, 2016 42David M. Corey, Ph.D., ABPP
Expanded Police Candidate Comparison Group
North American RegionGender
TotalMale Female
Pacific 136 136 272
West 381 381 762
Midwest 131 131 262
South 224 224 448
Northeast US and Canada
165 165 330
Total 1,037 1,037 2,074
PCIR Features Transparency
All interpreted scores are explicitly identified
The source(s) for every interpretive statement are annotated
Intended to provide users with economic efficiency
Provides comparison group base rates for interpreted scores
May 12, 2016 44David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 12
PCIR Features Organizes job-relevant behavioral
correlates by California POST-like dimensions
Guidance for interpreting deviant underreporting scale scores Lists items with potentially verifiable
historical content to facilitate scrutiny
Item-level information, incorporating comparison group base rates
May 12, 2016 45David M. Corey, Ph.D., ABPP
User’s Guide for the PCIR Complete description of
the structure and rationale for the report
Comprehensive list of references
Summaries of the key empirical studies underlying the report
Detailed instructions for producing reports via Q-Local and Q-global
May 12, 2016 46David M. Corey, Ph.D., ABPP
Case Example:
Mr. C
Moderate Elevations Profile
May 12, 2016 47David M. Corey, Ph.D., ABPP
Case Example:
Ms. D
Underreporting Profile
May 12, 2016 48David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 13
Historically Verified Deception In addition to presenting herself as very well-
adjusted and having an implausibly virtuous character reported by only 1.1% of police officer candidates, some of Ms. D’s responses are contradicted by other facts revealed in the interview and background investigation. The candidate’s responses to psychological testing preclude reliance on them as evidence of the absence of problems they measure and predict.
May 12, 2016 49David M. Corey, Ph.D., ABPP
Demonstration Case:
Mr. G
May 12, 2016 50David M. Corey, Ph.D., ABPP
Case Material Background Agency-provided background
investigation report Self-reported (PsyQ, interview)
Testing MMPI-2-RF (PCIR) CPI-434 (PPSSR)
Clinical interview
May 12, 2016 51David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 14
Integration Model Applied1. Assess protocol validity
a. No indications of over- or under-reporting
b. L-r=42T Lowering L-r is associated primarily with
increases in behavioral-externalizing domain (BXD, RC4, JCP, DISC-r)
c. K-r=66T
May 12, 2016 53David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 15
Integration Model Applied2.Assess substantive scale findings
a. BXD=60Tb. RC4=68Tc. JCP=70Td. SUB=55Te. AGG=56Tf. DISC-r=69Tg. RC3=34T
May 12, 2016 58David M. Corey, Ph.D., ABPP
59May 12, 2016 David M. Corey, Ph.D., ABPP
Clinical Findings: RC4/JCP
May 12, 2016 60David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 16
Diagnostic Considerations
May 12, 2016 61David M. Corey, Ph.D., ABPP
Comparison Group Findings BXD: Only 4.4% of comparison group
members convey this or a greater level of behavioral difficulties.
DISC-r: Only 2.1% of comparison group members give evidence of this or a greater level of disconstraint.
RC4: Only 0.9% of comparison group members demonstrate this or a greater level of antisocial behavior.
May 12, 2016 62David M. Corey, Ph.D., ABPP
Comparison Group Findings JCP: Only 3.8% of comparison group
members give evidence of this or a larger number of juvenile conduct problems.
SUB: Only 9.9% of comparison group members convey this or a greater level of misusing substances.
May 12, 2016 63David M. Corey, Ph.D., ABPP
Comparison Group Findings AGG: Only 5.1% of comparison group
members convey this or a greater level of inappropriately aggressive behavior.
May 12, 2016 64David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 17
Job-Relevant Correlates Possible job-relevant problems in multiple
domains: Emotional Control and Stress Tolerance Routine Task Performance Feedback Acceptance Social Competence and Teamwork Integrity Conscientiousness and Dependability Substance Use Impulse Control
May 12, 2016 65David M. Corey, Ph.D., ABPP May 12, 2016
May 12, 2016
Normal Personality TestingCalifornia Psychological Inventory-434 Community (general adult) norms Two highest: Do (Dominance)=70T Em (Empathy)=69T
Two lowest: So (Socialization)=49T Ac (Achievement via Compliance)=54T
May 12, 2016May 12, 2016 68David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 18
Normal Personality TestingCalifornia Psychological Inventory-434 Applicant Norms Two highest: Sa (Self-Acceptance)=68T Em (Empathy)=63T
Two lowest: So (Socialization)=28T Ac (Achievement via Compliance)=29T
May 12, 2016May 12, 2016 69David M. Corey, Ph.D., ABPP
Integration Model Applied3.Assess background findings
a. Up to age 19, rule-abiding, prosocialb. Age 19-25 Beginning in his freshman year in college,
while away from home for the first time, he was arrested for underage DUI on the college campus.
May 12, 2016May 12, 2016 70David M. Corey, Ph.D., ABPP
Background Findings (cont’d) A year later, he was arrested on a charge
of interfering with police officers during a political protest event at the college. (The record associated with this arrest was subsequently expunged.)
May 12, 2016 71David M. Corey, Ph.D., ABPP
Background Findings (cont’d) He worked as a sales associate for two
retail clothing stores to help pay for college, but he was fired from both positions for “no-call” and “no-show” absences that, according to Mr. G’s statement to the background investigator, were a result of hangovers from late night drinking and partying.
At the age of 23, he dropped out of college without completing his degree.
May 12, 2016May 12, 2016 72David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 19
Background Findings (cont’d)c. Age 25-current At the age of 25, he stopped drinking
alcohol. Collateral sources confirm his abstinence
from alcohol use since age 25.
May 12, 2016May 12, 2016 73David M. Corey, Ph.D., ABPP
Background Findings (cont’d) He wrote in his Statement of Personal
History, “My drinking in years past has historically been a contributing factor in many of the poor decisions I made. Unfortunately, it took me more time than it should have to realize that. Once I understood what alcohol was doing to me, I never took another sip.”
May 12, 2016May 12, 2016 74David M. Corey, Ph.D., ABPP
Background Findings (cont’d) Returned to college at the age of 29 and
completed it within the past year. In addition to attending college and
working as a security officer, Mr. G volunteered at two police agencies, providing mentorship to at-risk teens in juvenile diversion programs.
May 12, 2016May 12, 2016 75David M. Corey, Ph.D., ABPP
Background Findings (cont’d) Background investigator’s summary:
“[Mr. G] appears to be an overall responsible, honest, reliable, hardworking and trustworthy person. He was responsive and cooperative during the background investigation. He made lots of mistakes and was irresponsible when he was young, but he appears to have learned from his mistakes and has worked hard to improve his life by volunteering to help other young people avoid the same mistakes he made.”
May 12, 2016May 12, 2016 76David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 20
Integration Model Applied4.Assess interview findings
a. Claimed participation in weekly AA meetings since he quit drinking six years ago.
b. Evidenced strong personal insight, self-reflection, and acceptance of personal accountability.
c. No discrepancies between self-report and background information were identified.
May 12, 2016May 12, 2016 77David M. Corey, Ph.D., ABPP
Interview Findings (continued)d. Described his motivation for becoming a
police officer as stemming from his commitment to sobriety: “Being of service to others helps me combat my narcissism, and narcissism will lead me to think I can drink again. I’m happiest when I’m thinking of others.”
e. No indications of naïveté were found in the interview.
May 12, 2016May 12, 2016 78David M. Corey, Ph.D., ABPP
Interview Findings (continued)f. Persuasively expressed remorse and
accountability for his past irresponsibility (e.g., displayed a strong, non-self-referential understanding of the ways that his past misconduct posed harm to others).
May 12, 2016May 12, 2016 79David M. Corey, Ph.D., ABPP
Integration Model Applied5. Identify relevant risk findings
a. Identify evidence-based inferences1) PCIR Job-Relevant Correlates (BXD, RC4,
JCP, SUB, AGG, DISC-r)2) CPI-434 (So, Ac)3) Problematic behavioral history (ages 19-25)
May 12, 2016May 12, 2016 80David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 21
Integration Model Applied5. Identify relevant risk findings
a. Identify evidence-based inferences1) PCIR Job-Relevant Correlates (BXD, RC4,
JCP, SUB, AGG, DISC-r)2) CPI-434 (So, Ac)3) Problematic behavioral history (ages 19-25)
May 12, 2016May 12, 2016 83David M. Corey, Ph.D., ABPP
Relevant Risk Findings (cont’d)b. Identify divergent findings of sufficient reliability
and quality to mitigate the evidence-based risk related findings1) Strong evidence of prosocial behavior in the past 8
years2) No evidence of contemporary negative behavior in
the past 8 years (i.e., no support for contemporary behavior consistent with problematic constructs)
3) Behavioral-internalizing scale elevations derive almost entirely from behavior 8 to 12 years prior (i.e., static risk)
May 12, 2016May 12, 2016 84David M. Corey, Ph.D., ABPP
David M. Corey, Ph.D., ABPP May 12, 2016
Chicago 22
Integration Model Appliedc. Determine if surviving risk findings warrant
disqualification
May 12, 2016May 12, 2016 85David M. Corey, Ph.D., ABPP 86
Do Not Report Test Items Verbatim
Violates test security Violates copyright law If accessed by candidate, creates a
more sophisticated test taker and undermines reliability
May 12, 2016 87David M. Corey, Ph.D., ABPP
Sample Reports Mr. N Mr. Z
May 12, 2016 88David M. Corey, Ph.D., ABPP
top related