evaluation and policy decisions in alternative dispute...
TRANSCRIPT
Evaluation and Policy Decisions in Alternative Dispute Resolution:
Partnerships Across the State
J A M I E L . W A L T E R , P h . D .D i r e c t o r o f C o u r t O p e r a t i o n s , A d m i n i s t r a t i v e O f f i c e o f t h e C o u r t s
J O N A T H A N S . R O S E N T H A L , E s q .D i r e c t o r o f t h e M e d i a t i o n a n d C o n f l i c t R e s o l u t i o n O f f i c e ( M A C R O ) , A d m i n i s t r a t i v e O f f i c e o f t h e C o u r t s
J u l y 1 2 , 2 0 1 6
T e x t M D C O U R T S t o 2 2 3 3 3
ADR
What words come to mind when you think of alternative dispute resolution?
Text reply to….
Research
What words come to mind when you think of research?
Text reply to….
Why Research?
1. Significant investment in ADR in Maryland’s court system.
2. Lack of comprehensive, state-level studies that examine a variety of ADR practices and the full costs and benefits
3. Scarcity of empirical studies which directly connect different outcomes to different ADR strategies
Study of MD Court ADR
ADR Landscape
Emerging ADR Practices in Maryland
Efficiency and Effectiveness
Impact, Costs, and Benefits
http://mdcourts.gov/courtoperations/adrprojects.html
Research Support
Grant from the State Justice Institute
Support from Court Administration
Court Professionals and Judge served on advisory group
How many court affiliated ADR programs exist in Maryland?
Court of Special Appeals
Circuit Court
General Civil (14)
Domestic (24 child access, 20 child welfare, 18 marital property)
Juvenile (8)
District Court
Civil (16 day of trial, 7 pre-day of trial)
Criminal (12)
Orphans’ Court (2)
Circuit Court Domestic ADR
Child Access Mediation (24)All jurisdictions
Marital Property Mediation (18)All except: Baltimore, Calvert,
Harford, St. Mary’s, Wicomico Counties,
Baltimore City
Settlement Conferencing (16)Anne Arundel, Baltimore, Calvert,
Caroline, Carroll, Cecil, Garrett, Howard, Kent,
Montgomery, Prince George’s, Somerset,
Washington, Wicomico, Worcester Counties,
Baltimore City 2013
Circuit Court Facilitation and Child Welfare
Mediation Programs
2013
Facilitation (6)Anne Arundel, Baltimore,
Carroll, Montgomery, Somerset, Wicomico Counties
Child Welfare Mediation
CINA only (1): Allegany CountyTPR only (3): Harford, Howard,
Worcester Counties
Remaining 16 offer both processes
Circuit Court Civil Non-Domestic Mediation &
Settlement Conference Programs
2013
Mediation (14)Allegany, Anne Arundel,
Baltimore, Calvert, Charles, Frederick, Howard, Kent,
Montgomery, Prince George’s,Queen Anne’s, Talbot,
Worcester, & Baltimore City
Settlement Conferencing (14)Anne Arundel, Baltimore, Calvert,
Caroline, Carroll, Cecil,Charles, Howard,
Kent, Montgomery,Prince George’s, Queen Anne’s,
Worcester Counties, & Baltimore City
2013
Community Conferencing (8)
Anne ArundelBaltimore City
Baltimore CountyDorchester
Montgomery Prince George’s
Queen Anne’sTalbot
Circuit Court Delinquency Programs
Orphans’ Court
2016
Mediation (3)
Baltimore and Prince George’s
Counties
2012
Mediation (15)Anne Arundel, Baltimore, Calvert,
Carroll, Charles, Frederick, Harford, Howard, Montgomery, Prince George’s, St. Mary’s,
Somerset, Washington, Wicomico Counties, Baltimore City
Settlement Conference (11)Anne Arundel, Baltimore, Carroll,
Frederick, Harford, Howard, Montgomery, Prince George’s, Somerset, Wicomico Counties,
Baltimore City
Peace Order Mediation (5)Anne Arundel, Calvert, Charles, Montgomery, St. Mary’s Counties
District Court Volunteer Day-of-Trial
Programs
District Court Volunteer Pre-trial Mediation
Programs
2013
Mediation (7)Anne Arundel, Calvert,
Harford, Montgomery, St. Mary’s, Wicomico Counties, Baltimore
City
2013
Mediation (12)Anne Arundel, Calvert,
Carroll, Cecil, Kent, Harford, Montgomery,
Queen Anne’s, Somerset, Washington, Worcester
Counties, Baltimore City
District Court Criminal Mediation Programs
c
Emerging ADR Practices in Maryland
Community Conferencing
Program description and recidivism
Collaborative Law
National context, plus interviews, and surveys of court personnel and CL attorneys in Maryland
Barriers
Empirical Research Scope
Cost Benefit/ Comparison
IMPACT OF ADRCriminal
Day of TrialCivil
Efficiency & Effectiveness
WHAT WORKS IN ADR?Day of Trial
Family
Short term Short termLong term Attitudes Enforcement
Actions
Long termParticipant
report
Preparing for the Research Project:“Who ya Gonna Call?”
Early design and planning stage should include:
o Judges
o Court administrators
o ADR Practitioners
o ADR Service Provider Partners
o Other Court Personnel
o State’s Attorney’s Offices
o Other Justice Partners
Maryland ADR Research Jurisdictions
Landscape
Collaborative Law
Maryland ADR Research Jurisdictions
Landscape
Collaborative Law
Community Conferencing
Maryland ADR Research Jurisdictions
Landscape
Collaborative Law
Community Conferencing
District Court Day of Trial Comp
Maryland ADR Research Jurisdictions
Landscape
Collaborative Law
Community Conferencing
District Court Day of Trial Comp
District Court Strategies
Maryland ADR Research Jurisdictions
Landscape
Collaborative Law
Community Conferencing
District Court Day of Trial Comp
District Court Strategies
Family Strategies
Maryland ADR Research Jurisdictions
Landscape
Collaborative Law
Community Conferencing
District Court Day of Trial Comp
District Court Strategies
Family Strategies
Criminal
Maryland ADR Research Jurisdictions
Landscape
Collaborative Law
Community Conferencing
District Court Day of Trial Comp
District Court Strategies
Family Strategies
Criminal
Circuit Court Civil-General
H O W W E G A T H E R E D A N D M A D E S E N S E O F T H E D A T A
Methodology
Methodology
Compare treatment cases to similar control cases when possible
Use regression analysis to isolate impact of ADR
Hold constant
complexity of case, familiarity with ADR, attitude, animosity, demographics, police involvement, etc.
Data Collection Overview
Surveys with participants before and after ADR or trial:
Satisfaction
Sense of closure or justice
Change in attitude
Data Collection Overview
Observation:
Data Collection Overview
Observation:
Mediator’s strategies
Participant’s responses
Time in joint or caucus sessions
Follow-up Surveys
Follow-up survey 3 – 6 months later:
Final status of case
Further cost or impact to participant
Changes in attitude or ability to communicate with other party
Current interactions with other party
Further violence or police intervention
Long-term archival data
Archival data review six to twelve months later:
Further court action
appeal, re-litigation, modification, pursuit of judgments, etc.
Family Court Cases
WHAT WORKS?
Family CourtFactors: Mediator Strategies
Reflecting Strategies:• Reflecting
emotions & interests
• Clarifying what topics participants want to work on
Directing Strategies:• Introducing &
enforcing guidelines
• Explaining one participant to another
• Advocating for one participant’s ideas
Eliciting Strategies:• Asking
participants for their ideas of solutions
• Summarizing solutions
• Asking how solutions might work for them
Telling Strategies:• Sharing opinions• Offering solutions• Advocating for one
participant‘s ideas• Assessing legal
positions and options.
Reflect Direct Elicit Tell
Findings: Mediator Strategies,
Participants are MORE likely to:• Say the other
person listened & understood
• Become more able to work together
• Develop more personalized agr.
Participants are LESS likely to:• Dismiss the other’s
perspective• Reach agreement
Participants are LESS likely to:
• Report the mediator listened to them and respected them
Participants are MORE likely to:• Reach an
agreement• Say the other
person listened & understood
• Become clearer about their desires
• Say the underlying issues came out
• Become more able to work together
This strategy was not statistically significant in any positive or negative outcomes.
Reflect Direct Elicit Tell
When Mediator Strategies are Combined
Reflecting and Eliciting strategies combined:
Positive shift in participants’ reported ability to work together
Say that the other person listened and understands them better
Indicate that the underlying issues came out
Reach a personalized agreement
Reflect Elicit
Directing Strategies,
Participants are MORE likely to:• Say the other
person listened & understood
• Become more able to work together
• Develop more personalized agr.
Participants are LESS likely to:• Dismiss the other’s
perspective• Reach agreement
ReflectDirect
The greater percentage of mediator Directingstrategies used…
The more likely participants are to file an adversarial motion; and
The more adversarial motions they are likely to file
Family/Custody Cases: Caucusing
More time in caucus:
Mediator respected and listened to them
More hopeless about the situation after the mediation than they did before the session
Less likely to believe they can work with the other parent than they did before the session
District Court Day of Trial
Building Buy-in&
Building Better Processes
Creation of Control Groups
Treatment Cases
Pre-test
Observation & Practitioner Survey
Post-test
Follow-up test
Archival Data Review
Control Cases
Pre-test
- n/a -
Post-test
Follow-up test
Archival Data Review
Comparison of Control and Treatment Groups
CharacteristicBaltimore City
Mediated
Baltimore City
Control
Montgomery
Mediated
Montgomery
Control
TOTAL # Cases
(197)51 63 45 38
Contract 56.9% 41.3% 88.9% 100%
Breach of Lease 2.0% 12.7% 0.0% 0.0%
Detinue 2.0% 4.8% 0.0% 0.0%
Forcible Detainer 13.7% 7.9% 0.0% 0.0%
Replevin 3.9% 0% 0.0% 0.0%
Tenant Holding Over 11.8% 22.2% 0.0% 0.0%
Tort 2.0% 0% 11.1% 0.0%
Wrongful Detainer 7.9% 11.1% 0.0% 0.0%
Both sides
represented0.0% 0.0% 2.2% 0.0%
One side represented 7.8% 6.4% 35.6% 36.8%
Pro-se 92.2% 93.7% 62.2% 63.2%
District Court Day of Trial
OUTCOMES
IMPACT OF ADR
Short and Long Term
Impact of DOT ADRFindings: Short Term Outcomes
Participants in ADR (compared to litigation) are more likely to report that:
o They could express themselves, their thoughts, and their concerns.
o All of the underlying issues came out.
o The issues were resolved.
o The issues were completely resolved rather than partially resolved.
o They acknowledged responsibility during the process.
Holds true regardless of agreement in mediation
Impact of ADR Short Term Shifts in Attitude
We measured shifts in attitude from before to after
Compared the shifts in treatment and control groups
Found that those who go to ADR are more likely to:
1. To have an increase in their rating of their level of responsibility for the situation from before to after the intervention.
2.To disagree more with the statement “the other people need to learn they are wrong” from before to after the process.
Impact of ADR Short Term Findings
Finally, participants who developed a negotiated agreement in ADR were more likely to be satisfied with the judicial system than others, while participants who reached negotiated agreement on their own (without ADR) were not more likely to be satisfied with the judicial system than those without negotiated agreements.
This seems to imply that the process of reaching an agreement in ADR is the factor that led to higher satisfaction, rather than just the process of having negotiated a settlement.
56
Impact of ADR: Long Term Participant Attitude
People who participated in mediation are more likely than people who did not participate in mediation to report:
An increase in concern for everyone’s perspective and for the relationship
Issues are resolved and the outcome is working
Satisfaction with the Judiciary
57
Impact of DOT ADR: Long Term Costs to Court
58
The predicted probability of a case returning to court for an “enforcement action”
Is 45% if a case received a verdict
Is 21% if a case has a mediated agreement
Is not affected positively or negatively by an agreement the participants reach on their own.
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Verdict ADR Agreement
Probability of Returning to Court
Probability of Returning to Court
District Court Day of Trial
OUTCOMES59
Which Strategies Work in ADR?
Short and Long Term
Findings: Mediator Strategies
Offering Solutions:• Mediator offering
an opinion• Mediator offering
solutions• Legal assessment
Direct
Eliciting Strategies:• Asking
participants for their ideas of solutions
• Summarizing solutions
• Asking how solutions might work for them
Elicit
Reflecting Strategies:• Reflecting
emotions & interests
Reflect
60
Which Strategies Work in ADR: Short Term
Reflecting
Positively associated with participants reporting that the other person took responsibility and apologized;
Positively associated with an increase in a sense of self-efficacy (ability to talk and make a difference);
Positively associated with an increase in the sense that the court cares from before to after the mediation.
61
Which Strategies Work in ADR: Short Term
Eliciting
Positively associated with participants reporting that they listened and understood each other in the mediation, and jointly controlled the outcome;
Participants report that the other person took responsibility and apologized;
Negatively associated with participants report that the mediator controlled the outcome, pressured them into solutions, and prevented issues from coming out.
Eliciting Participant Solution was the only mediator strategy that had an impact on reaching an agreement, and the impact is a positive one.
62
Which Strategies Work in ADR: Short Term
Racial Match of mediator and participant
Positively associated with participants report that they listened and understood each other in the mediation and jointly controlled the outcome;
Positively associated with an increase in a sense of self-efficacy (ability to talk and make a difference);
Positively associated with an increase in the sense that the court cares from before to after the mediation.
63
Which Strategies Work in ADR: Short Term
CaucusMore likely the participants are to report that the ADR
practitioner controlled the outcome, pressured them into solutions, and prevented issues from coming out;
Negatively associated with participants reporting that they were satisfied with the process and outcome, and that the issues were resolved with a fair and implementable outcome.
Positively associated with an increase in a sense of powerlessness, an increase in the belief that conflict is negative, and an increase in the desire to better understand the other participant.
64
Which Strategies Work in ADR: Long Term
Mediator Offering Opinions and Solutions
Negatively associated with participants’ report that the outcome was working, they were satisfied with the outcome, and they would recommend mediation and
Negatively associated with participants’ report that they changed their approach to conflict.
65
Which Strategies Work in ADR: Long Term
Mediation Experience/Frequency
The number of cases the ADR practitioner had conducted in the previous twelve months was negatively associated with participants returning to court for an enforcement action in the subsequent twelve months
66
Which Strategies Work in ADR: Long Term
CaucusThe long term analysis finds that the more time
participants spent in caucus was associated witha decrease in:
participants’ consideration of the other person;
self-efficacy; and,
a sense that the court cares about resolving conflict from before the mediation to several months later.
And an increase in:
participants’ returning to court within twelve months for enforcement action.
67
District Court Criminal68
IMPACT OF ADR
Short Term
Criminal
Goal
Identify the impact of mediation on judicial resources, law enforcement resources, individual experiences, and conflict resolution
Short and Long Term Impact
Judicial Action:
Jury Trial Prayed, Guilty, Not Guilty, Probation Before Judgement (any court action); compared to Nol Prosse or Stet
Jury Trial Prayed
Supervised Probation or Jail
Resulted in Record
Long Term results show more criminal charges
Circuit Court Civil-General71
IMPACT OF ADR
Civil-General
Compare cases that went to ADR to those that did not
Complex data collection
Goals differ, barriers to ADR are important considerations
Those who settle at mediation are more likely to rate having their voice heard or telling their story as important goals
When Case Concluded
0
2
4
6
8
10
12
14
16
18
20
Within 90Days Filing
Before ADRDeadline
At MediationSession
Within 30Days
Mediation
BeforeSettlement
Conf
Between SC &Trial
After TrialBegan
ReceivedVerdict
Timing – Attend ADR vs. Did Not Attend ADR
Attended ADR Did Not Attend ADR
What Works with Evidenced Based Research and Policy
Policy questions become clear objectives
Involve policymakers and stakeholders with researchers
Be aware of political realities
Keep communication open and appropriate
Foster strong relationships
What’s Next?
77
Where do we go from here?
What questions are left unanswered?
What do YOU want to know?
What’s Next?
Where Do We Go from Here?
Symposium Maryland Judiciary
National Experts
In State Experts
Additional Programs for In State Practitioners and ADR Organizations
Policy Changes
Programmatic Changes Qualifications
Quality assurance
Continuing education
What’s Next?
What Questions Are Left Unanswered?
What’s Next?
What Do YOUWant To Know About Your Programs?
Contact Information
J A M I E W A L T E R , P H . D .
D i r e c t o r o f C o u r t O p e r a t i o n s ,
A d m i n i s t r a t i v e O f f i c e o f t h e C o u r t s
j a m i e . w a l t e r @ m d c o u r t s . g o v
4 1 0 - 2 6 0 - 1 7 2 5
J O N A T H A N S . R O S E N T H A L , E S Q .
D i r e c t o r , M e d i a t i o n a n d C o n f l i c t R e s o l u t i o n O f f i c e ( M A C R O ) , A d m i n i s t r a t i v e O f f i c e o f t h e C o u r t s
J o n a t h a n . r o s e n t h a l @ m d c o u r t s . g o v
4 1 0 - 2 6 0 - 3 5 4 8