using technology assisted review for insurance defense
TRANSCRIPT
![Page 1: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/1.jpg)
877.557.4273catalystsecure.com
A primer on the law and technology for insurance defense professionals
WEBINAR
TAR for Smart People Gordon J. Calhoun, Esq.
John Tredennick, Esq.
Presenters
&
![Page 2: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/2.jpg)
We welcome your questions via the chat feature.
Our Speakers
Gordon serves as Chair of the firm's E-Discovery Practice and specializes in representing clients in a variety of
complex, large exposure litigated matters. He also manages the Insurance Regulatory and Reinsurance Division of
the firm’s Insurance Coverage and Bad Faith Practice Group.
Partner, Lewis Brisbois Bisgaard & Smith LLP
Gordon J. Calhoun
John founded Catalyst, which designs, hosts and services the world’s fastest and most powerful document
repositories for large-scale discovery and regulatory compliance. He was named by the American Lawyer as one of
the top six e-discovery trail blazers in its special issue on the “Top 50 Big Law Innovators” of the past 50 years.
Founder and CEO, Catalyst
John Tredennick
![Page 3: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/3.jpg)
![Page 4: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/4.jpg)
What is Technology Assisted Review?
1. A process through which humans work with a computer to teach it to identify relevant documents.
2. Ordering documents by relevance for more efficient review.
3. Stopping the review after you have reviewed a high percentage of documents. [Optional]
![Page 5: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/5.jpg)
Is This New?
![Page 6: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/6.jpg)
We Already Use It
![Page 7: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/7.jpg)
What is the Process?
1. Collect and process your files
![Page 8: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/8.jpg)
Shredding the Documents
12
3
![Page 9: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/9.jpg)
What is the Process?
1. Collect and process your files
2. Train the system (seeds)
3. Rank the documents
WHY RELEVANT?
![Page 10: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/10.jpg)
How Does it Work?
![Page 11: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/11.jpg)
How Does it Work?
Support Vector Machines Naïve Bayes K-Nearest Neighbor Geospatial Predictive Modeling Latent Semantic
"I may be less interested in the science behind the "black box” than in whether it produced responsive documents with reasonably high recall and high precision.“
Magistrate Judge Andrew Peck
![Page 12: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/12.jpg)
How Does it Work?
Support Vector Machines Naïve Bayes K-Nearest Neighbor Geospatial Predictive Modeling Latent Semantic
![Page 13: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/13.jpg)
What Goes on Under the Hood?
The computer builds a big, complex search!
What terms are most likely to be associated with good documents?
What terms are most likely to be associated with bad documents?
![Page 14: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/14.jpg)
What is the Process?
1. Collect and process your files
2. Train the system (seeds)
3. Rank the documents
4. Continue your review
5. Test as you go
6. Stop when finished
![Page 15: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/15.jpg)
A Circular Process
![Page 16: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/16.jpg)
What Are the Savings?
![Page 17: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/17.jpg)
What Are the Savings?
Number of documents in the review
![Page 18: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/18.jpg)
What Are the Savings?
Percentage of relevant documents found
![Page 19: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/19.jpg)
What Are the Savings?25
Linear Review
![Page 20: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/20.jpg)
What Are the Savings?
![Page 21: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/21.jpg)
Review 12% and get 80% recall
What Are the Savings?
![Page 22: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/22.jpg)
Review 24% and get 95% recall
What Are the Savings?
![Page 23: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/23.jpg)
When Can I Stop?
1. When you run out of relevant documents
2. Courts have approved 70% and higher
3. Prove recall by sampling the non-reviewed documents
4. Sample size and margin of error
![Page 24: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/24.jpg)
Case Study: Large Production Review
Collection: 2.1 million documents Initial richness: 1% Review team used CAL Review richness: 25 to 35%
![Page 25: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/25.jpg)
Result: 98% Recall
Review: Less than 10%
![Page 26: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/26.jpg)
Japanese client faces U.S. litigation
Case Study: IP Litigation
Over 3.6 million documents collected
Six relevant per 1,000 documents
Started with 10,000 already-tagged training documents
Used Insight Predict and its CAL protocol
Batch richness grew to 35%
Stopped review when relevant documents stopped coming
![Page 27: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/27.jpg)
Results
Found 80% relevant documents after just 20% review
Reviewed 500,000 of 3.6 million total
Saved 80% of the review costs
Saved 80% of the review time
![Page 28: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/28.jpg)
How Can I Use it?
Prioritized Review
Studies show that review teams get through documents more quickly when similar documents are reviewed together.
![Page 29: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/29.jpg)
How Can I Use it?
Outbound productions
Reduce review costs and time dramatically by establishing a cutoff point and stopping review.
Save up to 90% on review costs!
![Page 30: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/30.jpg)
How Can I Use it?
In-bound productions
In-bound productions allow your team to find hot documents for depositions and trial.
Find relevant documents in a fraction of the time!
![Page 31: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/31.jpg)
How Can I Use it?
Early case/document assessment
The SEC and DOJ actively use TAR to get a quick handle on document produced in their investigations.
Greg Buckles, eDJ Analyst
![Page 32: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/32.jpg)
How Can I Use it?
Focus team on multiple issues during review to save time and effort
Issue review
![Page 33: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/33.jpg)
How Can I Use it?
Non-English documents
TAR works effectively on any language so long as you tokenize.
裁判所はどこにありますか ?
Where is the courthouse?
![Page 34: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/34.jpg)
TAR 1.0: One-Time Training
1. Rolling uploads?
2. Subject matter experts?
3. Low richness collections?
4. Only for big cases?
Problems
![Page 35: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/35.jpg)
Review = training
TAR 2.0: Continuous Training
![Page 36: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/36.jpg)
CAL: Solving Real-World Problems
1. One time training
2. Rolling uploads
3. Subject matter experts
4. Low richness collections
5. Only for big cases
![Page 37: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/37.jpg)
Simple and flexible
CAL Training
1. Train with anything you want
2. Use as many or as few as you want
3. Keep searching and feeding throughout the
process
4. Makes use of every attorney decision on
documents
5. Contextual diversity will help find what you
don’t know
6. QC helps ensure consistent training
![Page 38: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/38.jpg)
![Page 39: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/39.jpg)
Continuous active learning (CAL) is more effective than one-time
training used in TAR 1.0
![Page 40: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/40.jpg)
Continuous Active Learning vs. One-Time Training
TAR 2.0 Savings
![Page 41: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/41.jpg)
![Page 42: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/42.jpg)
What Have the Courts Said?
![Page 43: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/43.jpg)
Go Ahead, Dive In!
“This judicial opinion now recognizes that computer-assisted review is an acceptable way to search for relevant ESI in appropriate cases.”
Da Silva Moore v. Publicis Groupe (S.D.N.Y. 2012)
![Page 44: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/44.jpg)
Go Ahead, Dive In!
“This judicial opinion now recognizes that computer-assisted review is an acceptable way to search for relevant ESI in appropriate cases.”
Da Silva Moore v. Publicis Groupe (S.D.N.Y. 2012)
![Page 45: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/45.jpg)
Approving Predictive Coding
Global Aerospace v. Landau Aviation (Virginia) April 23, 2012
![Page 46: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/46.jpg)
Unanimous Approval
Nat’l Day Laborer Org. Network v. U.S. Immigration & Customs Enforcement Agency, No. 10 Civ. 2488 (SAS), 2012 WL 2878130 (S.D.N.Y. July 13, 2012).
EORHB, Inc. v. HOA Holdings, LLC, No. 7409-VCL (Del. Ch. Oct. 15, 2012).
Global Aerospace, Inc. v. Landow Aviation, L.P., No. CL 61040 (Vir. Cir. Ct. Apr. 23, 2012).
In re Actos (Pioglitazone) Prods. Liab. Litig., MDL No. 6:11-MD-2299 (W.D. La. July 27, 2012).
2012
![Page 47: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/47.jpg)
Unanimous Approval
2013Gordon v. Kaleida Health, No. 08-CV-378S(F), 2013 WL 2250579 (W.D.N.Y. May 21, 2013).
In re Biomet M2a Magnum Hip Implant Prods. Liab. Litig., 2013 U.S. Dist. LEXIS 84440 (N.D. Ind. Apr. 18, 2013).
In re Biomet M2a Magnum Hip Implant Prods. Liab. Litig., 2013 U.S. Dist. LEXIS 172570 (N.D. Ind. Aug. 21, 2013).
EORHB, Inc. v. HOA Holdings, LLC, No. 7409-VCL, 2013 WL 1960621 (Del. Ch. May 6, 2013).
Gabriel Techs., Corp. v. Qualcomm, Inc., No. 08CV1992 AJB (MDD), 2013 WL 410103 (S.D. Cal. Feb. 1, 2013).
![Page 48: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/48.jpg)
Unanimous Approval
2014
Bridgestone Americas, Inc. v. Int. Bus. Machs. Corp., No. 3:13-1196 (M.D. Tenn. July 22, 2014).
Dynamo Holdings Ltd. P’ship v. Comm’r of Internal Revenue, Nos. 2685-11, 8393-12 (T.C. Sept. 17, 2014).
FDIC v. Bowden, No. CV413-245, 2014 WL 2548137 (S.D. Ga. June 6, 2014).
In re Bridgepoint Educ., Inc., No. 12cv1737 JM (JLB), 2014 WL 3867495 (S.D. Cal. Aug. 6, 2014).
Progressive Cas. Ins. Co. v. Delaney, No. 2:11-cv-00678-LRH-PAL, 2014 WL 2112927 (D. Nev. May 20, 2014).
![Page 49: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/49.jpg)
The Law
“The case law has developed to the point where it is black letter law that if a party wants to utilize TAR for document review, courts will permit it.”
![Page 50: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/50.jpg)
Discussion and Q&A
![Page 51: Using Technology Assisted Review for Insurance Defense](https://reader035.vdocument.in/reader035/viewer/2022062420/55b88827bb61ebdf558b4584/html5/thumbnails/51.jpg)
Let’s continue the discussion.
Thank you!
Gordon J. Calhoun
John Tredennick