st. louis user group - amazon web services · st. louis user group relativity bag of tricks: ......
TRANSCRIPT
![Page 1: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/1.jpg)
© kCura LLC. All rights reserved.
St. Louis User Group
Relativity Bag of Tricks: Solving Common Problems in Unique Ways
![Page 2: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/2.jpg)
© kCura LLC. All rights reserved.
• Introductions
• Tricks from the Relativity bag
– User experiences and tips
– Let’s have a conversation
– Ask questions
– Contribute your stories
Agenda
![Page 3: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/3.jpg)
© kCura LLC. All rights reserved.
• QC Protocols for contract reviews (and in house too)
• Effective communication between the case team and the review team
• Hints to manage your project in Relativity
• Best practices to keep your stakeholders informed with reporting tools
• Quick wins when using Relativity Assisted Review
Bag of Tricks
![Page 4: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/4.jpg)
© kCura LLC. All rights reserved.
• Create a view for Privilege QC
• Encourage reviewers to do their own QC
• Use a second level review layout
• Take advantage of built in Sampling
QC workflows
![Page 5: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/5.jpg)
© kCura LLC. All rights reserved.
View: Privilege QC
![Page 6: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/6.jpg)
![Page 7: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/7.jpg)
![Page 8: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/8.jpg)
![Page 9: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/9.jpg)
© kCura LLC. All rights reserved.
View: Self QC
![Page 10: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/10.jpg)
![Page 11: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/11.jpg)
![Page 12: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/12.jpg)
![Page 13: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/13.jpg)
![Page 14: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/14.jpg)
![Page 15: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/15.jpg)
© kCura LLC. All rights reserved.
QC Layout
![Page 16: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/16.jpg)
© kCura LLC. All rights reserved.
QC: Create a Good Layout
![Page 17: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/17.jpg)
© kCura LLC. All rights reserved.
QC: Create a Good Report to Track Results
![Page 18: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/18.jpg)
© kCura LLC. All rights reserved.
Sampling for Success
![Page 19: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/19.jpg)
© kCura LLC. All rights reserved.
![Page 20: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/20.jpg)
© kCura LLC. All rights reserved.
QC: Use Sampling to Your Benefit
1 2
![Page 21: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/21.jpg)
© kCura LLC. All rights reserved.
QC: Save
the Results
![Page 22: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/22.jpg)
© kCura LLC. All rights reserved.
Effective communication between the case team and the review team
• Processing detail (summary)-
– De-duplication
– Exceptions
– Other issues- Missing files, OCR
failures
• QC searches-
– Responsive Set
– Family Conflicts
– Needs Redaction
– Tech Assist
Layouts-Document MetadataHas hidden data and exceptions
Reports-Batch progress- updateRAR-updateSTRs (key terms and privilege)- iterative
Security views-Layout mockup
Preview for case team
![Page 23: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/23.jpg)
© kCura LLC. All rights reserved.
• Establish the big picture
• Prepare for new workspace and data
• How do you handle new data
• Tips to conduct the review
• Readying a solid production
Hints to manage your Relativity project
![Page 24: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/24.jpg)
© kCura LLC. All rights reserved.
Keeping Stakeholders Informed
• Where in the process are you?
• Who are the stakeholders?
• What do they want to know?
![Page 25: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/25.jpg)
© kCura LLC. All rights reserved.
Processing
• Client: wants to know what they gave you and how much it is going to cost
– report showing data by custodian
– total size/cost
• Legal Team: needs data to support their discovery efforts
– Volume at start
– Volume deduped
– Volume left out from culling parameters
![Page 26: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/26.jpg)
© kCura LLC. All rights reserved.
Pre-Review
• Partner: wants to know how much there is to review and how long it will take.
– Report on average review times for that much data
– Report data to show whether, in this particular case, analytics will help save review
time/money
• Associate: wants to know best place to start review
– Reports on search terms, concepts, domains
![Page 27: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/27.jpg)
© kCura LLC. All rights reserved.
Review
• Dashboards to show progress of documents set to review will keep everyone up to date to the
minute. Can help forecast how much longer it will take.
• BUT…make sure to have a dashboard that shows status of QC as well, so there are no
surprises.
![Page 28: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/28.jpg)
© kCura LLC. All rights reserved.
Production
• QC dashboards from review will help alert Lit Support that a production is imminent, even if the
legal team does not communicate well!
![Page 29: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/29.jpg)
© kCura LLC. All rights reserved.
Post-Production
• Fact Manager can help the team keep up to date on what useful (or not useful) information has
been located in the documents.
![Page 30: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/30.jpg)
© kCura LLC. All rights reserved.
• Time to PLAN
• Train the Reviewers
• Rounds of judgmental sampling
• End of the Game – dealing with the NRs
Quick wins using Relativity Assisted Review
![Page 31: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/31.jpg)
© kCura LLC. All rights reserved.
RAR Case Checklist Meets Criteria
Minimum of 50,000 records with text
Concept-rich files, e.g. emails, word documents, limited excel files, or graphic
images
Issue or privilege coding in separate field not part of Assisted Review workflow
Is this previously reviewed data? If so, how were families reviewed?
Expected timelines given
Expected process given
Number of reviewers
Level of reviewers
Level of accuracy
PLANNING - Assisted Review Checklist*
- Does this dataset meet the criteria?
See https://help.kcura.com/9.4/Content/Recipes/Checklists/RAR_checklist.htm
![Page 32: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/32.jpg)
© kCura LLC. All rights reserved.
• Selecting good example documents
– Quality of the example is independent of responsiveness – it’s a second dimension
– Some documents may be highly responsive, yet undesirable as example documents for a
computer-assisted review project
– Four Corners Doctrine
• Applying the Use as Example field
• Using the Excerpt Text feature
Train the Reviewers
![Page 33: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/33.jpg)
© kCura LLC. All rights reserved.
• Hot Documents
• Important Custodian
• Keyword Search Hits
• Previously Coded Documents
• Clustering
PROJECT – Judgmental Sampling Options
![Page 34: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/34.jpg)
© kCura LLC. All rights reserved.
Options
• Save until last (still reviewing all)
• One last sample
• Keyword searches for hot documents
END GAME: Uncategorized/Non Responsive Documents
![Page 35: St. Louis User Group - Amazon Web Services · St. Louis User Group Relativity Bag of Tricks: ... – Report data to show ... • Dashboards to show progress of documents set to review](https://reader033.vdocument.in/reader033/viewer/2022051803/5b03fd117f8b9aba168ca9a5/html5/thumbnails/35.jpg)
© kCura LLC. All rights reserved.
Thanks for attending!