May 2013
Mass Report Generation
Managing High Volume Reporting
Overview of Volume Output Requirements Common Problems Report Resolution Demonstration
Agenda
In 2012 the WorleyParsons Environmental Data Management Team processed:· 6,500 lab EDDs,
· From 4 primary laboratories, accounting for
· 28,000 samples, and
· 1,436,000 results,
Into 3 databases containing 3,000 facilities across Canada
Overview of Volume
Common data table and graph outputs are displayed:· Historically, year over
year
· With guidelines (action levels) comparisons applied
· Grouped by analytes
· From scratch on a regular basis (i.e. quarterly)
Output Requirements
Table 7 of 8, Page 105 of 132 for 1 of 11 sites
Graphing sheet 3 of 800
Staffing:· It takes ~6 months for staff to get up to speed
· After 2.5 years they are burnt out
Timing:· We have several programs that require
reporting to be in the clients hand within 24 hours of sample drop off
Laboratory issues:· Inaccurate or incomplete reporting
· Reporting what we want, not what we need
· Labs sign reports, not data
System Issues:· An x86 32 bit machine has a 2 GB memory
limit
· Large data sets using a modified EQuIS report are time consuming, taking over 45 minutes to run
· Cross tab display uses 5mb/second
· Exporting to Excel 20mb/sec
Common Problems
Does anyone have an analyte group that looks like this?
One analyte, multiple methods.
In EQuIS 5.6 EarthSoft introduced a new Beta analytical report
Allows for a new approach to reporting, using dynamic SQL
Combined with a modified lab reporting methodology WorleyParsons has been able to reduce reporting times significantly
Core features include:· Significant time and memory reduction
− 47,000 record report (pulled in 23 seconds)
− cross-tabbed to 5,785 (37 seconds)
− with guidelines applied (41 seconds)
− Similar data pulled in standard EQuIS report without guideline comparison and no-cross tab (90 seconds)
· Simplified analytical grouping
· Auto Cross-Tab functionality without significant memory loss
· Direct Guideline (Action Level) comparison
· Location Specific Guideline Comparison
· Direct Web GIS Interfacing
Report Resolution
Demonstration (Standard Analytical Report)
Pick Selection
Analytical Report Applied to Web GIS
Analytical Report Pick Interface
Web GIS Report Results
Thank you.