may 2013 mass report generation managing high volume reporting
TRANSCRIPT
May 2013
Mass Report Generation
Managing High Volume Reporting
Overview of Volume Output Requirements Common Problems Report Resolution Demonstration
Agenda
In 2012 the WorleyParsons Environmental Data Management Team processed:· 6,500 lab EDDs,
· From 4 primary laboratories, accounting for
· 28,000 samples, and
· 1,436,000 results,
Into 3 databases containing 3,000 facilities across Canada
Overview of Volume
Common data table and graph outputs are displayed:· Historically, year over
year
· With guidelines (action levels) comparisons applied
· Grouped by analytes
· From scratch on a regular basis (i.e. quarterly)
Output Requirements
Table 7 of 8, Page 105 of 132 for 1 of 11 sites
Graphing sheet 3 of 800
Staffing:· It takes ~6 months for staff to get up to speed
· After 2.5 years they are burnt out
Timing:· We have several programs that require
reporting to be in the clients hand within 24 hours of sample drop off
Laboratory issues:· Inaccurate or incomplete reporting
· Reporting what we want, not what we need
· Labs sign reports, not data
System Issues:· An x86 32 bit machine has a 2 GB memory
limit
· Large data sets using a modified EQuIS report are time consuming, taking over 45 minutes to run
· Cross tab display uses 5mb/second
· Exporting to Excel 20mb/sec
Common Problems
Does anyone have an analyte group that looks like this?
One analyte, multiple methods.
In EQuIS 5.6 EarthSoft introduced a new Beta analytical report
Allows for a new approach to reporting, using dynamic SQL
Combined with a modified lab reporting methodology WorleyParsons has been able to reduce reporting times significantly
Core features include:· Significant time and memory reduction
− 47,000 record report (pulled in 23 seconds)
− cross-tabbed to 5,785 (37 seconds)
− with guidelines applied (41 seconds)
− Similar data pulled in standard EQuIS report without guideline comparison and no-cross tab (90 seconds)
· Simplified analytical grouping
· Auto Cross-Tab functionality without significant memory loss
· Direct Guideline (Action Level) comparison
· Location Specific Guideline Comparison
· Direct Web GIS Interfacing
Report Resolution
Demonstration (Standard Analytical Report)
Pick Selection
Analytical Report Applied to Web GIS
Analytical Report Pick Interface
Web GIS Report Results
Thank you.