report card scoring
DESCRIPTION
Report Card Scoring. Several options under consideration for scoring and aggregating data. Why turn data into indices?. Integrate multiple measures into thematic scores Evaluate overall trends and compare locations Simplify communication with decision makers and the public. Four Main Steps. - PowerPoint PPT PresentationTRANSCRIPT
Report Card Scoring
Several options under consideration for scoring and aggregating data
Why turn data into indices?
• Integrate multiple measures into thematic scores
• Evaluate overall trends and compare locations
• Simplify communication with decision makers and the public
Four Main Steps
• Select and group indicators
• Transform measured data into unit-less scores
• Aggregate scores from multiple indicators into a summary index
• Define breakpoints to bin index scores into descriptors of condition (grades or colors)
Central Coast RegionPreliminary Indices
1. Human Health WQ Index2. Aquatic Life WQ Index3. Toxicity Index4. Bioassessment Index5. Biostimulatory Risk Index6. Habitat Index7. Riparian Index
Indicators within Indices
Example from the Central Coast Region1. Human Health Water Quality Index
Fecal ColiformE. coliPathogensNitrate
Toxic chemicals
Indicators within Indices
Example from the Central Coast Region2. Aquatic Life Water Quality Index
AmmoniaNitrateOrtho-phosphateDO departure
Toxic chemicals
TurbidityTotal dissolve solidsTotal susp. sedimentpH departure
Scoring Approach 1
Scoring into categories using multiple thresholds
If the mean > 20 then the color = Dark RedIf the mean > 10 and the mean <= 20 then the color = RedIf the mean > 5 and the mean <= 10 then the color = Orange If the mean <= 5 then the color = Yellow If the mean <= 1 and the max < 5 then the color = Green If the mean <= 0.3 and the max < 1 then the color = Blue
This is the approach used on the CCAMP website
Scoring Approach 2
Quotient MethodDivide measurement by a standard
Pro: + scientific consensus behind standards
+ capture magnitude of excursion
Con: - no fixed upper end to scale
- appropriate standards must be available
Scoring Approach 3
Magnitude and Exceedance Quotient (MEQ)
Combines two terms that express number of threshold exceedances and magnitude of exceendance.
Derived from Canadian Ministry of Governments method
Canadian CCME WQ Index
Factor 1: Scope
Factor 2: Frequency
Factor 3: Amplitude
Canadian CCME WQ Index, cont.
CCME WQ Index
Quadratic mean (or root mean square) gives greater weight to larger values, thus emphasizing excursions.
Canadian CCME WQ Index, cont.
Pro: + widely excepted (esp. for drinking water)
+ considers scope, frequency, amplitude
Con: - resolution to detect differences among sites or times depends heavily on standards selected
- Not as useful for characterizing “healthy” because it only looks at failed tests- Scope is susceptible to site variability in number of analytes tested (best for very consistent study design)
MEQ: modifications to CCME
We are scoring all tests, not just failed tests. This provides a “good” end to the scoring tool
We eliminated the scope term (how many analytes fail) because we have inconsistent data availability across sites.
We do the calculations at the level of the analyte so that each analyte can also be scored
Calculating MEQ
Factor 1: Frequency
Calculate rate of exceedance (frequency) for each individual analyte (from CCME)
F1
F1
Calculating MEQ, cont.
Calculate unitless sample magnitudes:Magnitude = concentration/threshold
Calculate normalized sum of magnitudes
NSM = Σ magnitudes / sample count
Scale from 0 – 100= NSM / (0.01*magnitude average+0.01)
Calculate magnitude (amplitude) for each individual analyte (from CCME)
Calculating MEQ, cont.
•
MEQ = 100 – √ exceedance2 + magnitude2
1.414
Combine exceedance and magnitude factors and scale to 100 (modified from CCME)
Aggregate scores into an index1. Number of standard excursions per site
2. Means: arithmetic, geometric, harmonic, quadratic
3. Worst case
4. Weights and penalty factors
5. Regression of simple to comprehensive indices
6. Empirical formulae