Teaching SGT in the 21st Century
July 16-19
2012Data Filtering and Noise
Reduction
Scott T. MarshallDepartment of Geology
Appalachian State University
Course InformationGLY3160 / PHY3160 – Introduction to Geophysics• Prerequisites
– Calc I, Physics I, any Intro GLY course– Sophomores - Seniors
• Required for Quantitative Geoscience Degree– Elective for all other degrees (PHY, ENV too)~50% are GLY majors~25% PHY majors~25% ENV majors
• Given as second lab of the semester– First lab: Dimensional analysis and math refresher– Lab classroom: computer lab
The Textbook
Overarching Goals• Plot data & equations using a computer– Reasonable professional style– Efficient use of Excel– Learn to use equations and unknown functions
• Be comfortable with manipulating equations– Basic algebra– Translating / stretching a function– Calculating error / Detrending data
• Make a real-world recommendation based on quantitative synthetic data analysis
Misconceptions• 21st Century Geology Students are:– Good with computers
• can make graphs using computers• good at MS Excel
– Afraid of math• cannot do math• do not want to do math
• Interpreting plots is enough
www.education.ti.com
www.fanpop.com
*(in my opinion) The Reality*
• Most students can do basic math– get stuck on little things– a little help goes a long way
• Most students are comfortable with computers– have no idea how to use computers to solve quantitative
geological problems– are easily taught computer tasks– a little help goes a long way
• When students make a plot they learn more
• Teaching quantitative skills is important– keeps math skills fresh– better prepares for grad school and real world
What Are We Modeling?
Annual Variation in Daily High Temps in Boone, NC• Min: 39 °F
– Occurs on Jan 1st
• Max: 76 °F– Occurs 0.5 yr later
• Temp varies sinusoidally– Give general form of sinusoid
• Plot temps for 3 yrs (1095 days)– Requires students to be efficient with Excel
𝒚=𝒔𝒊𝒏(𝟐𝝅 𝒙𝝀 )
The model is unimportant*
*conceptually simple
Translating / Stretching
𝑻=𝒔𝒊𝒏(𝟐𝝅 𝒙𝟑𝟔𝟓 )
λ = 365
Translating / Stretching
𝑻=(𝟕𝟔−𝟑𝟗 )
𝟐∗𝒔𝒊𝒏(𝟐𝝅 𝒙
𝝀 )Range = 37 °FAmplitude = 18.5 °F
Translating / Stretching
𝒚=(𝟕𝟔−𝟑𝟗)
𝟐∗ 𝒔𝒊𝒏(𝟐𝝅 𝒙
𝝀 )+ (𝟕𝟔+𝟑𝟗 )𝟐
Mean = 57.5 °F
Translating / Stretching
𝒚=(𝟕𝟔−𝟑𝟗)
𝟐∗ 𝒔𝒊𝒏(𝟐𝝅 (𝒙+(𝟑𝟒∗𝟑𝟔𝟓))
𝟑𝟔𝟓 )+ (𝟕𝟔+𝟑𝟗)𝟐
Shift to left by ¾ λ
• How sensitive of a temp gauge is needed?– Capture annual variations, not day to day– Factor in reasonable costs of different devices
• What sampling rate is recommended?– Avoid aliasing– Stay practical
• How can we reduce noise/error?– Moving window / Running average filter?
• Implication = More processing
– Stacking?• implication = More devices, more processing
• Give final results and recommendation in a formal report
Questions For The Model
Adding Synthetic Noise
All measurements have unknown error• Add noise to the model to simulate 3 years
of data– i.e. synthetic data
• Use Excel’s “RAND()” function– Generate a tweakable noise parameter– Start with +/- 10 noise– Do not use “RANDBETWEEN(x,y)”
• Only integers!
Noise Function• Assume noise is random (for simplicity)• Use random number generator– Do not tell how the RAND() function works– Make them plot it and see what it does
– RAND() = random floating point num from 0 to 1• How can we get +/- 10 ?
• RAND() is a function– Use same stretching rules as before
• Make a cell in Excel to hold the +/- value– E.g. A1=10– ($A$1*2) * RAND() - ($A$1)
• Use $ to hold location constant
Noise Function
Noisy Synthetic Data
• Now add the noise model to the Temp model– A synthetic dataset
Noisy Synthetic Data
• What temp error is acceptable?• Where is error largest?
Data Filtering / Stacking
• Noise Reduction Tests– 3 and 5 day moving window filters (running
average)– 7 day weighted filter– Stacking (5, 20 datasets)
• Calculate residuals (i.e. detrend data)– Calculate Root Mean Squared (RMS) error for
each
Stacking Results
Final Report• Hand in a formal final report– Include plots as figures (must be referenced)
• Determine a research plan– Error of temp gauge– Sampling rate– Data processing
• Open-ended: lots of reasonable answers
