teaching sgt in the 21 st century july 16-19 2012

Teaching SGT in the 21st Century

July 16-19

2012Data Filtering and Noise

Reduction

Scott T. MarshallDepartment of Geology

Appalachian State University

Course InformationGLY3160 / PHY3160 – Introduction to Geophysics• Prerequisites

– Calc I, Physics I, any Intro GLY course– Sophomores - Seniors

• Required for Quantitative Geoscience Degree– Elective for all other degrees (PHY, ENV too)~50% are GLY majors~25% PHY majors~25% ENV majors

• Given as second lab of the semester– First lab: Dimensional analysis and math refresher– Lab classroom: computer lab

The Textbook

Overarching Goals• Plot data & equations using a computer– Reasonable professional style– Efficient use of Excel– Learn to use equations and unknown functions

• Be comfortable with manipulating equations– Basic algebra– Translating / stretching a function– Calculating error / Detrending data

• Make a real-world recommendation based on quantitative synthetic data analysis

Misconceptions• 21st Century Geology Students are:– Good with computers

• can make graphs using computers• good at MS Excel

– Afraid of math• cannot do math• do not want to do math

• Interpreting plots is enough

www.education.ti.com

www.fanpop.com

*(in my opinion) The Reality*

• Most students can do basic math– get stuck on little things– a little help goes a long way

• Most students are comfortable with computers– have no idea how to use computers to solve quantitative

geological problems– are easily taught computer tasks– a little help goes a long way

• When students make a plot they learn more

• Teaching quantitative skills is important– keeps math skills fresh– better prepares for grad school and real world

What Are We Modeling?

Annual Variation in Daily High Temps in Boone, NC• Min: 39 °F

– Occurs on Jan 1st

• Max: 76 °F– Occurs 0.5 yr later

• Temp varies sinusoidally– Give general form of sinusoid

• Plot temps for 3 yrs (1095 days)– Requires students to be efficient with Excel

𝒚=𝒔𝒊𝒏(𝟐𝝅 𝒙𝝀 )

The model is unimportant*

*conceptually simple

Translating / Stretching

𝑻=𝒔𝒊𝒏(𝟐𝝅 𝒙𝟑𝟔𝟓 )

λ = 365

Translating / Stretching

𝑻=(𝟕𝟔−𝟑𝟗 )

𝟐∗𝒔𝒊𝒏(𝟐𝝅 𝒙

𝝀 )Range = 37 °FAmplitude = 18.5 °F

Translating / Stretching

𝒚=(𝟕𝟔−𝟑𝟗)

𝟐∗ 𝒔𝒊𝒏(𝟐𝝅 𝒙

𝝀 )+ (𝟕𝟔+𝟑𝟗 )𝟐

Mean = 57.5 °F

Translating / Stretching

𝒚=(𝟕𝟔−𝟑𝟗)

𝟐∗ 𝒔𝒊𝒏(𝟐𝝅 (𝒙+(𝟑𝟒∗𝟑𝟔𝟓))

𝟑𝟔𝟓 )+ (𝟕𝟔+𝟑𝟗)𝟐

Shift to left by ¾ λ

• How sensitive of a temp gauge is needed?– Capture annual variations, not day to day– Factor in reasonable costs of different devices

• What sampling rate is recommended?– Avoid aliasing– Stay practical

• How can we reduce noise/error?– Moving window / Running average filter?

• Implication = More processing

– Stacking?• implication = More devices, more processing

• Give final results and recommendation in a formal report

Questions For The Model

Adding Synthetic Noise

All measurements have unknown error• Add noise to the model to simulate 3 years

of data– i.e. synthetic data

• Use Excel’s “RAND()” function– Generate a tweakable noise parameter– Start with +/- 10 noise– Do not use “RANDBETWEEN(x,y)”

• Only integers!

Noise Function• Assume noise is random (for simplicity)• Use random number generator– Do not tell how the RAND() function works– Make them plot it and see what it does

– RAND() = random floating point num from 0 to 1• How can we get +/- 10 ?

• RAND() is a function– Use same stretching rules as before

• Make a cell in Excel to hold the +/- value– E.g. A1=10– ($A$1*2) * RAND() - ($A$1)

• Use $ to hold location constant

Noise Function

Noisy Synthetic Data

• Now add the noise model to the Temp model– A synthetic dataset

Noisy Synthetic Data

• What temp error is acceptable?• Where is error largest?

Data Filtering / Stacking

• Noise Reduction Tests– 3 and 5 day moving window filters (running

average)– 7 day weighted filter– Stacking (5, 20 datasets)

• Calculate residuals (i.e. detrend data)– Calculate Root Mean Squared (RMS) error for

each

Stacking Results

Final Report• Hand in a formal final report– Include plots as figures (must be referenced)

• Determine a research plan– Error of temp gauge– Sampling rate– Data processing

• Open-ended: lots of reasonable answers