data analysis do it yourself!. what to do with your data? report it to professionals (e.g., aavso)...
TRANSCRIPT
Data Analysis
Do it yourself!
What to do with your data?
• Report it to professionals (e.g., AAVSO)– Excellent! A real service to science; don’t
neglect this
• Publish observations (e.g., JAAVSO)
• Analyze it – yourself!
But …
• I’m not a mathematician– Let the computer do the math
• I’m not a programmer– Get programs from the net (often free)
• I don’t know how to use or interpret them– Neither do the pros!– Practice, practice, practice …
Time Series Analysis
• A time series is a set of data pairs
• t is the time, x is the data value• Usually, times are assumed error-free• Data = Signal + Error
• x can be anthing, e.g. brightness of variables star, time of eclipse, eggs/day from a laying hen
Naxt aa ,...,3,2,1),,(
)()( tftx
Basic properties of data x
Actual
• Mean = = expected value
• Standard deviation = expected rms difference from mean
Estimated
• Average = estimated
• Sample standard deviation = estimated
Average and sample standard deviation
• Average
• Sample standard deviation
xN
x1
2)(1
1xx
Ns
Method #1: world’s best• Eye + Brain: Look at the data!
• Plot x as a function of t: Explore!
• Scientific name:
Visual Inspection
• World’s best – but not infallible
• Programs:– TS http://www.aavso.org– MAGPLOT http://www.aavso.org
Method #2: Fourier Analysis
• Period analysis and curve-fitting
• Powerful, well-understood, popular
• Programs– TS http://www.aavso.org– PerAnSo http://www.peranso.com
Method #3: Wavelet Analysis
• Time-frequency analysis
• Old versions bad, new version good
• Programs:– WWZ http://www.aavso.org– WinWWZ http://www.aavso.org
Visual Inspection
Let’s take a look
Fourier Analysis
Fourier analysis for period search
• Match the data to sine/cosine waves
• = frequency
• Period =
• Amplitude = A = size of fluctuation
• Obvious choice is period; mathematically sound choice is frequency
)2sin()2cos()( 210 tctcctf
/1P
Null Hypothesis (important!)
• Null hypothesis: no time variation at all
• So = constant
• So,
• Quite important! Often neglected. Even the pros often forget this.
)(tf
aax
Is it real?
• Fit produces a test statistic under the null hypothesis
• Is usually “ /degree of freedom” (d)
• Linear: is significant (not just by accident) at 95% confidence
• 95% confidence means 5% false-alarm probability
2
42
Meaning of significance
• Significance does not mean the signal is linear, sinusoidal, periodic, etc.
• It only means the null hypothesis is incorrect, i.e., the signal is not constant
• Important!!!
Pre-whitening
• If you find a significant fit, then subtract the estimated signal, leaving residuals
• Analyze the residuals for more structure
• This process is called pre-whitening
How to choose frequency?
• Test all reasonable values, get a “strength of fit” for each. Common is “chi-square per degree of freedom” (but there are many)
• Plot frequency .vs. fit – the Fourier transform (aka periodogram, aka power spectrum)
Fourier decomposition
Any periodic function of period P
(frequency ) can be expressed as a Fourier series:
P/1
...)2cos()2sin(
...)6cos()6sin(
)4cos()4sin(
)2cos()2sin()(
33
22
11
tnctnb
tctb
tctb
tctbatF
nn
Fundamental + harmonics
For a pure sinusoid, expect response at frequency
For a general periodic signal at a given frequency, expect a fundamental component at , as well as harmonics at frequencies etc.
,4,3,2
Lots of Fourier methods
• FFT: fast Fourier transform– Not just fast: it’s wicked fast
– Requires even time spacing
– Requires N=integer power of 2
– Beware!
• DFT: discrete Fourier transform– Applies to any time sampling, but incorrect results for
highly uneven (as in astronomy!)
– Beware!
Problems from uneventime sampling
• Aliasing: false peaks, often from a periodic data density
• Aliases at
• Common in astronomy: data density have a period P = 1 yr = 365.2422 d, so
• Solution: pre-whitening
datasignal n
002738.0data
Aliasing
Aliasing: UZ Hya
Problems from uneventime sampling
• Mis-calculation of frequency (slightly) and amplitude (greatly); sabotages prewhitening
Solution: better Fourier methods (for astronomy)
• Lomb-Scargle modified periodogram– Improvement over FFT, DFT
• CLEAN spectrum– Bigger improvement
• DCDFT: date-compensated discrete Fourier transform (this is the one you want)
• CLEANEST spectrum: DCDFT-like for multiple frequencies
DCDFT
• Much better estimates of period, amplitude
Let’s take a look
• Peranso (uses DCDFT and CLEANEST)
• Available from CBA Belgium– http://www.peranso.com
Fourier transform (CLEANEST) of TU Cas
Wavelet Analysis
Wavelets
• Fit sine/cosine-like functions of brief duration
• Shift them through time
• Gives a time-frequency analysis
Problems
• Same old same old: uneven time spacing, especially variable data density, invalidate the results
• But: even worse than Fourier
• Essentially useless for most astronomical data
Wavelet methods
• DWT: discrete wavelet transform– Just not right for unevenly sampled data
(astronomy!)
• Solution: WWZ = weighted wavelet Z-transform
Let’s take a look
Data Analysis• Do it yourself
• Use your eyes and brain
• Healthy skepticism
• Enjoy!