Download - Randomization workshop
Randomization workshop
eCOTSMay 22, 2014Presenters: Nathan Tintle and Beth Chance
Introductions Presenters
Nathan Tintle, Dordt College Beth Chance, Cal Poly
Participants Telling you a little about you!
Participant profile
Resea
rch
univ
ersit
y
4-ye
ar col
lege
Comm
unity
col
lege
High
scho
ol
Other
0
0.1
0.2
0.3
0.4
Institution
Wo
rksh
op
part
icip
an
ts
Participant profile Over 85% are here because of
AP-equivalent introductory statistics
Remainder Calculus based introductory statistics Other statistics course
Participant profile
0
0.1
0.2
0.3
0.4
0.5
0.6Previous experience with randomization in the
classroom
Wo
rksh
op
part
icip
an
ts
Participant profile Goals
What is ‘randomization-based’ inference? Why would I want it in my course? How does it look in my course? Challenged with new ideas Implementation tips and advice Comparisons with other randomization
texts And a lot more!
Our primary goals are for you Understand what a ‘randomization/ simulation’
based approach to statistical inference is Understand why it is an increasingly popular
approach to teaching introductory statistics Have experienced two concrete examples of
how it works in the classroom Have a sense of one of the major curriculum
options available for teaching with this approach
Overview Hour #1
First 15-20 minutes: Welcome/introductions/overview/goals Next 15-20 minutes: What is a randomization-based curriculum? Why?* Next 15-20 minutes: Activity: Meet Doris and Buzz *
Hour #2 (after short 5 minute break) First 10-15 minutes: The ISI curriculum: What, how, and why* Next 15-20 minutes: Activity: Is yawning contagious?* Final 10-15 minutes: Cautions, implementation, assessment* Final 10-15 minutes: Next steps, class testing, ongoing discussion*
*Ask questions both during and immediately following each presentationTo ask a question, type the question into the “Questions” pane of the GoToWebinar control panel. Do not use the ‘raise hands’ feature.
During all sessions we STRONGLY encourage you to ask questions!
Note: We will post slides after workshop, plus recording available via eCOTS
What do we mean by a ‘randomization-based’ curriculum and why consider it?
Overview Why look at the content of Stat 101?
Stat 101= general algebra based intro stats course (equivalent to AP Statistics)
George Cobb’s challenge about how the content might change Randomization/simulation as an overarching
approach to statistical inference Some general trends, themes in
randomization curricula to date
Brief history of stat ed Consensus curriculum by late 1990s, but nexus
in early 1980s Descriptive Statistics Probability/Design/Sampling Distributions Inference (testing and intervals)
GAISE College Report (2005) Six pedagogical suggestions for Stat 101:
Conceptual understanding, Active learning, Real data, Statistical literacy and thinking, Use technology, and Use assessments for learning
Brief history of stat ed No real pressure to change content
Major changes Computer changed from an institutionally owned behemoth
to the individually owned desktop or laptop.
As computers became ubiquitous, so did data collection, and with it the need for data analysis.
Statistical practice changed as well. More computer intensive methods, large datasets, multivariable methods, etc.
Recognition of the utility of simulation to enhance student understanding of random processes
Brief history of stat ed Other changes directly impacting Stat
101 Stats increasing in K-12 curriculum (NCTM,
Common Core, Advanced Placement) (Franklin eCOTS plenary talk)
Enrollments have skyrocketed (High school, two and four year colleges)
Stat ed research has given us more knowledge of how and what students learn in Stat 101
Potential shortcomings Overlap with K-12 treatment of descriptive statistics is inefficient
Eventually will also have exposure to informal inference
Although computer-intensive methods have become a central part of statistical practice, they are largely or wholly absent from the typical first course.
Assessment methods developed over the last decades show that student understanding of the logic of inference is typically limited at best (Cobb 2007, TISE http://escholarship.org/uc/item/6hb3k0nz)
The traditional first course does not devote sufficient time or space to the connections among the method of data production, the method used to analyze the data, and the scope of inference justified by the analysis. For randomization-based methods, these connections are simple and direct.
Intro Stat as a Cathedral of Tweaks (A George Cobb analogy)
Boswell famously describe Samuel Johnson as a “cathedral of tics.” Thesis: The usual normal distribution-worshipping intro course is a cathedral of tweaks.
The orthodox doctrine is simple Use the CLT to justify the normal Use the normal to compute tail areas Reject if |observed – expected0| > 2SEs Interval = Est +/- M95 with M95 = 2SEs
The Cathedral of Tweaks(a) z vs t: If we know the population SD we use z; when we estimate the SD we use t
(b) z vs. t: (a) holds except for proportions; then we use z, not t, even when we estimate the SD.
(c) Estimating the SD. For proportions, we estimate the SD for intervals, but use the null value for tests.
(d) n vs. (n-1) vs. (n-2). The SE is SD/(root n): We divide by n because we have n observations. But for estimating the SD we divide by (n-1), even though there are n deviations … except that when we get to regression we use (n-2).
Still More Tweaks
If your data set is not normal you may need to transform
If you work with small samples there are guidelines for when you can use methods based on the normal,
e.g., n > 30, or np > 5 and n(1-p) > 5
The consequence Few students ever leave our course seeing this
The consequence The better students may get a fuzzy impression
The consequence All too many noses stay too close to the canvas,
and see disconnected details
A potential solution? ‘Randomization’ = simulation, bootstrapping
and/or permutation tests
Use of computationally intensive methods to:
Estimate/approximate the null distribution for significance tests
Estimate/approximate the margin of error for confidence intervals
A potential solution: Simulation Flip coins or spin spinners to simulate
the binomial distribution instead of starting with Binomial distribution theory Normal approximation to the binomial
Simulation example What are the chances a basketball player is
shooting free throws better in the playoffs (16/20 in game 1) than they typically do, if they “typically” make 50% of their free throws?
Flip 20 coins to simulate performance of player if no change in free throw percentage
Repeat to assess the likelihood of such a player making 16 FTs
How often would we get such a statistic as in the study by chance alone? That is, if still 50%?
A potential solution: Bootstrap Bootstrap
Use 1000s of resamples (with replacement) of the observed data to generate an approximate sampling distribution which can be used to estimate the margin of error
Bootstrap example Example: Gather hours slept last night for 20
students
5,5,6,6,6,7,7,7,7,8,8,8,6.5,6.5,7,7.5,7.5,7.5,7,4
Mean=6.7 hours, SD=1.1 hours
Find: 95% CI for population average sleep hours
Bootstrap: Model the data gathering process: 1000 random samples of 20 with replacement, compute sample mean each time.
Keep going!
Bootstrap example
(re)Sample means
Bootstrap example Find middle 95% of resample means =
95% CI for true population mean
6.2 to 7.1 hours
How much might these sample means vary from sample to sample by chance alone?
A potential solution: Permutation tests Permutation testing
Compare 2 or more groups Null: No treatment effect; distribution of
response variable is the same in all groups Ex. Is new treatment better than placebo?
A potential solution: Permutation tests
Write the values of the response variable (cat or quant) on slips of paper. Shuffle slips and re-randomize to the two or more groups
Recompute value of the statistic and get empirical null distribution, compare to actual statistic
How often would we get such a statistic as in the study by chance alone if null is true?
A potential solution These methods may offer a quicker, less
abstract bridge to the logic of inference while also emphasizing the scope of inference (random sampling, random assignment)
May scaffold the transition to ‘traditional’ (asymptotic; theory-based methods) better than traditional theory/probability theory, etc.
General trends Momentum behind randomization-approach to
inference in last 8-10 years Cobb 2005 talk (USCOTS) Cobb 2007 paper (TISE) 2011 USCOTS: The Next Big Thing
New and coming soon curricula Lock5 (theory and randomization, more traditional
sequence of topics) Tintle et al. (theory and randomization, four pillars of
inference and then chapters based on type of data) CATALST (emphasis on modelling) Others
General trends Many sessions at conferences talking
about approach, benefits, questions/concerns
Assessment: Two papers (Tintle et al. 2011, Tintle et al. 2012); Better on some things, do no harm on others; more coming
Q+A
Doris and BuzzSimulation for a single proportion
Introduction First ‘main’ example (after brief
Preliminaries) Story Questions for students
Can we prove dolphins can communicate abstract concepts?
What other explanations are there? How explain/justify to someone else? Chance model, simulation
Three S Strategy Statistic: Compute the statistic from the observed
sample data.
Simulate: Identify a model that represents a “by chance” explanation. Repeatedly simulate values of the statistic that could have happened when the chance model is true.
Strength of evidence: Consider whether the value of the observed statistic is unlikely to occur when the chance model is true.
Dolphin CommunicationStatisticIn one set of trials, Buzz chose the correct
button 15 out of 16 times.
Based on these results, do you think Buzz knew which button to push or is he just guessing?
Dolphin CommunicationSimulate
coin flip = guess by Buzzheads = correct guess
tails = wrong guesschance of
heads=
probability of correct button when Buzz is just guessing
one set of 16 coin flips
= one set of 16 attempts by Buzz
Dolphin CommunicationSimulate
What mightbe on the frontboard in classof 25 students
Larger class fine!
Dolphin CommunicationSimulate
Still not convinced 15 is unlikely? Go to applet to get a ‘very large class’ flipping coins with you
http://math.hope.edu/isi Click on Applets, then click one proportion
Applets are javascript and so work on all platforms including iPhones, iPads, etc.
Moving past Doris and Buzz Null/Alt hypotheses, non-50/50 null (1.2) Parameter (1.1) Strength of evidence
P-value (1.2) Standardized statistic, Z (1.3)
Two-sided tests, what impacts strength of evidence (1.4)
Theory-based approaches (overlay normal) (1.5)
Q+A
End of hour #1 Short break-back in 1 minute!!
We’ll start at 38 minutes after the hour