randomization workshop
DESCRIPTION
Randomization workshop. May 22, 2014 e COTS workshop Presenters: Nathan Tintle and Beth Chance Hour #2. Overview of next hour. Hour #2 (after short 5 minute break) First 10-15 minutes: The ISI curriculum: What, how and why* Next 15-20 minutes: Activity: Is yawning contagious?* - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/1.jpg)
MAY 22 , 2014ECOTS WORKSHOP
PRESENTERS: NATHAN TINTLE AND BETH CHANCEHOUR #2
Randomization workshop
![Page 2: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/2.jpg)
Overview of next hour
Hour #2 (after short 5 minute break) First 10-15 minutes: The ISI curriculum: What,
how and why* Next 15-20 minutes: Activity: Is yawning
contagious?* Final 10-15 minutes: Cautions, implementation,
assessment* Final 10-15 minutes: Next steps, class testing,
ongoing discussion**Ask questions both during and immediately
following each presentation
![Page 3: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/3.jpg)
I N T R O D U C T I O N T O S TAT I S T I C A L I N V E S T I G AT I O N S
A U T H O R S : T I N T L E , C H A N C E , C O B B , R O SS M A N, R O Y, S WA N S O N, A N D VA N D E R S T O E P
P R E L I M I N A RY E D I T I O N AVA I L A B L E V I A W I L E Y FA L L 2 0 1 4
The ISI curriculum
![Page 4: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/4.jpg)
Goals
Introduce the particular curriculum that we have developed over the last 4+ years Goals Distinctive features Technology Key things to keep in mind
![Page 5: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/5.jpg)
Vision
An alternative Stat 101 (Algebra-based intro stats) course which uses randomization and simulation to motivate inference
GAISE from ground up
Not alienating client departments
![Page 6: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/6.jpg)
Six distinctive features to achieve the goals
1. Spiral approach to the 6-steps of statistical investigation
![Page 7: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/7.jpg)
Six-Step Process
![Page 8: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/8.jpg)
Six Step Process
Students have been able to consider the entire statistical process:Can dolphins
communicate?Performance of
Buzz/DorisAssessing statistical
significanceDraw appropriate
conclusions
![Page 9: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/9.jpg)
6 distinctive features to achieve the goals
1. Spiral approach to the 6-steps of statistical investigation
Start in Preliminaries
Revisit repeatedly throughout the book, starting with simpler data (e.g., single binary variable), and moving through a variety of more complex data situations
Deeper and deeper look at the 6-steps as the course moves on
Emphasizes a big picture, research-oriented view of statistical reasoning
![Page 10: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/10.jpg)
6 distinctive features to achieve the goals
2. Randomization-based introduction to statistical inference
Use simulation and randomization to first introduce statistical inference
Transition to traditional (asymptotic methods) as a prediction to the simulation/randomization results
Simple and direct connections between method of data production, method used to analyze the data, and the appropriate scope of conclusions
![Page 11: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/11.jpg)
6 distinctive features to achieve the goals
3. Focus on the logic and scope of inference (the “pillars” of statistical inference
Logic: Significance (How strong is the evidence?) Confidence (How large is the tendency for difference and how confident
can we be in our inferences?)
Scope: Generalizability (To which population can the conclusion be reasonably
generalized?) Causation (Is a cause-effect conclusion possible?)
Once we have the tools, we ask these four questions of nearly every data set we look at; as part of the entire 6-step statistical investigation process we walk through nearly every time
![Page 12: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/12.jpg)
6 distinctive features to achieve the goals
4. Integration of exposition, explorations, and examples.
Overview of each section in the book
Common introduction
ExplorationExample
Common conclusion
![Page 13: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/13.jpg)
6 distinctive features to achieve the goals
Lots of flexibility with how to walk through material within each section Key idea: Examples and explorations do not depend on
each other; for example, definition boxes are in both
![Page 14: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/14.jpg)
6 distinctive features to achieve the goals
5. Easy to use technology throughout
Freely available suite of web-applets Visualizing simulation and randomization Integration of simulation and theory-based approaches Pasting datasets
Allows for supplementing with a traditional software package
![Page 15: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/15.jpg)
6 distinctive features to achieve the goals
6. Real data from genuine studies
Taken from a variety of fields of interest; popular appeal
Real, published research in many cases; some student gathered datasets as well
Exercises, in-depth investigations, research articles
![Page 16: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/16.jpg)
Content sequencing
Traditional Stat 101 1. Descriptive statistics and study design 2. Probability and sampling distributions 3. Inference
Our Stat 101 Unit 1. Introduction to the four pillars of statistical
inference Unit 2. Comparing two groups Unit 3. Analyzing more general situations
![Page 17: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/17.jpg)
Content sequencing
Unit 1. Four pillars Preliminaries. Statistical thinking (6-steps), Variability, and
Probability (Long-run frequency)
Chapter 1. Significance (3-S process, chance model); one proportion
Chapter 2. Generalization (To whom can we generalize?); one proportion, one mean; types of errors
Chapter 3. Confidence (Range of plausible values; 2SD); one proportion, one mean
Chapter 4. Causation (Is cause-effect possible?)
![Page 18: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/18.jpg)
Content sequencing
The following chapters all have a similar flow Descriptive statistics Simulation/Randomization approach Theory-based approach
Unit 2. Comparing two groups Chapter 5. Comparing two proportions Chapter 6. Comparing two group on quantitative response (means or
medians) Chapter 7. Comparing two paired groups (on quantitative response; and,
one sample t-test)
Unit 3. More general situations Chapter 8. Comparing more than two groups using proportions Chapter 9. Comparing more than two groups using means Chapter 10. Analyzing two different quantitative variables
![Page 19: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/19.jpg)
Content sequencing
Comments Descriptive statistics are “just in time”; chapters are
focused more on type of data (allows for the application of all 6-steps)
Probability and sampling distributions come up throughout the book using tactile and computer simulations to estimate sampling distributions; no formal rules of probability needed
Theory-based approaches are merely convenient alternatives to simulation which predict what would happen if you simulated, assuming certain conditions are met
![Page 20: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/20.jpg)
Pedagogy
Built a course from the ground up that was based on GAISE principles Statistical literacy and thinking Conceptual Active Real data Technology to drive understanding Assessments for continuous improvement
![Page 21: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/21.jpg)
Is it working?
Preliminary evidence is positive Students, instructors enjoy it and appear to be learning more
We’ve documented learning gains in a number of key areas with preliminary versions of the curriculum (Tintle et al. 2011), with little to no evidence of declines vs. the standard curriculum in other areas
These gains are retained longer by students in this curriculum than with the traditional curriculum (Tintle et al. 2012)
Still are actively gathering assessment data across multiple institutions every semester with a long-term vision of continual improvement to maximize student learning
MORE LATER THIS HOUR
![Page 22: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/22.jpg)
Comparisons with other curricula
CATALST—focus on modelling
Lock5 – different content ordering; bootstrapping
**Note: Others are under-development
![Page 23: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/23.jpg)
Final remarks
Why so much time on proportions and not quantitative? Easiest place to start Cover 4-pillars, 6-steps and 3-S then apply them all,
everywhere
![Page 24: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/24.jpg)
Comparing Two Proportions
Chapter 55.1: Descriptive statistics for 2 proportions5.2: Inference with Simulation-Based Methods5.3: Inference with Theory-Based Methods
![Page 25: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/25.jpg)
Exploration 5.2: Is yawning contagious?
http://www.discovery.com/tv-shows/mythbusters/videos/is-yawning-contagious-minimyth.htm
![Page 26: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/26.jpg)
Is yawning contagious?
Are people who see someone yawn more likely to yawn themselves?Mythbusters recruited 50 peopleRandomly assigned to 3 rooms; 2 with yawn
seed planted, one without
![Page 27: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/27.jpg)
Example questions from guided discovery ‘exploration’
Think about why the researchers made the decisions they did.
Why did the researchers include a group that didn’t see the yawn seed in this study? In other words, why didn’t they just see how many yawned when presented with a yawn seed?
Why did the researchers use random assignment to determine which
subjects went to the “yawn seed” group and which to the control group?
Is this an observational study or a randomized experiment? Explain how you are deciding.
The researchers clearly used random assignment to put subjects into groups. Do you suspect that they also use random sampling to select subjects in the first place? What would random sampling entail if the population was all flea market patrons?
![Page 28: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/28.jpg)
Example questions
Yawn seed planted Yawn seed not planted
Total
Subject yawnedSubject did not yawnTotal
The researchers found that 11 of 34 subjects who had been given a yawn seed actually yawned themselves, compared with 3 of 16 subjects who had not been given a yawn seed.
Organize this information into the following 2×2 table:
![Page 29: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/29.jpg)
Is yawning contagious?
Results
Yawn seedNo yawn
seedTotal
Yawned 11 (32.4%) 3 (18.8%) 14
Didn’t yawn 23 13 36Total 34 16 50
Yawn seed No yawn seed0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Didn't yawnYawned
Did subject get randomly assigned to yawn seed group?
Did
subje
ct
yaw
n?
![Page 30: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/30.jpg)
Is yawning contagious?
The difference in proportions of yawners is 0.324 – 0.188 = 0.136.
There are two possible explanations for an observed difference of 0.136. A genuine tendency to be more likely to yawn with
seed The 14 subjects who yawned were going to yawn
regardless of the seed and random chance assigned more of these yawners to the “yawn seed” group
![Page 31: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/31.jpg)
Is yawning contagious?
Null hypothesis: Yawn seed doesn’t make a difference
Yawn seed or not has no association with whether someone yawns
seed = no seed or seed no seed = 0
Alternative hypothesis: Yawn seed increases chances of yawning
Yawn seed increases the probability of someone yawning; association
seed > no seed or seed no seed > 0
![Page 32: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/32.jpg)
Is yawning contagious?
The parameter is the (long-run) difference in the probability of yawning between yawn seed and no seed groups
Our statistic is the observed difference in proportions 0.324 – 0.188 = 0.136
![Page 33: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/33.jpg)
Is yawning contagious?
If the null hypothesis is true (yawn seed makes no difference) we would have 14 yawners and 36 non-yawners regardless of the group they were in.
Any differences we see between groups arise solely from the randomness in the assignment to the groups.
![Page 34: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/34.jpg)
Is yawning contagious?
We can perform this simulation with index cards. 14 blue cards represent the yawners 36 green cards represent the non-yawners
We assume these outcomes would happen no matter which treatment group subjects were in.
Shuffle the cards and put 34 in one pile (yawn seed) and 16 in another (no seed)
An yawner is equally likely to be assigned to each group
In class we do this!
![Page 35: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/35.jpg)
Is yawning contagious?
First simulation 9 blue (yawners) and 25 green (non-yawners) in yawn
seed group 5 blue (yawners) and 11 green (non-yawners) in no
yawn seed group
Difference in proportions? 9/34 – 5/16 = -0.048
“Chance” value of the statistic
Repeat many times
![Page 36: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/36.jpg)
Is yawning contagious?
Confession We tweaked the data!
Actually 10/34 yawners in seed group 4/16 in control group
Difference is only 4.4%!
By this time our students realize that’s not enough to be statistically significant (even though Adam and Jamie didn’t)
![Page 37: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/37.jpg)
Cautions, implementation and assessment
![Page 38: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/38.jpg)
Cautions
The good; Question on small p-values National sample (not randomization)
Pre-test: 50%, Post-test: 69% Fall 2013, dozen institutions using ISI text
Pre-test: 44%, Post-test 84% (some nearly 100%) SERJ article (2012)
Retention of this concept is good 4 months later
Not going to solve all of your problems! Assessment data is positive, but doesn’t mean
everything is better (concepts some better, much the same; attitudes similar)
![Page 39: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/39.jpg)
Cautions
Biggest misconceptions we create with this approach:
1. Need multiple samples in real life to analyze data
Solution: Emphasize reality vs. pretend world where null is true
2. Thinking you have proven /gotten evidence for the null hypothesis
Solution: focus on the idea of the assumptions behind the simulation and the idea of modelling in general
3. Still get a little dependent on mean/proportion
Solution: We are hoping to show more transfer questions so they can use any statistic they come up with
4. Assuming too much student background? Solution: Have included the preliminary chapter for those who want a real quick introduction to “background” ideas
![Page 40: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/40.jpg)
How to convince others
Focus on what they get Better understanding of logic and scope inference Focus on 6-steps (scientific reasoning) Real studies/research Still coverage of the theory based test or tests you
know Still coverage of descriptive statistics topics Conceptual understanding of probability and sampling
distributions Good transition to applied second course (stat/math
dept or client dept.
![Page 41: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/41.jpg)
How to convince others
Content re-ordering and re-focus more than content change
What are your needs? It will still meet them, and likely do even better at meeting them then the current course.
![Page 42: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/42.jpg)
How to convince others
Embracing active, guided discovery pedagogy which engages students and improves student learning (Guidelines of Assessment and Instruction in Statistics Education; GAISE)
“How do you do this all in one semester?” Efficiency of approach/similarities between framework of inference More accessible Focusing more on the important stuff (inference) Topics we don’t emphasize:
Reading probabilities from a table Notation (doing more and more “in words”) Using a random number table More getting less on data cleaning, other sampling
methods and other subtleties
![Page 43: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/43.jpg)
How to convince others
But what about probability? What about the central limit theorem? Not core to our approach Some are supplementing for AP More questions from math/stat colleagues than clients
Does it really work? Published assessment data
![Page 44: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/44.jpg)
How to convince others
But what about the second course? Developing second course materials that flows out of this Segway to research methods or other ‘traditional’ second courses fine
But what about a large class/online? Nathan and Beth both have done/are doing online/hybrid Applets are good outside of class; demo in class, use outside of class
But what about other text/software Integration has been done many ways so far (R, SPSS, Minitab, etc.);
just explorations as ‘labs’ with traditional text
But what about more ‘mathy’ students Faster More formulas
![Page 45: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/45.jpg)
Q+A
![Page 46: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/46.jpg)
Next steps, class testing, ongoing discussion
![Page 47: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/47.jpg)
Next steps
What first? Examine different curricula Think about an action plan. Try a day? Try a course? Talk with
colleagues? Talk with an author? What are your learning goals? How are you doing on them?
Participate in a longer workshop Chicago, Sioux Center (IA), San Luis Obispo, Flagstaff and Boston
(all this summer). See http://math.hope.edu/isi for details.
Lots of differences? Lots of options—what’s correct? No consensus Would like to make decisions based on assessment YOU have something to offer here!
![Page 48: Randomization workshop](https://reader036.vdocument.in/reader036/viewer/2022070406/568142d4550346895daf2bcb/html5/thumbnails/48.jpg)
Next steps
Sign up for copy of the book (on the post-workshop evaluations)
You will be contacted soon (mid-late summer) re: Participation in blog
Questions/discussion on randomization Monthly themes Stipend We will be up and running soon
Assessment project—could use your/colleagues students; we’ll provide reports. Especially non-randomization users!!
We are happy to help you work with your institution/colleagues to assist in implementation/discussions