tracking and analyzing student success data part 3
TRANSCRIPT
![Page 1: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/1.jpg)
Tracking and Analyzing Student Success Data
Part 3
![Page 2: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/2.jpg)
State/Territory Area (km2)
Afghanistan 645,807 606,467
Albania 28,748 25,643
Algeria 2,381,741 2,556,276
American Samoa 197 205
Andorra 464 577
Anguilla 96 88
Antigua & Barbuda 442 498
Argentina 2,777,409 2,537,555
Armenia 29,743 25,859
Aruba 193 210
Australia 7,682,557 7,365,389
Austria 83,858 81,434
Azerbaijan 86,530 91,752
Bahamas 13,962 9,125
Bahrain 694 725
Bangladesh 142,615 98,722
Barbados 431 517
Belgium 30,518 25,417
Belize 22,965 20,648
Benin 112,620 108,768
… … …
Which column lists the correct areas?
![Page 3: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/3.jpg)
A Bit of History
Late 1800s: Astronomer Simon Newcomb noticed that the early pages of log table books were more grubby than the later pages Users were looking-up numbers that started with digit 1 more often than numbers starting with, say, digit 5.
If the leading (first) digit is d, then the frequency of occurrence (probability) of the leading digit is
Log10 (1 + 1/d)
![Page 4: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/4.jpg)
If the leading (first) digit is d, then the frequency of occurrence (probability) of the leading digit is
Log10 (1 + 1/d)
Leading digit (d)
1 2 3 4 5 6 7 8 9
Probability of
occurrence
30% 18% 12% 10% 8% 7% 6% 5% < 5%
Number Leading (first) digit
350 3
42057 4
0.64 6
![Page 5: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/5.jpg)
A Bit of History (cont’d)
Late 1930s: Physicist Frank Benford rediscovered Newcomb’s formula: Log10 (1+1/d) Benford’s Law
Source: The Law of Anomalous Numbers, F. Benford, Proceedings of the American Philosophical Society, Vol. 78, 1938, pp: 551-572.
![Page 6: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/6.jpg)
Source: The Law of Anomalous Numbers, F. Benford, Proceedings of the American Philosophical Society, Vol. 78, 1938, pp: 551-572.
![Page 7: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/7.jpg)
State/Territory Area (km2)
Afghanistan 645,807 606,467
Albania 28,748 25,643
Algeria 2,381,741 2,556,276
American Samoa 197 205
Andorra 464 577
Anguilla 96 88
Antigua & Barbuda 442 498
Argentina 2,777,409 2,537,555
Armenia 29,743 25,859
Aruba 193 210
Australia 7,682,557 7,365,389
Austria 83,858 81,434
Azerbaijan 86,530 91,752
Bahamas 13,962 9,125
Bahrain 694 725
Bangladesh 142,615 98,722
Barbados 431 517
Belgium 30,518 25,417
Belize 22,965 20,648
Benin 112,620 108,768
… … …
![Page 8: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/8.jpg)
State/Territory Area (km2)
Afghanistan 645,807 606,467
Albania 28,748 25,643
Algeria 2,381,741 2,556,276
American Samoa 197 205
Andorra 464 577
Anguilla 96 88
Antigua & Barbuda 442 498
Argentina 2,777,409 2,537,555
Armenia 29,743 25,859
Aruba 193 210
Australia 7,682,557 7,365,389
Austria 83,858 81,434
Azerbaijan 86,530 91,752
Bahamas 13,962 9,125
Bahrain 694 725
Bangladesh 142,615 98,722
Barbados 431 517
Belgium 30,518 25,417
Belize 22,965 20,648
Benin 112,620 108,768
… … …
Correct List
Sou
rce:
Impr
ovis
ed fr
om R
. M. F
ewst
er, T
he A
mer
ican
Sta
tistic
ian,
Feb
. 200
9, V
ol. 6
3, N
o. 1
.
![Page 9: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/9.jpg)
Benford’s Law
in
MasteringTM
Rasil Warnakulasooriya
Pearson, Boston
![Page 10: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/10.jpg)
A Quick Demo of MasteringTM
masteringphysics.com
Thanks to Claire Masson for the Demo slides
![Page 11: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/11.jpg)
A Tutorial Problem in MasteringPhysics®
![Page 12: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/12.jpg)
Student enters 0 to Part A
![Page 13: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/13.jpg)
Students can open Hints
![Page 14: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/14.jpg)
Declarative Hint
Procedural Hint
![Page 15: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/15.jpg)
Feedback within Hints
![Page 16: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/16.jpg)
Student obtains the correct answers
![Page 17: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/17.jpg)
Time to Completion Data
Student Item Time interval (sec.)
[finished time-start
time]
First (leading) digit
S1 Tutorial X 350 3
S1 EOC Y 1048 1
S3 EOC Y 67 6
S4 Tutorial Z 42179 4
… … … …
![Page 18: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/18.jpg)
MasteringPhysics® time data follow Benford’s Law
closely
1 2 3 4 5 6 7 8 90.00
0.05
0.10
0.15
0.20
0.25
0.30
Pro
babili
ty
First Digit
Observed Benford's Law
= 0.006
![Page 19: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/19.jpg)
MasteringChemistry® time data follow Benford’s Law closely (better than in
physics)
1 2 3 4 5 6 7 8 90.00
0.05
0.10
0.15
0.20
0.25
0.30 = 0.002 P
robabili
ty
First Digit
Observed Benford's Law
![Page 20: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/20.jpg)
But what can we do with it?
![Page 21: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/21.jpg)
What can the deviations from the Benford’s Law tell
us?
![Page 22: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/22.jpg)
Deviation from Benford’s Law
1 2 3 4 5 6 7 8 90.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35Student 1
Pro
babili
ty
First Digit
Benford's Law Observed
less deviation
1 2 3 4 5 6 7 8 90.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
Pro
bab
ility
First Digit
Benford's Law ObservedStudent 2
more deviation
![Page 23: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/23.jpg)
Final Exam Score vs. Deviation from Benford’s Law
(Chemistry)
Thanks to professors Randall Hall & Leslie Butler for providing the exam score data
![Page 24: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/24.jpg)
Final Exam Score vs. Deviation from Benford’s Law
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40
-3
-2
-1
0
1
2
3
Correlation = -0.2095% CI [-0.34, -0.10]
Fin
al E
xam
Sco
re(in s
tandard
devi
atio
n u
nits
)
Deviation from Benford's Law
For every 1/5th of deviation from Benford’s Law, the exam score is decreased by 1/2 standard deviation
![Page 25: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/25.jpg)
To what degree is the average time
correlated with the final exam scores?
-0.03
(about 7 times less than the correlation found by using only the first digits; almost no correlation)
![Page 26: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/26.jpg)
The correlation (-0.2) is small but surprisingly good (?)
* Typical correlations encountered in educational research ~ 0.3 * High school GPA & SAT®-math correlated at 0.23
* SAT®-math and first-year college GPA correlated at 0.26
Source: College Board Research Report No 2008-5: Validity of the SAT® for Predicting First-Year College Grade Point Average, Jennifer L. Kobrin, et al.
![Page 27: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/27.jpg)
Can we identify at-risk students?
![Page 28: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/28.jpg)
Using average time: 0% of the students can be flagged
Students who sat for the final exam
Students who did not sit for the final exam
![Page 29: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/29.jpg)
Using Benford’s Law: ~19% of the students can be flagged
Students who sat for the final exam
Students who did not sit for the final exam
![Page 30: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/30.jpg)
* In time to completion data the first digit (or deviations from the Benford’s Law) carries more actionable information than all of the digits combined!
Summary
* Possibility of weighting the deviations from Benford's Law to identify at-risk students so that instructors can intervene effectively
* Possibility of weighting the deviations from Benford’s Law in deciding on specific study plans/personalized learning resources within Mastering
![Page 31: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/31.jpg)
![Page 32: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/32.jpg)
State/Territory Area (miles2)
Afghanistan 249,347 234,158
Albania 11,100 9,901
Algeria 919,595 986,983
American Samoa 76 79
Andorra 179 223
Anguilla 37 34
Antigua & Barbuda 171 192
Argentina 1,072,363 979,755
Armenia 11,484 9,984
Aruba 75 81
Australia 2,966,250 2,843,791
Austria 32,378 31,442
Azerbaijan 33,409 35,426
Bahamas 5,391 3,523
Bahrain 268 280
Bangladesh 55,064 38,117
Barbados 166 200
Belgium 11,783 9,814
Belize 8,867 7,972
Benin 43,483 41,996
… … …
![Page 33: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/33.jpg)
1 2 3 4 5 6 7 8 90
5
10
15
20
25
30
35
% o
ccurr
ence
first digit (time to completion in s)
Observed Benford's Law
Chemistry
![Page 34: Tracking and Analyzing Student Success Data Part 3](https://reader036.vdocument.in/reader036/viewer/2022081519/56649cdb5503460f949a5393/html5/thumbnails/34.jpg)
1 2 3 4 5 6 7 8 90
5
10
15
20
25
30
35
% o
ccurr
ence
first digit (time to completion in min)
Observed Benford's Law
Chemistry