do uneb results predict competencies required to excel academically in law school? by robert wamala...
TRANSCRIPT
Do UNEB results predict competencies required to excel academically in law
school?
ByRobert Wamala (Ph.D)
School of Statistics and Planning, CoBAMs; Email - [email protected]
The dean, school of law writes: “There has
been a mismatch between entry grades and the
performance during the LL.B program and the
legal profession” (Makerere University, 2011,
p.1)
PRESS RELEASE: Pre-Entry Exams for LL.B Entrants
Students are admitted with triple or even quadruple “As”, which unfortunately
is in many cases, not reflected in their performance in law school.
There is overwhelming evidence in support of successful performance of enrollees who
have excelled academically in the past. This evidence is supported by studies across:
1. Disciplines
• Accounting and business economics (Duff, 2004; Alan & Othman, 2005)
• Business management and national statistics (Halpern, 2007; Hoskins, Newstead
& Denni, 1997)
• Nursing (Navarro, Vitamog, Tierra, & Gonzalez, 2011) and Actuarial Science
(Wamala, 2013)
2. Education Levels
• Undergraduate (Alan & Othman, 2005 ; Halpern, 2007 ; Wamala et al, 2013)
• Graduate (e.g., Navarro et al., 2011; Gregory, 2004 )
Relevance of prior studies
Competencies for undertaking LawInstitution GuidelinesUniversity of Texas - Undergraduate catalogue [2012/2014] Enrollees of law should be able to demonstrate proficiency in:
communication - writing in English, critical proficiency in oral and graphic communication
Conceptual approaches and history of arts – ability to comprehend factual concepts and human creativity
Political and economic dimensions of a society Cultural diversity including nature and limits of knowledge
and academic fields.
Alabama State Bar [Association of lawyers in the USA] Importance of enrollees’ grounding in the disciplines:
Analytical writing, English and literature, Political science, Economics and accounting, History, philosophy, logic, scientific methods Public speaking
Competencies for undertaking Law [Cont.’]Institution Guidelines
John Hopkins University [The parent advising hand book] Competence in economics, history, political thoughts and
mathematics provides a basis for building enrollees’ grounding regarding knowledge in resolving disputes.
University of Canterbury Importance of language and writing competencies: Enrollees with science, mathematics, music or art
background will only succeed in law school if they possess good language and writing skills
Makerere University [Guidelines for all public Universities] There has been a shift in the requirements in recent past:
Up until early 2000s, subjects namely History, literature and Divinity were considered essential
Subsequently, any subjects done by enrollees in their A-Level were considered essential (MoES, 2005; 2006; 2007)
From 2012 onwards, outcome of pre-entry tests are adopted as a basis for admission
To investigate the relevance of prior studies in predicting competencies required to
excel academically in law school. Focus was made on A-Level grades obtained in
disciplines, namely, History, Divinity, Literature, Economics and Geography.
Objective of the study
Data and Methods
• Administrative records of 629 graduates in the enrollment cohorts 2005 to 2007
– over the four stipulated period of study – were adopted [N = 2485]
• Academic achievement [dependent] was assessed by CGPA obtained in the first,
second, third and fourth year of study
• Investigations were made by: grades obtained in the A-level subjects and/or
weighted score; students characteristics namely, sex, cohort, entry scheme and
Nationality
Characteristics of enrollees [2005-2007 Cohorts]Students’ Characteristics Overall Between
N Percentage (%) N Percentage (%)
Sex
Female 1465 58.9 371 59.0
Male 1020 41.1 258 41.0
Total 2485 100.0 629 100.0
Year of Enrollment
2005 843 33.9 215 34.2
2006 887 35.7 225 35.8
2007 755 30.8 189 30.1
Total 2485 100.0 629 100.0
Nationality
Non-Ugandan 224 9.0 58 9.2
Ugandan 2261 91.0 571 90.8
Total 2485 100.0 629 100.0
Entry Scheme
Government 993 40.0 249 39.6
Private (Day) 1220 49.1 312 49.6
Private (Evening) 272 10.9 68 10.8
Total 2485 100.0 629 100.0
Weighted Score [on Admission]
N Mean Std. Dev Min Max
2023 51.8 3.04 44.1 56.5
Table 4: Summary statistics of the combined weighted score
The questionable aspect is the whether enrollees were competent for law school?
Note. The maximum possible score is 60
Mean weighted score suggests that the enrollees were highly competent in undertaking a bachelor’s study.
With regards to subjects done at A-Level
Academic Achievement
Year of Study Mean Std. Dev Min Max
First 3.01 0.33 2.20 4.00
Second 3.04 0.31 2.23 4.05
Third 3.12 0.30 2.23 4.13
Fourth 3.19 0.29 2.30 4.30
Table 1: Descriptive summary of academic achievement
A “Lower second” class of degree obtained by a considerable number of graduates on the program
Note. Estimates are based on CGPA
Predictors of academic achievement
Note. Assessment based on panel data approach – Random Effects Model (RE) using MLE
Independent Variables Model 1 Model 11
Coef. Std. Err p-value Coef. Std. Err p-valueNationality
Non Ugandans† 1.000 . . 1.000 . .Ugandans -0.067 0.040 0.096 -0.094 0.059 0.113
Year of Enrollment 2005 † 1.000 . . 1.000 . .
2006 -0.081 0.028 0.004 -0.099 0.033 0.0032007 -0.089 0.028 0.001 -0.158 0.031 0.000
Entry Scheme Government † 1.000 . . 1.000 . .
Private -evening -0.089 0.038 0.018 -0.117 0.055 0.033Private-day -0.055 0.042 0.198 -0.083 0.049 0.097
Sex Female † 1.000 . . 1.000 . .
Male 0.001 0.024 0.696 0.041 0.026 0.112Literature
N/A † 1.000 . . . . .A 0.101 0.036 0.005 . . .B 0.058 0.043 0.180 . . .C 0.030 0.059 0.608 . . .
D++ -0.017 0.095 0.860 . . .
Predictors of academic achievement [Cont.’]
Note. Assessment based on panel data approach – Random Effects Model (RE) using MLE
Divinity Coef. Std. Err p-value Coef. Std. Err p-valueN/A † 1.000 . . . . .
A 0.065 0.029 0.026 . . .B 0.029 0.036 0.409 . . .C -0.062 0.058 0.278 . . .
D++ -0.075 0.159 0.639 . . .Economics
N/A † 1.000 . . . . .A -0.122 0.094 0.196 . . .B -0.252 0.094 0.008 . . .C -0.330 0.095 0.001 . . .
D++ -0.317 0.098 0.001 . . .Geography
N/A † 1.000 . . . . .A 0.057 0.040 0.152 . . .B 0.042 0.039 0.276 . . .C 0.007 0.051 0.894 . . .
D++ -0.068 0.068 0.316 . . .History
N/A † 1.000 . . . . .A 0.214 0.085 0.012 . . .B 0.169 0.087 0.052 . . .C 0.140 0.097 0.150 . . .
D++ -0.092 0.211 0.665 . . .Weighted Score . . . 0.026 0.009 0.003constant 3.220 0.054 0.000 3.331 0.065 0.000
Significant variations in academic achievement (CGPA) were noted by enrollees
characteristics namely, nationality, enrollment cohort, and entry scheme (p < 0.05).
1. Students on government entry scheme had a higher CGPA compared to those on
the private evening arrangement
2. Enrollees in the 2006 and 2007 cohorts had a lower CGPA compared to those in
2005
Summary of the Findings [Student’s characteristics]
Could it be that students in the recent cohorts are increasingly taking on A-Level subjects they can easily pass rather than those they require to excel academically in law school ?
Significant variations in academic achievement were noted by performance in
disciplines namely literature, divinity, economics and history (p < 0.05).
1. Students who obtained grade A in literature had higher CGPA compared to their
counterparts who did not take the subject at A-Level
2. Students who obtained grade A in divinity had higher CGPA compared to their
counterparts who did not take the subject at A-Level
3. Students who obtained grades A and B in history had a higher CGPA compared
to those who did not take the subject at A-Level
4. Students who obtained grade B and below in economics had a lower CGPA
compared to those who did not take the subject at A-Level
Summary of the Findings [A – Level subjects]
Significant variation in academic achievement was noted by the weighted score
adopted on admission to law school(p < 0.01).
Summary of the Findings [Weighted score]
2.5
33
.54
4.5
Pa
rtia
l p
red
icto
r+re
sid
ua
l of cg
pa
45 50 55 60weight
Fractional Polynomial (1),adjusted for covariates
Line of best fit is not steep enough – weighted score weakly predicts CGPA
1. Using ANY best done of all A-Level subjects as “Essential subjects”
(MoES, 2005; 2006; 2007) to determine competence of
candidates to law school
2. The most competent candidates are the ones with the highest
weighted score adopted on admission to law school (MoES, 2005;
2006; 2007)
Implications
Questionable aspects
Performance in A-level subjects namely History, Divinity and Literature
predicts competencies required to excel academically in law school
Is the issue A-Level results or guidelines adopted on admission to law school ?
What about the pre-entry test to law school ?
Is performance in the test predicted by grades obtained in History, Divinity and Divinity ?
Students’ characteristics N Percentage (%)Gender
Female 253 31.0 Male 562 69.0 Total 815 100.0
Enrollment cohort 2012 461 56.6 2013 354 43.4
Total 815 100.0Nationality
Non-Ugandan 17 2.1 Ugandan 798 97.9
Total 815 100.0Entry scheme
Government 156 19.1 Private (Day) 511 62.7
Private (Evening) 148 18.2 Total 815 100.0
Prior qualification a A-Level 612 75.1
Bachelors’ degree 102 12.5Others 101 12.4
Total 815 100.0
Enrollees admitted on the basis of the test [2012-2013 Cohort]
Enrollees admitted on the basis of the test [Grades Obtained]A-Level subjects N Percentage (%)Literature
N/Aa 504 61.8A 99 12.2B 140 17.2C 63 7.7
D+ 9 1.1Total 815 100.0
Divinity N/A 378 46.4
A 130 16.0B 158 19.4C 99 12.2
D+ 50 6.1Total 815 100.0
Economics N/A 230 28.2
A 77 9.5B 148 18.2C 172 21.1
D+ 188 23.1Total 815 100.0
Note. N/A denotes student who did not do a subject at A-Level
Enrollees admitted on the basis of the test [Grades Obtained]
Geography N Percentage (%)N/A 610 74.9
A 51 6.3B 73 9.0C 59 7.2
D+ 22 2.7
Total 815 100.0
History N/A 238 29.2
A 112 13.7B 333 40.9C 112 13.7
D+ 20 2.5
Total 815 100.0
Contrary to enrollees admitted on the basis of their performance in A-Level, the
highest proportion of students obtained Grade B in the subjects.
Performance in the Test
Year of Study N Mean (95% CI) Min Max
First 379 2.65 (2.59 – 2.69) 0.60 3.90
Table 3: Descriptive summary of performance in the test
Figure is lower than FYCGPA of enrollees admitted using grades obtained in A-Level
Note. Estimates are based on First Year CGPA
Enrollment cohort N Mean (95% CI) Min Max
2012 446 59.2(58.6 - 59.8) 50 83
2013 348 67.1(66.4 - 67.8) 50 89
Academic AchievementTable 4: Descriptive summary of academic achievement [2012 Cohort]
Combination of admission test and Undergraduate GPA predicts first-year GPA in
law school (Anthony, Harris & Pashley 1999; Dalessandro, Stilwell, & Reese 2005;
Evans, 1984; Norton, Suto, & Reese, 2006; Wightman 1993)
Predictors of performance in the TestIndependent variables Model I a Model II b
Coef. Std. Err p-value Coef. Std. Err p-valueGender
Male† 0.00 . . 0.00 . .Female -1.09 0.204 0.000 -1.24 0.666 0.062
Enrollment cohort 2012† 0.00 . . . . .2013 7.72 0.206 0.000 . . .
Nationality Non Ugandans† 0.00 . . 0.00 . .
Ugandans 5.18 0.963 0.000 2.28 2.667 0.291Entry Scheme
Government† 0.00 . . 0.00 . .Private -evening -10.00 0.244 0.000 -10.36 0.859 0.000
Private-day -5.27 0.294 0.000 -4.59 1.044 0.000Prior qualification
A-Level 0.00 . . 0.00 . .
Bachelors’ degree 3.36 0.835 0.000 -1.03 2.040 0.612
Others -3.90 0.840 0.000 -5.49 2.035 0.007Literature
A† 0.00 . . 0.00 . .B -0.18 0.482 0.585 -0.24 1.129 0.826C 0.54 0.607 0.197 0.40 1.331 0.760
D++ -0.72 1.243 0.401 -1.63 2.246 0.467N/Ac -0.27 0.499 0.431 -0.54 1.225 0.657
Note. Assessment is based on a quantile (median) regression
Predictors of performance in the Test [Cont.’]Divinity Coef. Std. Err p-value Coef. Std. Err p-value
A† 0.00 . . 0.00 . .B 0.09 0.308 0.768 0.39 0.873 0.649C -0.36 0.365 0.321 -0.11 1.188 0.920
D++ 0.18 0.450 0.687 0.24 2.121 0.909N/A 0.90 0.329 0.006 0.54 0.997 0.582
Economics A† 0.00 . . 0.00 . .B -1.45 0.353 0.000 -2.11 1.144 0.066C -1.36 0.354 0.000 -2.01 1.171 0.086
D++ -1.27 0.368 0.001 -1.91 1.258 0.130N/A -1.18 0.690 0.088 -2.76 2.281 0.226
Geography A† 0.00 . . 0.00 . .B 0.18 0.462 0.694 0.95 1.769 0.589C 0.18 0.500 0.717 0.78 1.812 0.666
D++ 0.27 0.642 0.671 1.42 2.159 0.510N/A 1.09 0.418 0.009 0.75 1.616 0.640
History A† 0.00 . . 0.00 . .B -0.54 0.287 0.058 -1.65 1.284 0.199C -1.09 0.362 0.003 -2.05 1.415 0.148
D++ 1.00 0.630 0.113 0.49 1.962 0.799N/A -0.63 0.600 0.290 3.50 2.649 0.186
FYGPA . . . 0.70 0.522 0.180Constant 61.90 0.919 0.000 66.15 3.000 0.000
Performance in the test was HIGHER among:
1. Males (p < 0.01)
2. Ugandan nationals (p < 0.01)
3. Bachelor’s degree holders at enrollment compared to those with A-Level (p <
0.01)
Performance in the test was LOWER among:
4. Privately sponsored day and evening students compared to government
scheme (p < 0.01)
5. Students admitted on the basis of diploma and mature entry compared to those
with A-Level (p<0.01).
Summary of the Findings [Characteristics]
Performance in the test was HIGHER among:
1. Enrollees who did not do Literature compared to those with grade A (p < 0.05).
2. Enrollees who did not do Geography compared to those with grade A (p < 0.05).
3. Enrollees who obtained grade A in Economics compared to other grades (p <
0.01)
Performance in the Test was LOWER among:
4. Enrollees who obtained grade C in history compared to those with grade A (p <
0.01).
Summary of the Findings [A-Level Subjects]
Grades obtained in History, Divinity and Literature do not predict performance in the pre-entry test.
1. Relying heavily on the outcome of the test to admit students
under the government entry scheme (Makerere University, 2011)
2. Relying heavily on the outcome of the test to admit students to
law school
Implications
Questionable aspects
Aptitude tests predict inherent mental rather than educational or
academic ability as applied to law (LNAT Consortium, 2010; Law School
Admission Council, 2013; 2001)
Although candidates for the test are determined by a pooled index of attainment in A-Level (any subjects), admission to law school is mainly determined by the outcome of the test
1. The outcome of the admission test does not
predict competencies required to excel
academically in law school
2. Performance in A-Levels subjects namely, History,
Divinity and Literature predicts competencies
required to excel academically in law school
Conclusions
A pooled index of attainment in prior studies - using ANY subjects done at A-level - should not be used as a basis for determining candidates to sit the pre-entry test
1. To obtain the most academically competent candidates
to law school, admission requirements should focus on
performance in the A-Level subjects namely History,
Divinity and Literature
2. Requirements to sit the admission test should be based
on performance in the subjects rather than a pooled
index of attainment in prior studies i.e. A-Level
Recommendations
Problem is NOT UNEB results but the guidelines for admission to
law school