student assessment research findings zadar – tempus project reform of croatian legal education
TRANSCRIPT
Student assessmentResearch findings
Zadar – Tempus projectReform of Croatian legal education
Research methodologyAna Barić, LL.B. (in spe)
• basic framework • questionnaire with 80 questions • 17 exams with written and oral part of
the exam• 12 oral exams • 10 students conducting research • 76 professors included in the research • exam terms in January and February• around 300 student working hours
altogether
Provisional and incompleteresults arising from the questionnaire
Prof. Dr. Alan Uzelac
Elements of the research• Methods
– analysis of the annual statistics
(24.602 exams)– analysis of one exam term (February)
(3.760 applications,1.809 exams)
• Parts of evaluation–preparation for exams–applications for examination–techniques of examination
• oral/written
–applied methods of examination–publicity–results
COMPULSORY SUBJECTSCOMPULSORY SUBJECTS ELECTIVE SUBJECTSELECTIVE SUBJECTS
Roman private law ** Roman private law **
Legal psychology **Legal psychology **
General history of law and state ## General history of law and state ##
International maritime law ## International maritime law ##
Croatian legal history in European context ** Croatian legal history in European context **
English I ** English I **
General theory of law and state ** General theory of law and state **
English II ** English II **
Sociology ** Sociology **
German I ## German I ##
Political economy ** Political economy **
German II ## German II ##
Economic policy ** Economic policy **
Family law ** Family law **
Criminal procedure ** Criminal procedure **
Criminal law ** Criminal law **
Statistics and information science ** Statistics and information science **
Civil procedure ** Civil procedure **
Civil law ## Civil law ##
International law ## International law ##
Financial law and financial science ** Financial law and financial science **
Administrative science ** Administrative science **
Labour law and social security ## Labour law and social security ##
Commercial law ## Commercial law ##
Private international law ** Private international law **
Administrative law ** Administrative law **
Company law ## Company law ##
Maritime and general transport law## Maritime and general transport law##
Constitutional law ## Constitutional law ##
parts of the exam
only oral exam
written and oral exam
17**
12##
Preparation for the examsIs teaching useful at all?
Frequency Percent
Lectures are recommended and facilitate pass
4 19,0
Lectures are useful but literature is more important
17 81,0
Lectures aren’t useful, it is better to learn from the literature
0 0,0
Lectures don’t take place 0 0,0
Total 21 100,0
Frequency
Percent
Seminars are recommended and greatly facilitate success in exams
9 42,9
Seminars are useful but literature is more important
10 47,6
Seminars aren’t useful, it is better to learn from the literature
1 4,8
Seminars don’t take place 1 4,8
Total 21 100,0
relationship of the
seminar attending and
the success on the
exam
seminars are recomended and facilitate pass
seminars are usefull, but literature is more important
seminars aren't usefull, it's better to learn literature
seminars don't take place
relationship of the
lecture attending and
the success on the
exam
lectures are recomended and facilitate pass
lectures are usefull, but literature is more important
Seminarists’ privileges?
earlier term of the exam for seminarists seminarists can choose for exam professor who was on the seminar
grade from the seminar have an influence on the final grade
0
0,2
0,4
0,6
0,8
1
Mea
n
Techniques of examinationWriting v. Orality
Methods of written tests• Written test: 17 courses (v. 12)• Anonymous: none (at the term)• Multi-part (test and essay): 5 (yes) v.
12 (no)• Quitting without fall: 5 (no) v. 12 (yes)• Types of written tests:
– multiple choice (28);– filling-in the form (22);– essays (10)– longer questions (7)– shorter questions (20)
Methods of oral examination
professors spontaneous
questions
prepared questions by the
professors choice
prepared questions by the random choice
roughly chosen theme, without subquestions
roughly chosen theme, with
subquestions
precise question from the particular
theme
0
0,2
0,4
0,6
0,8
Me
an
Methods of oral examination
How do students take oral examination?
Frequency
Percent
One student alone in the room 7 9,7
One student, others listen 45 62,5
Oscillating between students who take exam
20 27,8
TOTAL 72 100,0
Relation of written and oral part• Common elements
– pass at written part necessary for taking of the oral examination
• minor exceptions– (new) statutory option for examination boards
– borderline cases
• The meaning of pass at written exam?– possible options
• only ticket to oral part• correction of grading• great influence on grading• automatic pass
– with the grade from the written part
– with only the passing grade
– differences even within the same course
Publicity of exams
notice with the name of those
who have passed
notice with the name of those
who have failed
notice with the number of points those who have
passed
notice with the number of points those who have
failed
notice with the grade of those
who have passed
0
0,2
0,4
0,6
0,8
1
Mea
n
oral notice of the results
notice of the results on the notice board
notice of the results on the Internet
0
0,1
0,2
0,3
0,4
0,5
0,6
Mea
nAnnouncing the written test results
Publicity of oral examination• generally, exams are made in public
– first layer: students who take the exam• group exams – 7 (individual); 65 (other
students who take the exam present)
– second layer: other interested students• 5 reported cases of no public at the oral
examination• reasons stated:
– request of the teacher (1)– request of the student (1)– other (alleged lack of space) - 3
student doesn't
know the grade
before
professor
enters it in his
index
allways
often
rare
never
professor
proclaim the
grad before
entering it in
student's index
allways
often
rare
never
professor
explains why he
gave particular
grade
allways
often
rare
never
student can
argue the grade
given by the
professor
allways
often
rare
never
it is offered to
the student not
to take the
grade and to
came again on
the oral exam
allways
often
rare
never
it is offered to the
student not to
take the grade
and to continue
answering the
questions
allways
often
rare
never
Results
1 2 3 4
year of studying
0,00
10,00
20,00
30,00
40,00
Mea
n
percentage of unsufficient grades during the year
percentage of sufficient grades during the year
percentage of good grades during the year
percentage of very good grades during the year
percentage of excellent grades during the year
First to last year of study – average success
Comparison: annual and monthly statistics
percentage of unsufficient grades during the year
percentage of unsufficient grades on the exam
percentage of sufficient grades during the year
percentage of sufficient grades on the exam
percentage of good grades during the year
percentage of good grades on the exam
percentage of very good grades during the year
percentage of very good grades on the exam
percentage of excellent grades during the year
percentage of excellent grades on the exam
0,00 10,00 20,00 30,00 40,00
Mean
Compatibility grades during the year to the grades on this term
compulsory elective
compulsory/elective subjects
0,00
10,00
20,00
30,00
40,00
50,00
Med
ian
percentage of unsufficient grades during the year
percentage of sufficient grades during the year
percentage of good grades during the year
percentage of very good grades during the year
percentage of excellent grades during the year
Grades: percentage of success
11
22
33
44
55
5544
3322
11
Success/methods
only oral exam written and oral exam
parts of the exam
0,00
10,00
20,00
30,00
40,00
Mea
n
percentage of unsufficient grades on the exam
percentage of sufficient grades on the exam
percentage of good grades on the exam
percentage of very good grades on the exam
percentage of excellent grades on the exam
Grades regarding if the exam is only oral or written and oral
Success in individual courses
Adm
inis
trativ
e la
wA
dm
inis
trativ
e s
cie
nce
Civ
il law
Civ
il pro
cedure
Com
merc
ial la
wC
om
pan
y la
wC
on
stitu
tiona
l law
Crim
inal la
wC
rimin
al p
rocedu
reC
roatia
n le
gal h
isto
ry in
Eu
rope
an c
onte
xt
Econo
mic
polic
yE
nglis
h I
Englis
h II
Fam
ily la
wF
inancia
l law
and fin
ancia
l scie
nce
Gen
era
l his
tory
of la
w a
nd s
tate
Gen
era
l theory
of la
w a
nd
sta
teG
erm
an
IG
erm
an
IIIn
tern
atio
nal la
wIn
tern
atio
nal m
aritim
e la
wLa
bour la
w a
nd s
ocia
l se
curity
Le
gal p
sycho
logy
Ma
ritime
and g
ene
ral tra
nspo
rt law
Politic
al e
con
om
yP
rivate
inte
rnatio
nal la
wR
om
an p
rivate
law
Socio
log
yS
tatis
tics a
nd in
form
atio
n s
cie
nce
subject
0,00
20,00
40,00
60,00
80,00
100,00
120,00
Me
an
percentage of unsufficient grades during the year
percentage of sufficient grades during the year
percentage of good grades during the year
percentage of very good grades during the year
percentage of excellent grades during the year
withdrawal from the
written exam after
exit, without fall
no
yes
Reliability of comparison – negative grades
Success in individual courses
Adm
inistra
tive la
wA
dm
inistra
tive scie
nce
Civil la
wC
ivil pro
cedure
Com
mercia
l law
Com
pany la
wC
onstitu
tional la
wC
rimin
al la
wC
rimin
al p
roce
dure
Cro
atia
n le
gal h
istory in
Euro
pean co
nte
xtE
conom
ic policy
English
IE
nglish
IIF
am
ily law
Fin
ancia
l law
and fin
ancia
l science
Genera
l histo
ry of la
w a
nd sta
teG
enera
l theory o
f law
and sta
teG
erm
an I
Germ
an II
Inte
rnatio
nal la
wIn
tern
atio
nal m
aritim
e la
wLabour la
w a
nd so
cial se
curity
Legal p
sycholo
gy
Maritim
e a
nd g
enera
l transp
ort la
wP
olitica
l eco
nom
yP
rivate
inte
rnatio
nal la
wR
om
an p
rivate
law
Socio
logy
Sta
tistics and in
form
atio
n scie
nce
subject
0,00
20,00
40,00
60,00
80,00
100,00
Mea
n
percentage of sufficient grades during the year
percentage of good grades during the year
percentage of very good grades during the year
percentage of excellent grades during the year
Diploma (final exam) – success story
percentage of excellent grades on the final exam
percentage of very good grades on the final exam
percentage of good grades on the final exam
percentage of sufficient grades on the final exam
0,00
10,00
20,00
30,00
40,00
50,00
60,00
Mea
n
Number of defended final papers
0
20
40
60
80
100
120
# diplomskih
OTPD
RPP
OPP
PHPD
SOC
PolEk
UST
K AZ
K PP
ObP
P inf
EkPol
GP
GPP
FPFZ
UZ
MP
UpP
TrgP
Pdru
PomOP
MPP
Rsoc
EJ P
Diploma (final exam) – success story
The final paper was defended before…
Frequency Percent
3 teachers, continously10 52,6
Mainly 3, sometimes 2.4 21,1
Rarely 3, more often 1 or 2.5 26,3
Total19 100,0
Output: number of graduated studentsYear Inscr. Grad.
91/92 561 203
92/93 811 272
93/94 1123 237
94/95 1231 242
95/96 1496 268
96/97 1440 294
97/98 1552 366
98/99 1427 354
99/00 1145 396
00/01 1133 426
01/02 956 453
02/03 1111 453
03/04 969 445
04/05 1006 497
05/06 1050 216
TOTAL
0
100
200
300
400
500
600
0
200
400
600
800
1000
1200
1400
1600
1800Graduated
Inscribed
Final Questionsfor
not so Final Solution
Prof. Dr. Hrvoje Sikirić
Open questions ???????????• Teaching (lectures, seminars) / Exams
• Number of exam terms per year, plan of terms – Avoiding overlapping of terms
• Distribution of exams into subsections
• Techniques of examination – writing v. orality
• Written exams – Standards – unanimity v. diversity– Anonymity– Structure – questions, case analysis, essays– Evaluation – grading, selectiveness– Duration
• Oral exams– Ensuring objectivity?– Publicity
• Diploma exam– Writing standards– Evaluation
Thank you for your attention!
1.1.1900 2.1.1900 3.1.1900 4.1.1900 5.1.1900 6.1.1900 7.1.1900 8.1.1900 9.1.1900 10.1.1900
Series1