COURSE QUALITY ASSURANCE

AND EVALUATION

EMLE AS A GOOD PRACTICE

EXAMPLE

EMAP 2

TRAINING SEMINAR FOR FUTURE

EMMC CONSORTIA

WARSAW (POLAND)

28 JANUARY 2011

22

PRESENTATION

BY

WICHER SCHREUDERS

• ERASMUS UNIVERSITY ROTTERDAM

• ROTTERDAM INSTITUTE OF LAW AND ECONOMICS (RILE)

• ERASMUS MUNDUS ASSISTANT COORDINATOR EMLE PROGRAMME

3

OVERVIEW OF THIS

PRESENTATION

• WHAT IS THE EMMC IN LAW &

ECONOMICS (EMLE)?

• QUALITY ASSURANCE – IN GENERAL

• QUALITY ASSURANCE – IN EMLE

• CHALLENGES

• RECOMMENDATIONS

3

44

THE EMLE HISTORY

EMLE STARTED IN 1990

EMLE WAS SELECTED FOR / PARTICIPATED

IN:

• 2003: THE EUA ‘TOP JOINT MASTER’

PROJECT

• 2004: ERASMUS MUNDUS (AT THE START)

• 2005/2006: ENQA’S TEEP II PROJECT

• 2009: ERASMUS MUNDUS II (START 2010)

55

THE EMLE PROGRAMME

• A ONE-YEAR MASTERS COURSE (60 ECTS)

• 85-105 PARTICIPANTS EACH YEAR

• TOPIC: THE ECONOMIC ANALYSIS OF LAW

• EM COORDINATOR: ROTTERDAM

• PARTNERS: FROM EU AND NON-EU

COUNTRIES

6

COURSE PROGRAMME

• 1ST TERM (OCTOBER - DECEMBER):

4 COURSES (20 ECTS)

• 2ND TERM (JANUARY – MARCH):

4 COURSES (20 ECTS)

• 3RD TERM (APRIL – AUGUST):

2 COURSES + THESIS (20 ECTS)

= 10 COURSES + THESIS (60 ECTS)

6

7

EMLE STRUCTURE

PARTNERS PER TERM

1 TRACK

(1990-1993)

2 TRACKS

(1993-2002)

3 TRACKS

(2002-….)

1st TERM 1 2 3

2nd TERM 1 2 3

3rd TERM (SMALLER

GROUPS)

2 5 - 8 7

MAXIMUM NUMBER

OF STUDENTS

35 70 105

7

888

EMLE PARTNERS 2010 (EM II)

1. ROTTERDAM (THE NETHERLANDS)

2. GHENT (BELGIUM)

3. AIX/MARSEILLE (FRANCE)

4. HAMBURG (GERMANY)

5. VIENNA (AUSTRIA)

6. BOLOGNA (ITALY)

7. HAIFA (ISRAEL)

8. WARSAW (POLAND)

9. MUMBAI (INDIA)

9

EMLE PROGRAMME FROM

2010/2011 ONWARDS (EM II)

1st

TERM

BOLOGNA ROTTER-

DAM

HAMBURG

2nd

TERM

BOLOGNA GHENT HAMBURG

3rd

TERM

AIX/MARSEILLE, HAIFA, HAMBURG,

MUMBAI, ROTTERDAM, VIENNA,

WARSAW

9

10

EMLE’S NUMBER OF

STUDENT APPLICATIONS

NON-EU EU TOTAL

2005/2006 138 116 254

2006/2007 265 114 379

2007/2008 334 114 448

2008/2009 375 115 490

2009/2010 330 156 486

2010/2011 374 236 610

2011/2012 431 N/A N/A

10

11

COMPLICATED STRUCTURE

• GIVEN THE NUMBER OF PARTICIPATING

STUDENTS (85-105)

• GIVEN THE THREE TERMS

• GIVEN THE THREE TRACKS

URGENTLY NEEDED:

- HORIZONTAL INTEGRATION

- VERTICAL INTEGRATION

- QUALITY ASSURANCE / EVALUATIONS

11

12

HORIZONTAL INTEGRATION

• IN CASE THERE ARE PARALLEL CLASSES

AT DIFFERENT PARTNERS AT THE SAME

TIME: HARMONIZATION IS NEEDED FOR

HORIZONTAL INTEGRATION

• THE PARTICIPANTS MUST HAVE AN

(MORE OR LESS) IDENTICAL PROGRAMME,

REGARDLESS OF TAKING THE COURES AT

PARTNER A OR B

12

13

VERTICAL INTEGRATION

• THIS REGARDS TEACHING IN SUCCESSIVE

TERMS

• TEACHERS IN THE 1ST TERM MUST KNOW

WHAT IS NEEDED FOR THE 2ND AND 3RD

TERM (EVEN AT ANOTHER PARTNER!)

• TEACHERS IN THE 2ND AND 3RD TERM

MUST KNOW WHAT IS STUDIED IN

EARLIER TERMS

• HARMONIZATION IS NEEDED FOR

VERTICAL INTEGRATION13

14

QUALITY ASSURANCE

IN GENERAL

• EXTERNAL QA

• INTERNAL QA

• RELEVANT QA ISSUES

1515

QUALITY ASSURANCE:

EXTERNAL

• EXTERNAL QUALITY ASSURANCE:

- ACCREDITIATIONS AT NATIONAL LEVEL

(IN ALL PARTNER COUNTRIES, WITH DIFFERENT REQUIREMENTS)

- AT THE ERASMUS MUNDUS LEVEL

(SELECTION; TWO REPORTS PER YEAR; SPECIFIC REQUIREMENTS AS WELL)

16

QUALITY ASSURANCE:

INTERNAL

• INTERNAL QUALITY ASSURANCE:

- BY THE PARTNERS

- AT THE CONSORTIUM’S CENTRAL LEVEL

1717

QUALITY ISSUES (1)

• IS THE MASTER REALLY SCIENTIFIC?

• INTERACTION BETWEEN TEACHING AND

RESEARCH

• CONFORMITY WITH INTERNATIONAL

STANDARDS

• LABOR MARKET PERSPECTIVES

1818

QUALITY ISSUES (2)

• CONTENTS OF THE COURSES

• ORGANIZATION OF THE LECTURES

• STUDY WORKLOAD

• ADMISSION CRITERIA

• EXAMINATION CRITERIA

1919

QUALITY ISSUES (3)

• QUALITY OF TEACHING STAFF

• ADMINISTRATIVE SUPPORT

(VISA, ACCOMMODATION, ETC.)

• INTERNAL QUALITY ASSURANCE SYSTEM

• FINANCING AND CONTINUITY

2020

QUALITY ISSUES (4)

ALL THIS:

• IN SEVERAL STANDARDS, GUIDELINES

AND GOOD PRACTICES

• IN GENERAL (UNESCO/OECD) OR MORE

SPECIFIC (ESG, BY ENQA)

• VISIT: WWW.EMQA.EU

HOW TO ARRANGE THIS IN PRACTICE?

21

QUALITY ASSURANCE

IN EMLE

TO AVOID DIFFERENCES IN QUALITY

DEPENDING ON THE MOBILITY TRACK:

• HORIZONTAL / VERTICAL INTEGRATION

• USE IDENTICAL QUESTIONNAIRES FOR

ALL COURSES AT ALL PARTNER

UNIVERSITIES

• COMPARE AND USE THE OVERVIEW OF

ALL THESE RESULTS

21

22

INTERNAL QUALITY

ASSURANCE IN EMLE (1)

• CONS. AGREEMENT / REGULATIONS

• STUDENT AGREEMENT

• BOARD MEETINGS (OCT, FEBR) /

MEETINGS OF TEACHERS (JUNE)

• QUALITY ASSURANCE COMMITTEE /

CHAIR = ‘QA OFFICER’ (2 MEMBERS FOR 4

MAIN PARTNERS)

• INTERACTION WITH STUDENTS (ELECTED

REPRESENTATIVES) AND ALUMNI22

2323

INTERNAL QUALITY

ASSURANCE IN EMLE (2)

• QUESTIONNAIRES FOR ALL COURSES (AT

THE END OF THE EXAM)

• MEETINGS WITH STUDENTS BY QA

OFFICER (= CHAIR QA COMMITTEE)

• MEETINGS WITH STUDENTS BY LOCAL

(ASSISTANT) COORDINATOR

• OMBUDSMAN (= ONE OF THE LOCAL

COORDINATORS)

2424

QUALITY ASSURANCE:

BEFORE, DURING AND AFTER

1. BEFORE ARRIVAL OF STUDENTS: WHAT

WE OFFER, WHAT WE WANT, SELECTION

PROCEDURE

2. DURING THE ACADEMIC YEAR:

EVALUATIONS PER COURSE

3. AFTER GRADUATION: CONTACTS WITH

GRADUATES AND ALUMNI

2525

1. BEFORE ARRIVAL OF

THE STUDENTS

• CLEAR INFORMATION TO ATTRACT QUALIFIED APPLICANTS

• CLEAR SELECTION CRITERIA

• CLEAR SELECTION PROCEDURE

• SELECTION AND RANKING BY THE JOINT SELECTION COMMITTEE (LOCAL COORDINATORS)

2626

2. DURING THE YEAR (1)

EVALUATIONS PER COURSE

• QUESTIONS ABOUT:

- TEACHING

- CONTENTS OF THE COURSE

- EXAM

- OVERALL OPINION

- WORKLOAD PER WEEK

- OPEN QUESTION (REMARKS, SUGGESTIONS)

2727

2. DURING THE YEAR (2)

• REPORT BY QA OFFICER (ON EVALUATION

FORMS AND MEETINGS WITH STUDENTS)

• TO BE DISCUSSED IN BOARD MEETINGS

AND MEETINGS OF TEACHERS

• COMPARE RESULTS:

- PER COURSE: WITH EARLIER YEARS

- PER COURSE: BETWEEN PARTNERS

- PER TERM: BETWEEN PARTNERS

2828

COURSE TORT LAW (ROTTERDAM)

COMPARISON BETWEEN YEARS

Score Evolution

OVERALL OPINION

3,50

4,50

Understanding Challenging Overall

1st Term

(2005-

2006)

1st Term

(2006-

2007)

2929

COURSE PUBLIC LAW

COMPARISON PARTNERS

Average Course Evaluation

OVERALL OPINION

0,00

0,50

1,00

1,50

2,00

2,50

3,00

3,50

4,00

4,50

5,00

Understanding Challenging Overall

Prof. Pacces

(Rotterdam)

Prof. Parisi

(Bologna)

Prof.

Fiorentini

(Bologna)

Prof. Curti

(Hamburg)

3030

OVERALL AVERAGES 1st TERM

PER PARTNER AND OVERALL

Average Course Evaluation

OVERALL OPINION

3,35

3,40

3,45

3,50

3,55

3,60

3,65

3,70

3,75

3,80

3,85

Understanding Challenging Overall

Average

Rotterdam

Average

Bologna

Average

Hamburg

Total Average

3131

2. DURING THE YEAR (3)

• DISCUSS HIGH SCORES AS ‘BEST

PRACTICE’

• DISCUSS LOW SCORES, SEARCHING FOR

IMPROVEMENTS

• BRING TOGETHER TEACHERS PER COURSE

3232

3. AFTER GRADUATION

EVALUATIONS BY:

• MAINTAINING RELATIONSHIPS WITH

ALUMNI AND THE COMPANIES /

INSTITUTIONS THEY WORK FOR (LABOR

MARKET)

• RELATIONS WITH EMLE STAKEHOLDERS

AS ASSOCIATED MEMBERS

3333

CONCLUSIONS

REGARDING EMLE

• 1. SELECTION: SEEMS TO BE OK (SUCCESS RATE IN EMLE = APPROX. 95%)

• 2. EVALUATIONS PER COURSE: COMPARISON AND COMPETITION BETWEEN TEACHING CENTERS IMPROVES QUALITY

• 3. EVALUATIONS AFTER GRADUATION (GRADUATES / ALUMNI)

3434

CHALLENGES (1)

IN CONSORTIUM MEETINGS:

• BE OPEN (TRANSPARENT) IN GIVINGCOMMENTS TO YOUR COLLEAGUES, FOR EXAMPLE REGARDING THE METHODS OF TEACHING, THE CONTENTS OF THE COURSE, THE LITERATURE WHICH IS BEING USED

OTHERWISE YOU ARE RESTRICTED IN IMPROVING THE PROGRAMME

3535

CHALLENGES (2)

IN CONSORTIUM MEETINGS:

• BE OPEN IN RECEIVING ALL COMMENTS AND SUGGESTIONS: BY COLLEAGUES, ALUMNI AND STUDENTS, AND REGARDING THE RESULTS OF THE EVALUATION QUESTIONNAIRES

ONLY IN THIS WAY THE PROGRAMME CAN BE IMPROVED

3636

RECOMMENDATIONS (1)

• THE INTERNAL QA SYSTEM SHOULD BE

STATED CLEARLY IN THE CONSORTIUM

AGREEMENT

• MAKE USE OF EVALUATION FORMS (PER

COURSE, PER TERM)

• GIVE ATTENTION TO THE COMMENTS

MADE BY THE STUDENT

REPRESENTATIVES

37

RECOMMENDATIONS (2)

• CONTACT ALUMNI AND ASSOCIATED

MEMBERS REGULARLY, ASK FOR THEIR

COMMENTS / SUGGESTIONS

• BE OPEN TO ALL COMMENTS /

SUGGESTIONS

3838

CONCLUSION

QUALITY ASSURANCE

=

PERMANENT EVALUATIONS

3939

INFO & CONTACT

• WEBSITES:

WWW.EMLE.ORG

WWW.RILE.NL

• EMAIL: SCHREUDERS@FRG.EUR.NL