selecting and training students for the external review panels
DESCRIPTION
Selecting And Training Students For The External Review Panels: The Romanian Experience. Presented at 3rd Quality Assurance Forum in BudapestTRANSCRIPT
![Page 1: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/1.jpg)
Traian Brumă and Violeta Caragea
University of Bucharest
SELECTING AND TRAINING STUDENTS FOR SELECTING AND TRAINING STUDENTS FOR THE EXTERNAL REVIEW PANELS: THE EXTERNAL REVIEW PANELS: THE ROMANIAN EXPERIENCETHE ROMANIAN EXPERIENCE
![Page 2: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/2.jpg)
GREVA
20032003
![Page 3: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/3.jpg)
![Page 4: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/4.jpg)
SBU 2
![Page 5: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/5.jpg)
![Page 6: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/6.jpg)
Students trying to
CHANGE universities
![Page 7: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/7.jpg)
Students trying to
IMPROVE universities
![Page 8: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/8.jpg)
ARACIS starts in 2006
![Page 9: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/9.jpg)
![Page 10: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/10.jpg)
ANOSR was entrusted with selecting, training
and nominating students for evaluation
teams.
![Page 11: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/11.jpg)
Timeline
March 2007: ANOSR’s MethodologyApril and May 2007: training sessions for 64
students. May – June 2007: 11 pilot institutional
evaluations involving 13 student evaluators. August – October 2007: evaluating the impact
of student participation in the review teams. February – March 2008: Rethinking the
student evaluators training.April 2008: 27 more students were trained June – July 2008: 6 more institutional
evaluations
![Page 12: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/12.jpg)
Our aim is to ...
![Page 13: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/13.jpg)
Continue the
European conversation
![Page 14: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/14.jpg)
Selection
![Page 15: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/15.jpg)
analyzing the self-evaluation report and
other documents,
preparing the site visit,
participating to different visit activities
(interviews, visiting academic infrastructure
etc.),
writing the external evaluation report.
Job description:
![Page 16: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/16.jpg)
Job description -> profile:
(1) Knowledge,
(2) Skills,
(3) Personal values and,
(4) Motivation.
![Page 17: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/17.jpg)
Profile -> selection intruments:
A letter of intent,
An essay,
A curriculum vitae.
+ An assessment grid (scoring1 to 100)
![Page 18: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/18.jpg)
main criterion - the score (more than
50pt) ,
participation in training – mandatory,
additional criteria were (eg: to evaluate an
institution with a similar profile)
Nomination:
![Page 19: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/19.jpg)
exclude the criteria related to personal
values;essay replaced by an interview taken after
the training; a peer evaluation exercise was introduced.
Changes:
![Page 20: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/20.jpg)
Training
![Page 21: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/21.jpg)
Learning outcomes for 2007: Understanding the global context of higher
education and the role of QA; Understanding the role of QA in the Romanian
context and the challenges ahead; Operating with basic concepts related to QA; Understanding the emerging principles of the
QA process in Romania;Being able to work with the basic documents
regarding QA; Understanding the students' role in
institutional evaluations reviewers’ team.
![Page 22: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/22.jpg)
Training delivery
Facilitated by two students from
ANOSR.
Seven presentations delivered by
guest speakers;
![Page 23: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/23.jpg)
Participants’ feedback was very positive.
Some improving suggestions:
better time management,
more practical activities.
Feedback
![Page 24: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/24.jpg)
interviews with student evaluators,
evaluation of students' reports and
the report of the independent audit of the
institutional evaluations pilot phase.
Assessing the impact of the training program:
![Page 25: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/25.jpg)
students raised a significant number of
(negative) issues ;
students reports contained specific
recommendations, grounded in the reality
of the university life;
their involvement was seen as mature,
pragmatic and competent.
Results (+):
![Page 26: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/26.jpg)
a need for improvements regarding the site visit organization;
there is little awareness about the students’ role;students reports didn't covered enough the issues for
which their experience and perspective is valuable;in some cases students’ interventions during the site
visit contained an imperative tone, lack of strong arguments, tendentious generalizations of situations encountered in one department for the whole university;
a guide for writing the report would have been necessary.
Results (-):
![Page 27: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/27.jpg)
Rethinking the Training:
the input consisting of: results of the feedback meeting of the 2007 training coordinating team;
student interviews;an independent evaluation report of the pilot phase.
![Page 28: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/28.jpg)
Conclusions
The training for students was
insufficient;
The major gap was the lack of
evaluation skills.
![Page 29: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/29.jpg)
Changes in training:
Learning outcomes focus shifted to practical evaluation skills
Training delivery: roleplays, simulations.
![Page 30: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/30.jpg)
To do list (1)
Better promoting QA to students and the possibility to be involved;
Connecting the training program for the external reviewers with the programs for the students involved in internal QA;
Joint training for students and other experts;
Developing additional modules to the initial training program;
Building an online platform for the student evaluators community;
![Page 31: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/31.jpg)
To do list (2)
Blended learning;Further improving the training methods;Improving the documentation (describing the
role of students in review teams, developing a handbook for student evaluators);
Inviting foreign students evaluators as guest speakers;
Finding a dynamic balance between refreshing the pool and building on the experience;
Recognition of ECTS credits awarded by all Romanian universities.
![Page 32: Selecting And Training Students For The External Review Panels](https://reader036.vdocument.in/reader036/viewer/2022070317/5569bfded8b42a5c5d8b4736/html5/thumbnails/32.jpg)
QUESTIONS
Does it worth to extend the student involvement to the study programmes evaluations? Do we have the capacity to do it?
How should we design the "pool" / "community"/ "system" in order to have it refreshed every year and to build on the expertise and experience that is being generated every year?
How many reviews a student should participate in? How can we improve the integration of the student
in the reviewers’ team? Could we have a paid secretariat (project manager
+ 1 staff); should it be run by ARACIS or by the NUS?