experiences of the evaluator rndr. zuzana boukalovÁ crossczech, ccss, geo group
TRANSCRIPT
EXPERIENCES OF THE EXPERIENCES OF THE EVALUATOREVALUATOR
RNDr. Zuzana BOUKALOVÁRNDr. Zuzana BOUKALOVÁ
CROSSCZECH, CCSS, GEO CROSSCZECH, CCSS, GEO Group Group
Professional Professional ExperienceExperience Charles University, Faculty of Charles University, Faculty of
Science, Dept. of Hydrogeology, Science, Dept. of Hydrogeology, Civil EngineeringCivil Engineering
Geological Institute of Geological Institute of Czechoslovak Academy of ScienceCzechoslovak Academy of Science - - hydrogeologisthydrogeologist
University Politecnica de Valencia, University Politecnica de Valencia, SpainSpain – groundwater modelling – groundwater modelling
HR Wallingford Ltd. – water HR Wallingford Ltd. – water managementmanagement
Back in Prague from Back in Prague from 19961996 GEO Group, a.sGEO Group, a.s.:.: participation in the management, design and participation in the management, design and
realization of groundwater and non-saturated realization of groundwater and non-saturated zone pollution survey and remediation projects zone pollution survey and remediation projects
supervising expert in the team for government supervising expert in the team for government remediation programmes in groundwater and remediation programmes in groundwater and soil pollution. soil pollution.
PHARE, PHARE-CREDO, ISPA and SAPARD PHARE, PHARE-CREDO, ISPA and SAPARD project coordinatorproject coordinator
55- th- th and 6 and 6-th-th FP projects managing director FP projects managing director (CEGRMOMA, (CEGRMOMA, IRON CURTAIN, IRON CURTAIN, IMPACT, IMPACT, TRANSCAT, FLOODsite)TRANSCAT, FLOODsite)
From the year 2000From the year 2000
VZ GLS, a.s.VZ GLS, a.s. as the 5.FP project as the 5.FP projectss LOWRGREP and ENERGY FOREST managing LOWRGREP and ENERGY FOREST managing directordirector
CROSSCZECH a.s.CROSSCZECH a.s. : Head of the : Head of the Department, Hydrogeologist, ConsultantDepartment, Hydrogeologist, Consultant
Czech Center for Strategic Czech Center for Strategic StudiesStudies (CCSS), the member of the (CCSS), the member of the Managing BoardManaging Board, ARMONIA project WP leader , ARMONIA project WP leader
EDUCEUMEDUCEUM – the pool of 6 experts – the pool of 6 experts having ahaving a long lasting long lasting experience in different experience in different areas of European research and EU funding.areas of European research and EU funding.
Evaluator, Evaluator, Rapporteur
Marie Curie ActionsMarie Curie Actions (2003, 2004)(2003, 2004) eContent eContent (2003, 2005)(2003, 2005) Member of the Member of the „„Sounding Sounding
BoardBoard““ of the Commissioner Janez of the Commissioner Janez Potočnik, responsible for Science and Potočnik, responsible for Science and Research within the European Research within the European Commission (simplification of Framework Commission (simplification of Framework Programme procedures, FP7), 2005Programme procedures, FP7), 2005
Marie Curie Marie Curie ActionsActions Marie Curie Excellence Grants: Marie Curie Excellence Grants:
Excellence Grants, Awards, Chairs
(remote evaluation)
Marie Curie Conferences and Training Courses
(remote evaluation)
Marie Curie Marie Curie ActionsActions For the evaluation, the proposals are For the evaluation, the proposals are
divided to 8 areas (disciplines):divided to 8 areas (disciplines):
CHE:CHE: chemistry chemistry ECO:ECO: economics economics ENG:ENG: engineering engineering ENV:ENV: environment environment LIFE:LIFE: life sciences life sciences MAT:MAT: mathematics and information society mathematics and information society PHY:PHY: physics physics SOC:SOC: social sciences and humanities social sciences and humanities
Section Z:Section Z: multidisciplinar projects multidisciplinar projects
Marie Curie Marie Curie ActionsActions Year 2004:Year 2004: GenderGender MM FF TotaTota
ll
Heiger Heiger EducatioEducationn
3434 2525 5959
RESRES 1313 1212 2525
INDIND 44 33 77
OTHOTH 33 22 55
TotalTotal 5454 4242 9696
MC Actions – MC Actions – SCORE SCORE (evaluations by (evaluations by experts)experts) 0 – the proposal fails or cannot be 0 – the proposal fails or cannot be
judget against the criteria due to judget against the criteria due to missing or uncomplete informationmissing or uncomplete information
1 – poor1 – poor 2 – fair2 – fair 3 – good3 – good 4 – very good4 – very good 5 – excellent5 – excellent
MC actions MC actions EVALUTIONEVALUTION
Remote evaluationRemote evaluation (3 evalators -independent (3 evalators -independent experts- per 1 proposal); confidentiality and non-experts- per 1 proposal); confidentiality and non-conflict of interest: conflict of interest: individual report forms (IRF)
A rapporteur will be nominated to prepare the Consensus Report (CR) for a given proposal and obtain approval from the other evaluators
Panel meeting in BrusselPanel meeting in Brussel; f; for each proposal a consensus should be reached and a CR will be prepared and signed by the triplet. One CR form per proposal.
The experts will make a list of the proposals ranked The experts will make a list of the proposals ranked The experts’ conclusions are examined by the ECThe experts’ conclusions are examined by the EC Evaluation Summary ReportEvaluation Summary Report (ESR)(ESR) giving the giving the
opinion of the evaluators to the Coordinator of the opinion of the evaluators to the Coordinator of the proposal, on the basis of the Consensus Reportproposal, on the basis of the Consensus Report
eContent programeContent program
A multiannual Community A multiannual Community programme to simulate the programme to simulate the development and use of European development and use of European digital content on the global digital content on the global networks and to promote the networks and to promote the linguistic diversity in the linguistic diversity in the Information SocietyInformation Society
eContenplus programme (May 2005)eContenplus programme (May 2005) Call for experts!!!Call for experts!!!
eContent eContent evaluationevaluation LUXEMBOURGLUXEMBOURG 1 week evaluation1 week evaluation Score 1 – 5 Score 1 – 5 Triplet per 6 – 10 Triplet per 6 – 10
proposalsproposals Panel meetingsPanel meetings Evaluation and reports, all Evaluation and reports, all
finished in Luxembourgfinished in Luxembourg
Review ProcedureReview Procedure of the eContent of the eContent proposalsproposals
1=Unsatisfactory, 1=Unsatisfactory, 2=Poor, 2=Poor,
3=Satisfactory, 3=Satisfactory, 4=Good, 4=Good,
5=Excellent5=Excellent
Agenda of the reviewAgenda of the review
IntroductionIntroduction, , Objectives, context and purpose of a Objectives, context and purpose of a final reviewfinal review by PO by PO
Presentation: Aim and progress of workPresentation: Aim and progress of work (coordinator)(coordinator)
Individual Work-packages Presentation and Individual Work-packages Presentation and discussiondiscussion (WP leaders) (WP leaders)
Demonstration of the Web/CD-ROM product Demonstration of the Web/CD-ROM product (prototype)(prototype) – consortium – consortium
Questions and answer sessionQuestions and answer session PO + reviewers – evaluation PO + reviewers – evaluation PO close, recommendationsPO close, recommendations Successful completion Successful completion – Modify – Rejected– Modify – Rejected Review reports finished by experts from Review reports finished by experts from
home till 1 monthhome till 1 month
„„Sounding Sounding BoardBoard““ smaller actors in the context of smaller actors in the context of
simplification of Framework simplification of Framework ProgrammeProgramme procedures and procedures and implementation implementation
to incorporate the to incorporate the views of views of experienced stakeholdersexperienced stakeholders into the into the development of the Seventh development of the Seventh Framework Programme (FP7)Framework Programme (FP7)
3 – 4 meetings per year3 – 4 meetings per year 1-st meeting: 17.3. 20051-st meeting: 17.3. 2005
SB meetingSB meeting
Commission first ideas to achieve Commission first ideas to achieve substantial simplification of the FP7substantial simplification of the FP7
FP Action Plan on Rationalisation and FP Action Plan on Rationalisation and AccelerationAcceleration
I. Actions to simplify and accelerate I. Actions to simplify and accelerate II. Actions to improve quality and II. Actions to improve quality and
effectivenesseffectiveness
Actions to improve Actions to improve quality and quality and
effectivenesseffectiveness The established sets of The established sets of
uniform beings will be uniform beings will be the the basis for briefing all external basis for briefing all external evaluatorsevaluators to ensure to ensure consistency of approachconsistency of approach
DONE:DONE: standard briefing standard briefing available on internal websiteavailable on internal website
1
Actions to improve Actions to improve quality and quality and
effectivenesseffectiveness Evaluation Summary Reports Evaluation Summary Reports
sent to proposerssent to proposers MUSTMUST always be of high qualityalways be of high quality
EC will closely monitor the output EC will closely monitor the output from consensus groups and from consensus groups and panelspanels
DONE:DONE: introduced in evaluation introduced in evaluation workshops and briefingsworkshops and briefings
2
Actions to improve Actions to improve quality and quality and
effectivenesseffectiveness Quality of evaluators!!!Quality of evaluators!!! may include contacts by senior may include contacts by senior
officials with industrial umbrella officials with industrial umbrella groups, requesting them groups, requesting them to to nominate highly qualified nominate highly qualified individualsindividuals – experts – experts
ONGOINGONGOING
3
Actions to improve Actions to improve quality and quality and
effectivenesseffectiveness Further use of two stage proposal Further use of two stage proposal
submission and evaluation (for IP, NoE,…? submission and evaluation (for IP, NoE,…? STREP ?) more experts for evaluationsSTREP ?) more experts for evaluations
Guidelines - revisedGuidelines - revised: : first stagefirst stage is to be is to be as light as possible, based on limited number as light as possible, based on limited number of criteria with limited administrative dataof criteria with limited administrative data
The The second stagesecond stage will be based on will be based on FULL FULL SETSET of evaluation criteriaof evaluation criteria
DONEDONE
4
Actions to improve Actions to improve quality and quality and
effectivenesseffectiveness Rules on annual reviews of Rules on annual reviews of
the projects will be developedthe projects will be developed
ONGOINGONGOING
Review guidelines to be finalised Review guidelines to be finalised and put on CORDISand put on CORDIS
5
THANKS for your THANKS for your attention !attention !
Zuzana BoukalovaZuzana Boukalova
Contact:Contact:
[email protected]@crossczech.cz
[email protected]@geo-praha.cz