models of evaluation for research proposals in turkey prof. dr. omer cebeci vice president –...
TRANSCRIPT
![Page 1: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/1.jpg)
Models of Evaluation For Research
Proposals in Turkey
Prof. Dr. Omer CEBECIVice President – Funding, TUBITAK
(The Scientific and Technical Research Council of Turkey)
&S. GENC, A. KERC, H. KARATAS, A. FEYZIOGLU,
F. COSKUN, O. OZPEYNIRCI, B. DIKMEN,G. KOZANOGLU, H. GULER & O. KARA
![Page 2: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/2.jpg)
SUMMARY
• Research funding in Turkey, TUBITAK & recent boost
• Three-dimensional evalution used by TUBITAK for the evaluation/selection of research project proposals grouped under three categories:
• (1) curiosity driven academic research, and• (2) customer driven applied research,• both in universities and research institutions, and• (3) technological and innovation driven research
conducted by the private industry, plus• (4) research equipment & infrastructure.
• Details of the “Phrase-anchored rating scale”
![Page 3: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/3.jpg)
Actors of Turkish National Science and Technology System (policy makers)
BTYK- Supreme Council of Science and Technology
ME-Ministry of Education
MIT-Ministry of Industry and Trade
TUBITAK-The Scientific and Technical Research Council of Turkey
HEC-Higher Education Council (YÖK)The Turkish Academy of Sciences (TÜBA)
KOSGEB- Small and Medium Sized Industry and
Development Organizations
TAEK- The Turkish Atomic Energy Commission
TOBB-Union of Chambers and Commodity Exchange of Turkey
DTM-Foreign Trade and Treasury
TTGV- Technology Development Foundation of Turkey
DPT- State Planning Organization
TÜRKAK- Accreditation Board
![Page 4: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/4.jpg)
TUBITAK National Research Centers and Institutes
Marmara Research Center (MAM)Institute of Energy
Chemistry and Environment InstituteFood Institute
Materials Institute Information Technologies Research InstituteEarth and Marine Sciences Research InstituteInformation Technologies and Electronics
Research Institute (BİLTEN)National Electronics and Criptology
Research Institute (UEKAE)Defence Industry Research and Development
Institute(SAGE)Basic Science Research Institute
National Academic Network CenterGenetic Engineering
and Biotechnology Research InstituteNational Metrology Institute
Turkish Industry Management InstituteNational Observatory
Actors of Turkish National Science and Technology System (science and research performers)
Turkish State
Universities
Turkish Industry
OtherNational Research
Centers and Institutes
Turkish Private
Universitiesand Research
Centers
![Page 5: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/5.jpg)
President of Republic
Prime MinisterBTYK
YÖK TÜBİTAK
STB
BTYK- Supreme Council of Science and TechnologySTB- Minister of Industry and TradeMEB- Ministry of National EducationTÜBİTAK- The Scientific and Technological Research Council of TurkeyYÖK- Higher Education Council (YÖK)TÜBA- The Turkish Academy of Sciences (TÜBA)DPT- State Planning OrganizationTÜRKAK- Accreditation Board
MEB
Universities DPT TÜBA
TÜRKAK
TPEKOSGEB
TSE
TTGV
R&D InstitutionsTPE- Turkish Patent InstituteTSE- Turkish Standards InstitutionDIE- State Institute of Statistics DTM- Undersecretariat of the Prime Minister for Foreign Trade TOBB-Union of Chambers and Commodity Exchange of TurkeyKOSGEB- Small and Medium Industry Development OrganizationsTTGV- Technology Development Foundation of Turkey
DTM
TOBB
DIE
Turkish National Science, Technology and Innovation System
![Page 6: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/6.jpg)
National S&T Indicators
EU-15 Values (2000-2001):Number of triadic patents per million population:36
Number of papers per million population:822
GERD per capita population:467. 6 (PPP $)GERD as a percentage of GDP:1,89
Total R&D personnel per 1000 employment: 10.4
0
0,5
1
1,5
2
2,5Number of triadic patents per million population
Number of papers per millionpopulation
GERD per capita population
GERD as a percentage of GDP
Total R&D personnel per1000 employment
EU-15
Japan
US
Germany
![Page 7: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/7.jpg)
0,5
1
GERD as a percentage of GDP
Spain
EU-15
Hungary
Turkey
National S&T Indicators
Total R&D personnel per1000 employment
Number of triadic patents per million population
Number of papers per millionpopulation
GERD per capita population
![Page 8: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/8.jpg)
Scientific Publications
0,460,52
0,57
0,64
0,86
0,94
1,11
0,40
0,50
0,60
0,70
0,80
0,90
1,00
1,10
1,20
1998 1999 2000 2001 2002 2003 2004
Year
Sh
are
of
Tu
rkey
in W
orl
d P
ub
licat
ion
s (%
)
![Page 9: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/9.jpg)
Knowledge as a Tool
Knowledge is a tool that can be used for a variety of social objectives, including:
Meeting Basic Human Needs Increasing Safety & Security Improving the Quality of LifeEconomic Growth and Development
![Page 10: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/10.jpg)
100.000 $Communication Satellite
4.000.000 $Microprocessor (Si) Chip
10.000 $Combat aircraft
3.000 $Fiberoptical Cable
100 – 1.000 $Passenger Aircraft
10 – 100 $Automobile
1,5 $Aluminium
50 centSteel
5 centCement
1 centConcrete
Knowledge - R&D Cost per Kg
![Page 11: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/11.jpg)
8
10
15
11
10
8 8 7
5
8
10 9
11 10
9 8
7
8 8 9
12 13 13
9 8 8 8 8 9
10 10 10
16 17
46
0
5
10
15
20
25
30
35
40
45
50
1971 1973 1975 1977 1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001 2003 2005
ON
BİN
DE
Share of TUBITAK inTR Fiscal Budget
![Page 12: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/12.jpg)
Research Proposals
*from universities and research institutions:1) curiosity driven academic research2) customer driven applied research
*from private industry:3) technological & innovation driven
research
![Page 13: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/13.jpg)
TUBITAK Research Project Proposals
Science and Technology Project Proposals
0
500
1000
1500
2000
2500
3000
1999 2000 2001 2002 2003 2004
Years
Nu
mb
er o
f P
rop
osa
l
Career
International
Sector
Units
Infrastructure
Research
![Page 14: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/14.jpg)
Running Research Projects
264
430
587694 750
885 889 929 899833 837
982
1212 11991353
2600
0
500
1000
1500
2000
2500
3000
1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005
![Page 15: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/15.jpg)
EVALUATION & MEASUREMENT
![Page 16: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/16.jpg)
TARİHTEN BUGÜNETARİHTEN BUGÜNESAĞLIK ve SAĞLIK ve ÖLÇMEÖLÇME
• Ateş ve nabız – 1625(Santario ve Galileo)
• Torricelli - 1643• Tansiyon - 1733 (Hales)
• 1896 - Riva-Rocci• Korotkov - 1905• Ateş - nabız - tansiyon ölçüm ve
kayıtları ile hasta izleme - 1920
![Page 17: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/17.jpg)
ÖLÇME ve DEĞERLENDİRMEÖLÇME ve DEĞERLENDİRME“ Ölçemediğinizi anlıyamazsınız ! ”
Lord Kelvin (1824 - 1907)
(If you can’t measure it -you don’t understand it)
“Her hesaba katılması gerekeninsayılır olması gerekmez ! “
Albert Einstein (1879 - 1955)(Not everything that can be counted counts &
not everything that counts can be counted)
![Page 18: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/18.jpg)
Çıktı mı ? Sonuç mu ? Etki mi?
Çıktıların sonucu olarak gerçekleşen ETKİ !
![Page 19: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/19.jpg)
OUTPUT – OUTCOME – IMPACT
IMPACTS achieved as consequences of the
OUTCOMES resulting from the
realisation of the OUTPUTS
![Page 20: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/20.jpg)
OUTPUT – OUTCOME – IMPACT
IMPACTS achieved as consequences of the
OUTCOMES resulting from the
realisation of the OUTPUTS
( My eye jelly project )
![Page 21: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/21.jpg)
Intended / Expected / Desired IMPACT :
. . . in full control of my mandate . . .
Unintended / Unexpected / Unwanted
BUT ACTUAL REAL IMPACT :
. . . under strict control of my wife ! . .
![Page 22: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/22.jpg)
Evaluation Criteria
three-dimensionalevaluation criteria
developed in collaboration withresearchers & reviewers
![Page 23: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/23.jpg)
1) Curiosity DrivenAcademic Research
- the three dimensions -
1. intellectual/scientific/professional merit
2. expected impact of the anticipated outcome
3. achievability of the research with the proposed research team, equipment/facilities and methods
![Page 24: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/24.jpg)
1. research and development merit
2. expected impact of the anticipated outcome
3. achievability of the research with the proposed research team, equipment/facilities and methods
2) Customer Driven Applied Research
- the three dimensions -
![Page 25: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/25.jpg)
1. technological level of the research
2. innovative level of the product/outcome
3. feasibility of the process
3) Technological & Innovation
Driven Research- the three dimensions -
![Page 26: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/26.jpg)
Evaluation Process
all three-criteria are given equal weights
Phrase-Anchored Rating Scaleversus
Likert Scale
sub-criteria phrases describing
*very competitive *competitive *not-competitive
features of the proposal
![Page 27: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/27.jpg)
Very competitive
*scientifically and professionally outstanding & very well justified project
&*points to an opportunity for a major
contribution to the advancement of the knowledge and/or to the resolution of a problem of practical importance
![Page 28: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/28.jpg)
Competitive
*scientifically and professionally competent and justified proposal which will make a contribution to the advancement of knowledge &/or the resolution of a problem of practical value
& therefore*support is suggested if funds are
available
![Page 29: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/29.jpg)
Not competitive
*work routine in character*scientifically and professionally
unsatisfactory and poorly organized
![Page 30: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/30.jpg)
Evaluation Process
evaluate 10-15 proposals by referring to the
sub-criteria phrases
a meeting for a final verdicta meeting for a final verdict
Panels
5-8 Individual Reviewers
![Page 31: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/31.jpg)
Curiosity / Merit = V. Comp.
• Outcomes have high potentials for publication in journals or books listed in international indexes
• The originality of the work has been supported by extensive and critical literature survey
• Hypothesis for evaluating the research topic is very well defined
• Explanation and analysis of the expected outcomes reveal the superiority of the work in comparison to the existing science and technology.
![Page 32: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/32.jpg)
Curiosity / Merit = Not comp.
• Scientific consistency and the rationale of the research are not clearly explained.
• A clear scientific / technological question is not put forward.
• Project is more like an investigation / data collection / routine work.
• References provided include similar studies & literature survey does not point out the basis / importance of the project.
• The research is based on unreliable data and hypothesis.
![Page 33: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/33.jpg)
Curiosity / Impact = V. Comp.
• Very important in terms of sustainable development of the country /very likely to be implemented to find solutions for the problems of society
• Very likely to be employed in different scientific & technological fields
• Very likely to generate new projects• Commercialization potential of the
outcomes is very high• The project is supported by international,
national or industrial sources.
![Page 34: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/34.jpg)
Curiosity / Impact = Not comp.
• Potential for adding value to science and technology is low
• Subject of the project is not among the priorities of the country
• Not likely to result in intellectual property worth-protecting.
![Page 35: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/35.jpg)
Curiosity / Achiev. = V. Comp.Project Team• The team is experienced in national /
international projects related with same/similar subject
• They have experience as advisors /authors / referees / editors / book authors
• The can allocate enough time for the project• Competencies / responsibilities / roles of the
team members are well defined and adequate
• End users of the project outcomes are also members of the team.
![Page 36: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/36.jpg)
Curiosity / Achiev. = Not Comp.Project Team• The team is not experienced in conducting
projects of this size• The knowledge and awareness of the team
is not sufficient• They do not have important publications in
the subject of the project• Some of the team members are irrelevant
for the project• Essential competencies are lacking
![Page 37: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/37.jpg)
Curiosity / Achieve. = V. Comp.
Infrastructure• Infrastructure of the institution is very
adequate for the project• Additional equipment requested within the
scope of the project is very compatible with the existing infrastructure and the project
• Existing sources / equipment are used rather than purchasing new sources
• Requested equipment can also be used in other / future projects
![Page 38: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/38.jpg)
Curiosity / Achieve. = Not Comp.
Infrastructure• Infrastructure of the institution is not
adequate for the project, unless supported with major equipment
• Equipment requested within the scope of the project is not compatible with the existing infrastructure and the project
![Page 39: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/39.jpg)
Curiosity / Achieve. = V. Comp.
Methodology• Approach / methodology are very well
designed to reach the target• Methodology is correct and well-defined;
and standard methods and literature are cited
• Preliminary experiments have been conducted to rationalize the hypothesis
• Alternatives (plan B) have been considered if difficulties are encountered
![Page 40: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/40.jpg)
Curiosity / Achieve. = Not Comp.
Methodology• Methodology is not adequate to reach the
target• Relations between the experiments and
hypothesis are not well defined• Methodology is not explained with a
common scientific basis• Possible problems and limitations are not
considered• Statistical analytical requirements are not
considered
![Page 41: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/41.jpg)
Curiosity / Achieve. = V. Comp.
Timeline• Proposed period & time schedule are
realistic
Budget• Proposed budget is realistic and well-
justified• Project is also supported by other
institutions
![Page 42: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/42.jpg)
Curiosity / Achieve. = Not Comp.
Timeline• Proposed period and time schedule are not
synchronized• Time schedule is not adequate
Budget• Budget is not well-define and requested
amount is too low / high• There is no possibility of support from
other institutions
![Page 43: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/43.jpg)
Customer / Merit = V. Comp.• Aims to develop a national / international novel
technology (methodology, system, product, process / technique)
• Brings comparable superiority to the existing system• Outcome will be a technology to be protected under
the intellectual property rights• Work consists of a scientific and rationale approach• An interdisciplinary project with the collaboration of
the Public- Private Industry-University-Research Institutions
![Page 44: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/44.jpg)
Customer / Merit = Not comp.
• Literature survey and market survey are not satisfactory, work is based on unreliable data
• No scientific / technological rationale and integrity in the project
• It is a study that had been done before in the country
• Aim, objective and motivation are not clear• Outcome is not qualified as
“applicable/usable”• Not related with a real need / problem
![Page 45: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/45.jpg)
Technological Level = V. Comp.
• Technology / product developed aims to fill a gap in the existing technology or replace the existing technology within the following 3 years
• Has an interdisciplinary approach to solve more than one problem
• Rationale of the R&D is well established (theoretical / analytical / experimental)
• Added value of the anticipated outcome of the R&D project is considered
• Has contribution to increase R&D staff• A doctorate / masters study is incorporated within
the project and the outcomes have potentials to be published in national/international journals.
![Page 46: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/46.jpg)
Technological Level = Not comp.
• Literature survey and market survey are not adequate
• No scientific/technological advance and integrity in the project
• R&D rationale (analytical and /or experimental) is not adequate
• Aim, objective and motivation are not clear.- Procedure that has been used will not provide a change / improvement in the methodology and technology
![Page 47: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/47.jpg)
Innovation Level = V. comp.
• Outcomes may lead to spin-off company• Outcomes include production standards
and technical specifications• Outcomes will affect other sectors as well• Outcomes very likely to increase the
export capacity of the country• Very likely to be a basis for generating
new projects / products
![Page 48: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/48.jpg)
Innovation Level = Not comp.
• Outcome is not qualified as “applicable/usable”
• Period of usefulness is very limited or not effective
• Does not provide new areas of work and as a result does not increase employment
• Expenditure for the R&D study is far beyond the expected economical benefit of the product
• Support provided by the private organization is not sufficient.
![Page 49: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/49.jpg)
CommentsReviewers (hundreds):• The model reduced the,
– burden of the review process on them, &– subjectivity and variability of the opinions of the
individuals• Convenience of selecting from a comprehensive
list of phrases as well as • the freedom of offering their own judgments
Researchers (thousands):• Sub-criteria phrases to be very instrumental in
guiding the development of their proposals
![Page 50: Models of Evaluation For Research Proposals in Turkey Prof. Dr. Omer CEBECI Vice President – Funding, TUBITAK (The Scientific and Technical Research Council](https://reader035.vdocument.in/reader035/viewer/2022062717/56649e575503460f94b503d0/html5/thumbnails/50.jpg)
Thank you
Hundreds of scientists, researchers and reviewers participated in
workshops for the development of the criteria and phrases listed in the
tables. Their contributions are gratefully
acknowledged.