locang the cwts as a professional expert organisaon in the jurisdicon … · 21st nordic workshop...
TRANSCRIPT
21stNordicWorkshoponBibliometricsandResearchPolicy,Copenhagen
04.11.2016
LocaBngtheCWTSasaprofessionalexpertorganisaBoninthejurisdicBonof
quanBtaBveresearchassessment
SabrinaPetersohn
TheoreBcalperspecBvesonindicatorproducersandusers
• STSperspec)vebasedontheconceptof„infrastructure“(Star/Bowker2006)developedbyAström(2016),„boundaryobjects“(Gieryn1983;Star/Griesemer1989)developedbyLeyesdorff/Wouters/Bornmann(inpress)or„regulatoryscience“(Jasanoff1990)byDeRijcke/Rushforth(2015)
• Sociologyofprofessionsperspec)vebasedonAbbo\‘s„professional
jurisdicBonsinasystemofprofessions“developedbyPetersohn(2016)
Bibliometricindicatorsasboundaryobjects(Leydesdorff,WoutersandBornmann)
• BibliometricevaluaBonpracBcesareshapedbythewayfourdifferentgroupsofactors:1)dataandindicatorproducers,2)bibliometricians,3)researchmanagersand4)theevaluatedscienBstsperceive,actonandconstructbibliometricindicators.
• AsboundaryobjectsindicatorshavedifferentimplicaBonsindiffering
contexts,resulBngfromtheinteracBonoftheseactors• EvaluaBonpracBcesaswellasthepublicdiscourseaboutresearch
assessmentisshapedbytheinteracBonofthesegroups
BibliometricresearchevaluaBonasacontestedprofessionaljurisdicBon
• ExperBseisinsBtuBonalizedinthreedisBnctforms(Abbo\1991):professions,organizaBonsandcommodiBes
• TheyapplyabstractknowledgetoparBcularcasesinsocietallyrelevant
problemareas(jurisdicBons)• ManyactorsstriveforcontroloftheproducBonandapplicaBonof
experBseinajurisdicBoncreaBngacompeBBonandthusaninterrelatedevolvingsystemofprofessions,organizaBonsandcommodiBes(Abbo\1988)
ShibingjurisdicBonsinthetaskareaofresearchevaluaBon
PeerReview
OldestformofassessingscienBficquality,relatedthescienBficpublicaBonandcareersystem
NaBonalmonitoring+insBtuBonalevaluaBon
FirstgeneraBonofS&Tindicators:naBonalresearchperformance,input-outputmodel(comparabilityatnaBonallevelduetoriseofnaBonalsciencepolicies)
StrategicposiBoning
SecondgeneraBonofS&Tindicators:indicatorssupportstrategicposiBoningofincreasinglyautonomousresearchorganizaBons
Hemlin(1996),Godin(2000)andLepori,Barré&Filliatreau(2008)
Peers StaBsBcaloffices
(input),ISI(output)
Firms,Dataproviders,Consultancies,ResearchgroupsGovernmental
Agencies
Researchevalua)onasprofessionaltask
area:Howtodeterminescien)fic
quality?
Commodity:Thomson‘SWebofScience
ExpertorganisaBon:ContractresearchinsBtute
Profession:AcademicLibrarians
JurisdicBonalBe:EmpoweringandeducaBnglibraryusersinbibliometrics
KnowledgebaseofprofessionalpracBce:Academicabstractknowledge->scienBficfield
ofbibliometrics
OccupaBonalgroupstriving
toprofessionalize:
Researchmanagers
DesignandproducBonofindicators
ProfessionalservicefirmsasexpertorganizaBons
• OrganizaBonsbecomesitesofprofessionalworkwhencomplexityandscaleofproblemsislargeandmulBprofessionalacBonneeded(Abbo\1991)andwhenthereclientsarealsoorganizedandhighlyeducated(Muzio/Kirkpatrick/Kipping2011)
• Differencetoprofessions:„(…)individualexper1seisincreasinglytheicing
onthecake(…)withthefundamentalprocessofbakingthecakeincreasinglyreliantinallPSFsonmoreexplicitandcommodi1zedformsofknowledge,managedbydatabases“(Faulconbridge2015:439)
• InsteadofbeingstronglydetachedfromclientsrelaBonalknowledgeandco-producBonwithclientsisemphasized(Muzio/Kirkpatrick/Kipping2011)
OpeningjurisdicBonsforbibliometricresearchevaluaBon:socio-poliBcal
backgroundsMassificaBonofhighereducaBon,accounBngforpublicexpenses,societalrelevanceofresearch
RiseofnaBonalsciencepolicies
IncreasinginsBtuBonalautonomonyinreturnforaccountabilityandinternalqualitycontrol(f.e.HOAKnota1985,WHWAct1992NLor1991Flanders)
Standardizedqualitycontrolmechanisms(VSNUProtocol1993NL,UnivGrantsCommi\eeresearchperformanceranking1986,1989,1992UK
EmergenceofactorsforbibliometricresearchevaluaBoninEurope
• NL:RAWBconductedownbibliometricstudies(f.eRigter,1986)andproducedfirstS&TreportonDutchscience
• NL:Adhocreviewcommi\ees(Verkenningscommissies):peersmadeuseof
publicaBonandcitaBondatatodeterminestrengthsandweaknessesoffields,firstcitaBonanalysisinVKBio-chemistry(1984)
• NL:Leidenproject(1981-1983):WorkinggroupatRectorsOfficeofLeiden
Universitydevelopedabibliometricmodeltoinformre-allocaBonoffundingtoimproveresearchfinanciaBon,publicaBonandcitaBonanalysesappliedtoFaculBesofMedicineandNaturalSciences
• UK:MarBn/IrvineSciencePolicyResearchUnit(SPRU)„BigScienceProject“1978performanceassessmentoflargelaboratories
ProfessionalclaimsoftwomaincompeBtorsCWTS(Netherlands)(e.g.VanRaan1996,1996)
SPRU(UnitedKingdom)(Mar)n&Irvine1983,Mar)n1996)
AssessingscienBficqualityneedsdivisionoflabour:peerreviewneedstobesupplementedbybibliometrics
AssessingscienBficqualityneedsdivisionoflabour:peerreviewandbibliometricsboth„par)alindicators“
BibliometricmethodsgivesinformaBononpastresearchperformance
BibliometricindicatorsareparBalindicatorsofscien)ficprogress
Outputasanaspectofscien)ficqualityarerepresentedbypublicaBonindicators
Aspectsofscien)ficproduc)vityarerepresentedbypublicaBonindicators
Impactasanaspectofscien)ficqualityisrepresentedbycitaBonindicators
Impactasanaspectofscien)ficprogressisrepresentedbycitaBonindicators
Bibliometricsonlyappliestofieldswithajournal-basedcommunicaBonstructure
CWTSasaprofessionalservicefirmandexpertorganizaBon
• MulBprofessionality:bibliometricexperBsegeneratedbyscienBficstafffromdifferentdisciplinesandtechnicalstaff
• Academicknowledgebaseistheevolvingfieldofscientometrics• Serviceprovisionisbasedonanenricheddata-systemthatcanbe
regardedasaspecializedexpert-systemonforresearchperformanceassessmentsandsciencemonitoringby‘mapping’
• Serviceprovisionisbasedonco-producBonwithclientsondifferent
levels:datacollecBonandverificaBonaswellasindicatorconstrucBonwhichisorientedtoclientsinformaBonneeds
FormaBonphase1986-2001TriggeredcreaBonofSOOS/ECOOM(BE)Luwel2000
FormaBonphase1986-2001
0
10
20
30
40
50
60
UNIV RESINST MINIST RESCOUNCIL
ADVCOUNCIL
INTORG EU ASSOC COMPANY ACADSCI INSTAPPLRES
FOUND OTHER
FormaBonphase1986-2001
Expansionphase2002-2010Crownindicator
debate:CompeBngnormalizaBonapproachesby
KarolinskaInsBtute,SciMagoGroup.ScienceMetrics
Expansionphase2002-2010
0
5
10
15
20
25
30
35
UNIV RESINST MINIST RESCOUNCIL
ADVCOUNCIL
INTORG EU ASSOC COMPANY ACADSCI INSTAPPLRES
FOUND OTHER
Expansionphase2002-2010
Reflexivephase2011-current
Reflexivephase2011-current
0
10
20
30
40
50
60
70
80
90
Reflexivephase2011-current
Researchevalua)onasprofessionaltask
area:Howtodeterminescien)fic
quality? Commodity:
Thomson‘sWebofScience
CWTS
AcademicLibrarians
JurisdicBonalBe:EmpoweringandeducaBnglibraryusersinbibliometrics
KnowledgebaseofprofessionalpracBce:Academicabstractknowledge->scienBficfield
ofbibliometrics
NIFU,ECOOM
JurisdicBonalBe:reflexiveandcontextualizedscientometricperformanceandbenchmarkstudies
CRIS
ThomsonAnalyBcs/InCites
ReferencesAbboY,A.(1988).Thesystemofprofessions:Anessayonthedivisionoflabor.AbboY,A.(1991).Thefutureofprofessions:OccupaBonandexperBseintheageoforganizaBon.ResearchintheSociologyofOrganiza1ons,8(1),17-42.Åström,F.(2016).OutliningananalyBcalframeworkformappingresearchevaluaBonlandscapes.InInterna1onalConferenceonScienceandTechnologyIndicators.Gieryn,T.F.(1983).Boundary-workandthedemarcaBonofsciencefromnon-science:StrainsandinterestsinprofessionalideologiesofscienBsts.Americansociologicalreview,781-795.Hemlin,S.(1996).ResearchonresearchevaluaBon.SocialEpistemology,10(2),209-250.Jasanoff,S.(1990).Thefibhbranch:scienBficadvisorsaspolicymakers.Lepori,B.,Barré,R.,&Filliatreau,G.(2008).NewperspecBvesandchallengesforthedesignandproducBonofS&Tindicators.ResearchEvalua1on,17(1),33-44.Leydesdorff,L.,Wouters,P.,&Bornmann,L.(2016).ProfessionalandciBzenbibliometrics:complementariBesandambivalencesinthedevelopmentanduseofindicators—astate-of-the-artreport.Scientometrics,1-22.Luwel,M.(2000).AbibliometricprofileofFlemishresearchinnatural,lifeandtechnicalsciences.Scientometrics,47(2),281-302.Mar)n,B.R.,&Irvine,J.(1983).Assessingbasicresearch:someparBalindicatorsofscienBficprogressinradioastronomy.Researchpolicy,12(2),61-90.Mar)n,B.R.(1996).TheuseofmulBpleindicatorsintheassessmentofbasicresearch.Scientometrics,36(3),343-362.Petersohn,S.(2016).ProfessionalcompetenciesandjurisdicBonalclaimsinevaluaBvebibliometrics:TheeducaBonalmandateofacademiclibrarians.Educa1onforInforma1on,32(2),165-193.Rijcke,S.,&Rushforth,A.(2015).Tointerveneornottointervene;isthatthequesBon?OntheroleofscientometricsinresearchevaluaBon.JournaloftheAssocia1onforInforma1onScienceandTechnology,66(9),1954-1958.Star,S.L.,&Bowker,G.C.(2006).Howtoinfrastructure.Handbookofnewmedia:SocialshapingandsocialconsequencesofICTs,230-245.Star,S.L.,&Griesemer,J.R.(1989).InsBtuBonalecology,translaBons'andboundaryobjects:AmateursandprofessionalsinBerkeley'sMuseumofVertebrateZoology,1907-39.Socialstudiesofscience,19(3),387-420.VanRaan,A."AdvancedbibliometricmethodsfortheevaluaBonofuniversiBes."Scientometrics45.3(1999):417-423.vanRaan,A.(1996).AdvancedbibliometricmethodsasquanBtaBvecoreofpeerreviewbasedevaluaBonandforesightexercises.Scientometrics,36(3),397-420.