projects on ssh scholars’ notions of research...

European Network for Research Evaluation in the Social Sciences and Humanities. COST action 15137. www.enressh.eu

1

Listof

ProjectsonSSHScholars’NotionsofResearchQuality

inParticipatingCountries

ENRESSHReport

MichaelOchsner,FORS,LausanneandETHZurich,SwitzerlandIoanaGalleron,ParisIIIUniversity,France

ArleenIonescu,UniversityofPloiesti,Romania

WithinputfromJuditBar-Ilan,PeterBiegelbauer,AlexandraBitusikova,KarinByland,DianaEerma,AldisGedutis,HarisGekis,EleaGiménez-Toledo,KatyaDe

Giovanni,JonHolm,SéverineLouvel,ReetaMuhonen,DejanPajić,ElenaPapanastasiou,SanjaPeković,GinevraPeruginelli,JannePölönen,AdPrins,HuldaProppé,Anna-SofiaRuth,ElíasSanz-Casado,DagmarSimon,AndrejaStarčič,Angelo

Tramountanis,MarcVanholsbeeck,MajaVehovec,AlbenaVutsova

WG to which the report/ deliverable is related: WG 1 Grant period to which the report/ deliverable is related: Grant Period 1

Version1.4,July2017


2


3

Executivesummary

What does research quality mean? This question becomes important when research is evaluated but remains largely unanswered, especially regarding the social sciences and humanities (SSH). This report gives an overview on empirical projects that explicitly investigate the notions of research quality of SSH scholars. Such projects are scarce and often not much known, yet of high importance as only with knowledge on what is to be measured, we can find adequate indicators for research quality. The report concludes that there is a growing interest in studies on SSH research evaluation and its methods and thus an increasing number of projects on such topics. However, the question what exactly research quality means in the eyes of the scholars regarding their every-day practice remains under-researched even though it stands at the centre of research evaluation. Therefore, Work Group 1 should fill this gap of knowledge

Keywords

Perceptions of Quality, SSH Scholars, Bottom-up, Project Overview


4

Introduction

In the last decades, a shift in university management practices towardsaccountability and new public management all over Europe came along with theimplementation of research assessment and evaluation procedures.While at first,researchassessmentconcernedmainlythenaturalandlifesciencesduetothecostlycharacterofresearchinthesedisciplines(seee.g.Krull&Tepperwien,2016),latelythesocialsciencesandhumanities(SSH)arealsosubject tosystematicevaluation.Themethodscommonlyused forresearchevaluation, i.e.bibliometricapproaches,relate to thepractices in thenatural and life sciences.However, suchbibliometricapproaches to research evaluation lead to unsatisfactory results when applied tosocialsciencesandhumanitiesasbibliometricresearchshows(see,e.g.,Hicks,2004;Glänzel,1996;Larivière,Gingras&Archambault,2004;Nederhof,2006;Nederhof,Zwaan,deBruin&Dekker,1989).This isduetoseveralreasons,suchasdifferentandmorediversepublicationpractices(Hicks,2004;Mutz,2013),differentresearchhabitsandpractices,aregionalorlocalorientationandtheuseofdiverselanguages(foranoverview,seeNederhof,2006).Butmostimportantly,therehasbeenmorethanahundredyearsofscientific inquiryonresearchpractices in thenaturalandlifescienceswhileknowledgeonhowresearchisconductedintheSSHisscarce(see,e.g.,Hemlin,1996).EspeciallywhatcharacterizesgoodSSHresearchremainslargelyunknown terrain, even in research exercises that do not resort to bibliometricapproaches.Whatdoesresearchqualityreallymean?Thisquestionbecomeshighlyimportant when research is evaluated but remains largely unanswered (see e.g.Brooks,2005),especiallyregardingthesocialsciencesandhumanities.While many stakeholders are involved in research evaluation, the notions ofresearch quality of the SSH scholars themselves are of pivotal importance if thequality of SSH research is to be judged. However, there is a lack of research onquality perceptions of SSH scholars. Given the inadequacy of currentmethods forresearchevaluationregardingSSHdisciplines, therewereacoupleof initiatives inEuropefocusingonresearchevaluationpractices intheSSHandhowtoadaptthemethodsforamoreadequateuseintheevaluationofSSHresearch(foranoverview,see Ochsner, Hug & Galleron 2017). Many of them investigate how to adaptbibliometric measures to fit to the SSH (e.g. Gimenez-Toledo et al., 2016;Hammarfeldt,2012;Lauer,2016;Sivertsen,2016),othersfocusonhowtoimproveevaluationofSSHgrantproposals(see,e.g.,König,2016;Krull&Tepperwien,2016).Inthisreport,wefocusonempiricalprojectsthatexplicitlyinvestigatethenotionsof research quality of SSH scholars. Such projects are scarce and often notmuchknown,yetofhighimportanceasonlywithknowledgeonwhatistobemeasured,wecanfindadequateindicatorsforresearchquality.This report serves as a starting point for thework inWork Group 1 “ConceptualFrameworks for Research Evaluation Practices in the SSH” of the COST-Action15137 “European Network for Research Evaluation in the SSH (ENRESSH)”. It isbasedonalistassembledinApril-June2016andwascontinuouslyexpandedasnewcountriesjoinedtheAction.Itcoversrecentprojectsthatstartedbetween2000and


5

2015,thefifteenyearsbeforetheActionstarted.ItwillnotonlyprovideliteratureonthiscentraltopicoftheWorkGroupbutalsodrawamaponwhatisdoneinthisregardinthedifferentparticipatingcountries.Thiswillhelptowidenthenetworkas much as to bring together the knowledge gained through these projects.Therefore, this report is not a comprehensive bibliography of research on SSHscholars’notionsof researchquality,but the listwillbecontinuedduring the life-timeoftheActionandwillfeedintothefinalreportsandbibliography.Because projects on SSH scholars’ notions of quality are very scarce, we alsoincluded projects that take a bottom-up approach to describe disseminationpracticesintheSSHorproposealternativeevaluationprocedures.The report is structured as follows: The first part will list projects on qualityperceptionsofSSHscholarsinparticipatingcountriessortedbycountry.Thesecondpartwilladdselectedprojectsondisseminationpracticesoralternativeevaluationprocedures. The third part concludes drawing a map of research on notions ofresearchqualityinEurope.

ReferencesBrooks,R.L.(2005).Measuringuniversityquality.TheReviewofHigherEducation,

29(1),1–21.doi:10.1353/rhe.2005.0061Giménez-Toledo,E.,etal.(2016).Takingscholarlybooksintoaccount:Current

developmentsinfiveEuropeancountries.Scientometrics,107(2),1–15.Glänzel,W.(1996).Abibliometricapproachtosocialsciences.Nationalresearch

performancesin6selectedsocialscienceareas,1990–1992.Scientometrics,35(3),291–307.

Hammarfelt,B.(2012).Followingthefootnotes:Abibliometricanalysisofcitationpatternsinliterarystudies.Doctoraldissertation.SkrifterutgivnavidinstitutionenförABMvidUppsalaUniversitet(Vol.5).Uppsala:UppsalaUniversitet.

Hemlin,S.(1996).Socialstudiesofthehumanities.AcasestudyofresearchconditionsandperformanceinancienthistoryandclassicalarchaeologyandEnglish.ResearchEvaluation,6(1),53–61.

Hicks,D.(2004).Thefourliteraturesofsocialscience.InH.Moed,W.Glänzel&U.Schmoch(eds)HandbookofQuantitativeScienceandTechnologyResearch(pp.473–496).NewYork,NY:KluwerAcademicPublishers.

König,T.(2016).PeerreviewinthesocialsciencesandhumanitiesattheEuropeanLevel:TheexperiencesoftheEuropeanresearchcouncil.InM.Ochsner,S.E.Hug&H.-D.Daniel(eds)ResearchAssessmentintheHumanities.TowardsCriteriaandProcedures(pp.151–163).Cham:SpringerOpen.

Krull,W.,&Tepperwien,A.(2016).Thefour‘I’s:Qualityindicatorsforthehumanities.InM.Ochsner,S.E.Hug&H.-D.Daniel(eds)ResearchAssessmentintheHumanities.TowardsCriteriaandProcedures(pp.165–179).Cham:SpringerOpen.


6

Lariviere,V.,Gingras,Y.,&Archambault,É.(2006).Canadiancollaborationnetworks:Acomparativeanalysisofthenaturalsciences,socialsciencesandthehumanities.Scientometrics,68(3),519–533.

Mutz,R.,Bornmann,L.,&Daniel,H.-D.(2013).Typesofresearchoutputprofiles:AmultilevellatentclassanalysisoftheAustrianScienceFund'sfinalprojectreportdata.ResearchEvaluation,22(2),118–133.doi:10.1093/reseval/rvs038

Nederhof,A.J.(2006).Bibliometricmonitoringofresearchperformanceinthesocialsciencesandthehumanities:Areview.Scientometrics,66(1),81–100.

Nederhof,A.J.,Zwaan,R.,deBruin,R.,&Dekker,P.(1989).Assessingtheusefulnessofbibliometricindicatorsforthehumanitiesandthesocialsciences—acomparativestudy.Scientometrics,15(5),423–435.

Ochsner,M.,Hug,S.E.,&Galleron,I.(2017).Thefutureofresearchassessmentinthehumanities:bottom-upassessmentprocedures.PalgraveCommunications,3,17020.doi:10.1057/palcomms.2017.20

Sivertsen,G.(2016).Publication-basedfunding:TheNorwegianmodel.InM.Ochsner,S.E.Hug&H.-D.Daniel(eds)ResearchAssessmentintheHumanities.TowardsCriteriaandProcedures(pp.79–90).Cham:SpringerOpen.


7

PARTI:ListofProjectsonSSHScholars’NotionsofQuality inEurope

BelgiumLanotionde«qualité»despublicationsdansl'évaluationdelarechercheetdeschercheursenSHS.MarcVanholsbeeck,UniversitéLibredeBruxellesPeriod:2008-2016TheprojectwasthePhDprojectofMarcVanholsbeeck.Publications:Vanholsbeeck,M.(2012).Dépôtnumériquedespublicationsetrenouveaudes

relationsentrelesacteursdelarecherche.InS.Proulx,&A.Klein(Dirs.),Connexions:communicationnumériqueetliensocial(pp.119-134).Namur:PressesuniversitairesdeNamur.

Vanholsbeeck,M.(2012).Entrequalitéprescriteetqualitésouhaitable:l’ambivalencedeschercheursencommunicationfaceàl’évaluationdeleurspublications.Quaderni,77,71-84.

Griffioen,D.,Hug,S.E.,&Vanholsbeeck,M.(2013).CriteriaforResearchQuality:InternationalPerspectives.InForumoftheEuropeanAssociationforInstitutionalResearch(EAIR)(Rotterdam,August2013).

Websites,Furtherinformation:http://difusion.ulb.ac.be/vufind/Record/ULB–DIPOT:oai:dipot.ulb.ac.be:2013/226291/Holdings

CroatiaResearchactivity,collaborationandorientationinsocialscienceinCroatiaandotherpost-socialistEuropeancountriesInstituteofSocialResearch,InstituteofEconomics,FacultiesofLaw,Psychology,PoliticalScienceandEducationoftheUniversityofZagrebPeriod:2015-2019TheprojectstudiesmanyaspectsofSSHresearchproductionusingquantitative(bibliometrics,survey,socialnetworkanalysis)andqualitative(interviews)methods.ToprovideacomprehensivecomparativedescriptionofthestateofsocialsciencesinCroatia,theprojectincludes10EUpost-socialistcountries(Estonia,Latvia,Lithuania,Poland,CzechRepublic,Slovakia,Hungary,Romania,Bulgaria,andSlovenia)aswellasex-Yugoslavcountries(BosniaandHercegovina,Macedonia,MontenegroandSerbia).Publication(s):Letina,S.,Robins,G.,&MaslićSeršić,D.(2016).ReachingOutfromaSmallScientific

Community:TheSocialInfluenceModelsofCollaborationacrossNationalandDisciplinaryBoundariesforScientistsinThreeFieldsofSocialSciences.Revijazasociologiju,46(2).doi:10.5613/rzs.46.2.1.


8

Jokić,M.;Petrušić,I.(2016).Nekioduzrokaslabezastupljenostihrvatskihsveučilištanasvjetskimrangljestvicamasveučilišta[SomeoftheReasonsforPoorRepresenationofCroatianUniversitiesinWorldUniversityRankings].Medijskaistraživanja,22,1,5-40.

Letina,S.(2016).Networkandactorattributeeffectsontheperformanceofresearchersintwofieldsofsocialscienceinasmallperipheralcommunity.JournalofInformetrics,10(2),571–595.

Jokić,M.&Sirotić,G.(2015).DotheInternationalEditorialBoardMembersofCroatianSocialSciencesandHumanitiesJournalsContributetotheirVisibility?Medijskaistraživanja,21(2),5-33.

Websites,Furtherinformation:http://www.idi.hr/racoss/index_en.html

Germany

EuropeanEducationalResearchQualityIndicators(EERQI)IngridGogolin,UniversityofHamburgThemissionofthisprojectwastodevelopnewapproachesfortheevaluationofqualityofeducationalresearchpublications.EERQIaimedtodevelopaprototypeframeworkfortheintelligentcombinationofnewindicatorsandmethodologiesfortheassessmentofqualityineducationalresearchtexts,makethisframeworkoperationalonamultilingualbasis(startingwithEnglish,German,FrenchandSwedish),testthetransferabilityoftheEERQIframeworktoanotherfieldofsocialsciencesandthehumanities.Publications:Gogolin,I.(2016).EuropeanEducationalResearchQualityIndicators(EERQI):An

Experiment.InM.Ochsner,S.E.Hug&H.-D.Daniel(eds.)ResearchAssessmentintheHumanities(pp.103-111).Cham:SpringerOpen.

Gogolin,I.,Astrom,F.,&Hansen,A.(eds.)(2014).AssessingQualityinEuropeanEducationalResearch.SpringerVS:Wiesbaden,Germany.

ItalyOLTRE-TheevaluationoflegalresearchmonographsGinevraPeruginelli,SebastianoFaro&TommasoAgnoloni,InstituteofTheoryandTechniquesofLegalInformationoftheNationalResearchCouncilPeriod:2015-2016Theprojectisfocusedonassessmentoflegalstudieswithaparticularattentiontotheroleofresearchmonographs.TheaimistodescribethestateoftheartinItalyinthisdisciplineandtodeterminethevariablesandindicatorsthathavegreaterconsensusamongItalianresearchers.Toachievethisgoal,anationalsurvey(allItalianlegalscholars)andacomparativeanalysisofdifferentcountrieshavebeencarriedout.Websites,Furtherinformation:http://www.ittig.cnr.it/progetti/valutazione-monografia-giuridica/


9

ROBINA-TheRoleofBookinnon-bibliometricareasTizianaLipiello,AntonellaBasso,UniversitàCa’FoscariVenezia,ItalyIoanaGalleron,GeoffreyWilliams,UniversitédeBretagne-Sud,FrancePeriod:2015-2017Thedefinitionofbook/monographinnon-bibliometricareas,basedonperceptionofscholarsinItalyandinEuropeandonQuantitative(databases,statisticalanalyses)andonQualitative(Questionnaires,FocusGroups,ComputerassistedQualitativeDataAnalysisSoftware).Theimpactofresearchevaluationprotocolsonpublicationhabits,andmorepreciselyonbookpublishing.Publication:Williams,G.,Basso,A.,Galleron,I.,&Lipiello,T.(forthcoming).More,lessorbetter:

theproblemofevaluatingbooksinSSHresearch.InABonnaccorsi(ed.)TheEvaluationofResearchinSocialSciencesandHumanities.Springer.

FranceThesocialsciencesandhumanitiesinthefaceofstandardsforqualityresearchAstudyofacademicjudgmentinaFrenchevaluationagencyClémentineGozlan,SciencePo,Paris,FrancePeriod:2012-2016Basedonqualitativeandquantitativedata,thisprojectanalysedhowevaluatorsinthedisciplinesofliteratureandgeographymobiliseasetofheterogeneousresourcestobuildtheirjudgmentinevaluationsofanationalevaluationagency,theAERES,andhowtheyenforcetheirdisciplinarystandardswithinaveryconstrainedevaluationframework.AnalysingthemechanismsbywhichthemembersoftwodisciplinestendtoinvalidatetheAgency’snormsorconformtoitsvaluescale,thispaperreflectsonthewayacademics(re)definethelegitimacyoftheirprofessionalpracticeswhentherulesofscientificconsecrationaresubjecttonormalization.Publications:Gozlan,C.(2016).Lesscienceshumainesetsocialesfaceauxstandardsd’evaluation

delaqualiteacademiqueEnquetesurlespratiquesdejugementdansuneagencefrançaise.Sociologie,7(3),261-280.doi:10.3917/socio.073.0261

Websites,Furtherinformation:http://www.cso.edu/cv_equipe.asp?per_id=187QualiSHS–Critèresdequalitédelarecherchédanslesscienceshumainesetsociales:perceptionsetpratiquesAprojectsupportedbythe“RéseaunationaldesMSH”.CoordinatingMSH:MSHB;participatingMSH:ISHLyonandMSHAlpes.Researchunitsinvolved:LICORN,PACTE,LAHRA,UPEC.PI:IoanaGalleron,GeoffreyWilliams,LICORN,FrancePeriod:2013-2014Theprojectaimedat:(1)collectperceptionsandidentifyconceptsofqualityaboutSSHresearchintheSSHcommunity;(2)identifydisciplinarydifferencesbetweenSSHdisciplineswithregardstotheconceptualisationofresearchquality;(3)understandinghow


10

researchqualityconceptsaregenerated;(4)exploretherelationshipbetweenconceptsofqualityandevaluationpracticesinFrance.Consideringthetimeframeandtheawardedfinancing,theprojectconcentratedparticularlyonthefourthgoal.ItgatheredandanalysedevaluationreportsproducedbyAERESaboutSSHresearchunits.Thesereportsshowthatthemaincriteriaforqualityduringtheevaluationexercisetakenintoconsiderationwasthe„coherence“oftheconsideredresearchunits;therewasnoattempttoqualifywhata„publicationdequalité“means,eveniftheexpressioncameoutofteninthereports.Publications:Shortdescriptionoftheprojectin:Ochsner,M.,Hug,S.E.,&Galleron,I.(2017).Thefutureofresearchassessmentin

thehumanities:bottom-upassessmentprocedures.PalgraveCommunications,3,17020.doi: 10.1057/palcomms.2017.20

SpainOpinionofresearchersinSocialSciencesandHumanitiesabouttheevaluationsystems.QualitativeapproachEleaGiménez-Toledo,SpanishNationalResearchCouncil(CSIC)Period:2011-2013ContentanalysisofanswersfromasurveywhereresearchersgavetheiropinionondifferentaspectsofevaluationprocessesPublications:Giménez-Toledo,E.(2016).Malestar:losinvestigadoresantesuevaluación

[Discontent.Researchersfacingtheirevaluation].Madrid:IberoamericanaVervuert.

Websites,Furtherinformation:https://www.iberoamericana-vervuert.es/FichaLibro.aspx?P1=109703

SwitzerlandDevelopingandTestingResearchQualityCriteriaintheHumanities,withanemphasisonLiteratureStudiesandArtHistoryMichaelOchsner,SvenE.Hug&Hans-DieterDaniel,UniversityofZurichandETHZurichPeriod:2009-2014TheprojectservedtodevelopaframeworkforqualitycriteriaforSSHresearchbasedonscholars'notionsofquality.Theoutputwasacatalogueofgenericdiscipline-specificqualitycriteriaforthreehumanitiesdisciplines,includingacollectionofindicatorsforresearchperformancelinkedtoqualitycriteria.Publications:Hug,S.E.,&Ochsner,M.(2014).Aframeworktoexploreanddevelopcriteriafor

assessingresearchqualityinthehumanities.InternationalJournalofEducationLawandPolicy,10(1),55–68.


11

Hug,S.E.,Ochsner,M.,&Daniel,H.-D.(2013).Criteriaforassessingresearchqualityinthehumanities:aDelphistudyamongscholarsofEnglishliterature,Germanliteratureandarthistory.ResearchEvaluation,22(5),369–383.doi:10.1093/reseval/rvt008

Ochsner,M.Hug,S.E.,&Daniel,H.-D.(2012).IndicatorsforResearchQualityintheHumanities:OpportunitiesandLimitations.Bibliometrie–PraxisundForschung,1,4.

Ochsner,M.,Hug,S.E.,&Daniel,H.-D.(2013).Fourtypesofresearchinthehumanities:Settingthestageforresearchqualitycriteriainthehumanities.ResearchEvaluation,22(2),79–92.doi:10.1093/reseval/rvs039

Ochsner,M.,Hug,S.E.,&Daniel,H.-D.(2014).Settingthestagefortheassessmentofresearchqualityinthehumanities.Consolidatingtheresultsoffourempiricalstudies.ZeitschriftfürErziehungswissenschaft17(6),111–132.

Ochsner,M.,Hug,S.E.,&Daniel,H.-D.(Eds.).(2016).ResearchAssessmentintheHumanities.Cham:SpringerOpen.doi:10.1007/978-3-319-29016-4

Websites,Furtherinformation:http://www.performances-recherche.ch/projects/entwicklung-und-erprobung-von-qualitaetskriterien-fuer-die-forschung-in-den-geisteswissenschaftenApplicationofBottom-Up-CriteriaintheAssessmentofGrantProposalsofJuniorResearchersSvenE.Hug,MichaelOchsner&Hans-DieterDaniel,UniversityofZurichandETHZurichPeriod:2015-2017BasedonqualityperceptionsofallscholarsinthehumanitiesdisciplinesinSwitzerland,anevaluationsheetforpeerreviewforgrantapplicationsiscreated.Websites,Furtherinformation:http://www.psh.ethz.ch/en/research/anwendung-von-bottom-up-kriterien-zur-beurteilung-von-geisteswis.htmlAssessmentoflegalresearchinSwitzerlandAndreasLienhard,FabianAmschwand&EvaHerrmann,UniversityofBernThierryTanquerel,AlexanderFlückiger&KarinByland,UniversityofGenevaPeriod:2013-2016ThisprojectisexploringthefieldofassessmentoflegalresearchinSwitzerland.Theexploratorystudyisbasedonananalysisoflegalnorms,asurveyamongsttherelevantstakeholders(legalscholars,editorsoflawjournals,lawschooldeans,lawyers)inSwitzerlandandaninternationalcomparativeanalysis.Seven“situations”inwhichlegalresearchisbeingassessedweredefined.Withinthisframework,methodsandcriteriaforassessingthequalityoflegalresearchwereidentifiedandanalysed.Publications:Lienhard,A.,Tanquerel,T.,Flückiger,A.,Amschwand,F.,Byland,K.,&Herrmann,E.

(2016).ForschungsevaluationinderRechtswissenschaft:GrundlagenundempirischeAnalyseinderSchweiz.Bern:Stämpfli.


12

Lienhard,A.,Tanquerel,T.,Amschwand,F.,Herrmann,E.,&Byland,K.(2015).L‘évaluationdelarechercheendroitenSuisse.InT.Tanquerel&A.Flückiger(eds.)L’évaluationdelarechercheendroit:enjeuxetméthodes(pp.373-407).Brussels:Bruylant.

Tanquerel,T.,&Flückiger,A.(eds.)(2015).L’évaluationdelarechercheendroit:enjeuxetméthodes.Brussels:Bruylant.

Lienhard,A.,Amschwand,F.,&Herrmann,E.(2013).ForschungsevaluationinderRechtswissenschaft:Ausgangslage,EntwicklungenundAusblick.LeGes,2013(2),411-435.

Lienhard,A.,&Amschwand,F.(2010).Forschungsevaluation,FachtagungderrechtswissenschaftlichenFakultätenderSchweizvom25.Juni2010.Bericht.November2010.Bern:UniversitätBern.

Websites,Furtherinformation:http://www.rechtswissenschaft.unibe.ch/ueber_uns/qualitaetssicherung/qualitaet_in_der_forschung/index_ger.htmlhttp://www.unige.ch/droit/cetel/recherches/evalRecherche.htmlResource-orientedinstrumentforthevisualisationofhumanitiesresearchontheexampleoftheologyWolfgangSchatz,SilviaMartens,UniversityofLuzernDésiréeDonzallaz,UniversityofFribourgPeriod:2013-2016Theproject’sgoalistheadequatevisualisationandpositioningoftheologicalresearchasanexampleforahumanitiesdiscipline.Firstly,qualitycriteriaandindicatorsaredevelopedinclosecollaborationwiththescholarsinthefield.Secondly,atechnicalinstrumentwillbedevelopedintheformofaweb-basedsoftware.Websites,Furtherinformation:http://www.performances-recherche.ch/projects/ressourcen-basiertes-instrument-zur-abbildung-geisteswissenschaftlicher-forschung-am-beispiel-der-theologieQualityofresearchJudithCzellar,JaquesLanarès,UniversityofLausanne,SwitzerlandPeriod:2009-2013Traditionalbibliometricindicatorsareconsideredtoolimitedforsomeresearchareassuchashumanitiesandsocialsciencesbecausetheymostlyrevealaspecificaspectofacademicperformance(quantityofpublications)andtendtoignoreasignificantpartofresearchproduction.Theprojectinvestigatedthelinksbetweenpracticesforassessingacademicperformance,bibliometricmethods’useandunderlyingvaluesofresearchqualitywithinthescientificcommunityofUniversityofLausanne,Switzerland.Publications:Czellar,J.,Lanarès,J.(2013).Qualityofresearch:Whichunderlyingvalues?

Scientometrics,95(3),1003-1021.


13

UnitedKingdomExpressionsofExcellenceinappliedandpractice-basedresearchCriticsofeducationresearchintherecentyearshavepointedthefingeratwhattheysawasitslowquality,impact,and‘valueformoney’.InthecontextoftheResearchAssessmentExercise,particularconcernshavebeenraisedaboutappliedandpractice-basededucationalresearchandhowbesttoassessitsquality.arguesthatqualityinappliedandpractice-basedresearchcannotbereducedtonarrowdefinitionsof‘scientificity’,‘impact’oreconomicefficiency.Itproposesanaccountofqualityinappliedandpractice-basededucationalresearchwhichencompassesmethodologicalandtheoreticalsolidity,useandimpact,butalsodialogue,deliberation,participation,ethicsandpersonalgrowth.Publications:Furlong,J.,&Oancea,A.(2005).Assessingqualityinappliedandpractice-based

educationalresearch:Aframeworkfordiscussion.Oxford:OxfordUniversityDepartmentofEducationalStudies.Retrievedfromhttp://www.esrc.ac.uk/ESRCInfoCentre/Images/assessing_quality_shortreport_tcm6-8232.pdf

Oancea,A.,&Furlong,J.(2007).Expressionsofexcellenceandtheassessmentofappliedandpractice-basedresearch.ResearchPapersinEducation,22(2),119–137.doi:10.1080/02671520701296056

DefinitionsofOriginality,SignificanceandRigourThevariouspanelsandsub-panelsconvenedtoundertaketheevaluationsinthe2008UnitedKingdomResearchAssessmentExercisewereaskedtoprovidetheirowninterpretationsofboththecriteriaandthecategories.Thepaperanalysesthose,andfindsthattheyarenolesssubjectivethanthegenericdescriptionsprovidedbythecentralorganisation.Publications:JohnstonR(2008)Onstructuringsubjectivejudgements:Originality,significance

andrigourinRAE2008.HigherEducationQuarterly;62(1/2):120–147.


14

PARTII:ProjectsandpublicationsonSSHdisseminationpracticesandSSH-specificevaluation(selection)

AustriaSpecifitiesoflawstudiesandtheireffectonresearchassessmentMaier,E.(submitted).BesonderheitenindenRechtswissenschaftenundihre

AuswirkungenaufdieForschungsbewertung.QualitätinderWissenschaft(QiW).

BelgiumAcriticaldiscussionofthetendencytoalignthe(performancebased)fundingofthehumanitiesonthebibliometricsinusageintheSTEM.DeLanghe,R.(2007).NaareenPluralistischModelvanOnderzoeksfinancieringin

deHumaneWetenschappen.InP.Loobuyck,G.,Vanheeswijck,W.,VanHerck,E.,Grieten,E.,&K.,Vercauteren(eds.),Welkeuniversiteitwillenwe(niet)?(pp.205-216).Gent:AcademiaPress.

CentreforResearch&DevelopmentMonitoring(ExpertisecentrumOnderzoekenOntwikkelingsmonitoring,ECOOM)Engels,T.,Ossenblok,&T.,Spruyt,E.(2012).Changingpublicationpatternsinthe

socialsciencesandhumanities,2000-2009.Scientometrics,93(2),373-390.Ossenblok,T.,Engels,T.C.E.,&Sivertsen,G.(2012).Therepresentationofthesocial

sciencesandhumanitiesintheWebofScience.AcomparisonofpublicationpatternsandincentivestructuresinFlandersandNorway(2005-09).ResearchEvaluation,21(4),280-290.

Verleysen,F.,&Engels,T.C.E.(2012).Alabelforpeerreviewedbooks.JournaloftheAmericanSocietyforInformationScienceandTechnology,64(2),428-430.

Website:https://www.ecoom.be/nl/research/publications/Sociale%20en%20Humane%20wetenschappenColloquesdel'InstitutIacchosServais,P.(ed.)(2011).L'évalutiondelarechercheenscienceshumainesetsociales.

Regardsdechercheurs.Louvain-la-Neuve:Academia-Bruylant,coll."Intellection–14"

Dufays,J.-L.,&Servais,P.(eds)incollab.deGourbin,C.,Laurent,P.-J.,Marquet,J.,&Zune,M.(2013).Publierenscienceshumaines-Quelsenjeux,quellesmodalités,quelssupports,quellediffusion?Louvain-la-Neuve:AcademiaL'Harmattan,coll."Intellection–20".


15

PenserlaScience–SéminairesIlyaPrigogineZaccai,E.,Timmermans,B.,Hudon,M.,clerbaux,B.,Leclercq,B.,&Bersini,H.(2016).

L'évaluationdelarechercheenquestion(s).Bruxelles:AcadémieroyaledeBelgique.

CzechRepublicEthnographyofUniversityDepartments:MasshighereducationininstitutionalsettingsStöckelová,T.(2012).Immutablemobilesderailed:STSandtheepistemic

geopoliticsofresearchassessment.Science,Technology&HumanValues,37(2),286-311.

Šima,K.(2015).EvidenceinCzechresearchevaluationpolicy:measuredandcontested.Evidence&Policy,13(1),81-95.doi:10.1332/174426415X14467432784664.

ThechangingstructureofacademictimeVostal,F.(2016).AcceleratingAcademia.Basingstoke,UK:PalgraveMacmillan.

Denmark/theNetherlandsMetricAssessmentofMonographsZuccala,A.A.,&Cornacchia,R.(2016)Datamatching,integration,andinteroper-

abilityforametricassessmentofmonographs.Scientometrics,108(1),465–484.

BookReviewsinEvaluationsZuccala,A.A.,&vanLeeuwen,T.(2011).Bookreviewsinhumanitiesresearch

evaluations.JournaloftheAmericanSocietyforInformationScienceandTechnology,62(10),1979–1991.

EstoniaBibliometricanalysisofEstonianfolkloreresearchLauk,K.(2016).BibliometricanalysisofEstonianfolkloreresearchandfolklore:

ElectronicJournalofFolklore.TRAMES,20(70/65),1,3-16.BibliometricanalysisofEstonianhistoricalsciencesMust,Ü.(1999).Estonianhistoricalscienceinthe1990s.ResearchEvaluation,8(2),

77−81.10.3152/147154499781777531.

FinlandDisciplinarydifferencesinpublishingpracticesPuuska,H.-M.,&Miettinen,M.(2008).Julkaisukäytännöteritieteenaloilla.

Opetusministeriönjulkaisuja2008:33.Helsinki:Opetusministeriö.


16

FranceIMPRESHS–ImpactsdelarecherchedanslesSHS:étudesdecaseenBretagneGalleron,I.,&Williams,G.(2013).TheGood,theBadandtheDownrightMediocre:

QualityJudgmentsinBookReviews.WordandText:AJournalofLiteraryStudiesandLinguistics,3(1),102-118.

GermanyResearchRatingbytheWissenschaftsratPlag,I.(2016).Researchassessmentinaphilologicaldiscipline:Criteriaandrater

reliability.InM.Ochsner,S.E.Hug&H.-D.Daniel(eds)ResearchAssessmentintheHumanities.TowardsCriteriaandProcedures(pp.235–247).Cham:SpringerOpen.

Riordan,P.,Ganser,C.,&Wolbring,T.(2011).ZurMessungvonForschungsqualitat.EinekritischeAnalysedesForschungsratingsdesWissenschaftsrats[Measuringthequalityofresearch-AcriticalanalysisoftheForschungsratingoftheGerrnanWissenschaftsrat].KölnerZeitschriftfürSoziologieundSozialpsychologie,63(1),147-l72.

Wissenschaftsrat.(2010).EmpfehlungenzurvergleichendenForschungsbewertungindenGeisteswissenschaften.Köln:Wissenschaftsrat.

Wissenschaftsrat.(2011).ForschungsratingAnglistik/Amerikanistik.Koln:Wissenschaftsrat.

ItalyCAVIBScholar:Coverage,reliabilityandvalidityofGoogleScholarbibliometricindicators.ThecaseofsocialsciencesinItalyBiolcati-Rinaldi,F.,&Salini,S.(2014-2016)Assessmentofthepresenceandavailabilityofmonographsusingonlinelibrarycatalogues.Biagetti,M.T.,Iacono,A.,Trombone,A.,&Schaerf,M.(2015-2016)EVA:Extraction,validationandanalysisofGoogleScholardatafornon-bibliometricscientificsectorsFerrara,A.,Montanelli,S.,&Verzillo,S.(2015-2016)


17

IsraelFundingresearchinthehumanities:challengesandopportunitiesroundtableHewitt,T.,&Hovav,M.(2015).Fundingresearchinthehumanities:challengesand

opportunities.FinalReport.Jerusalem:VanLeerJerusaleminstitute.http://www.vanleer.org.il/sites/files/atttachment_field/Report_Funding%20Research%20in%20the%20Humanities%2023-24.11.15.pdf

Website:http://www.vanleer.org.il/en/event/funding-research-humanities-challenges-and-opportunities

LithuaniaDatabaseLituanistikaPetrauskaitė,R.,Bloveščiūnienė,L.,Kolesinskienė,N.,&Štreimikis,A.(2006-2019)

FormerYugoslavRepublicofMacedoniaPublicationandResearchConditionsintheSocialSciencesDokmanovic,M.&Gicevska,S.(2013).Perspectivesandchallengesforyoung

researchersinthefieldofsocialsciencesandhumanitiesintheRepublicofMacedonia.InS.Indzevska&A.DimovaMancevska(eds)ResearchinsocialsciencesinMacedonia.State-of-affairs,challengesandrecommendationsforpublicpolicyimprovements(pp.73-102).Skopje:FoundationOpenSociety–Macedonia.

TheNetherlandsJudgingResearchonitsMeritsRoyalNetherlandsAcademyofArtsandSciences.(2005).Judgingresearchonits

Merits.AnadvisoryreportbytheCouncilfortheHumanitiesandtheSocialSciencesCouncil.Amsterdanm:RoyalNetherlandsAcademyofArtsandSciences.

QualityIndicatorsforResearchintheHumanitiesRoyalNetherlandsAcademyofArtsandSciences.(2011).QualityIndicatorsfor

ResearchintheHumanities.RoyalNetherlandsAcademyofArtsandSciences:Amsterdam,TheNetherlands.

NorwaySSHPublicationPatternsSivertsen,G.(2016).Publication-basedfunding:TheNorwegianmodel.InM.

Ochsner,S.E.Hug&H.-D.Daniel(eds)ResearchAssessmentintheHumanities.TowardsCriteriaandProcedures(pp.79–90).Cham:SpringerOpen.


18

PolandContemporaryPolishHumanitiesinthefaceoftheChallengesofScientometricKulczycki,E.(2017).AssessingPublicationsthroughaBibliometricIndicator:The

CaseofComprehensiveEvaluationofScientificUnitsinPoland.ResearchEvaluation,26(1),41-52.doi:10.1093/reseval/rvw023

Kulczycki,E.,Drabek,A.andRozkosz,E.A.(2015).Publikacjeazgłoszeniaewaluacyjne,czylizniekształconyobraznaukiwPolsce[Publicationsandevaluationitems:ThedistortedimageofscienceinPoland].Nauka,3,35–58.

Kulczycki,E.,Rozkosz,E.andDrabek,A.(2015).PublikacjepolskichbadaczywczasopismachzlistERIHwkontekscieewaluacjijednosteknaukowych[PublicationsofPolishScholarsinjournalsindexedontheERIHlists:aperspectiveoftheevaluationofscientificunits].KulturaiEdukacja,107(1),149–72.

PortugalSSHscholars’perceptionsonqualityassessmentCardoso,S.,Rosa,M.J.,&Santos,C.S.(2013).Differentacademics'characteristics,

differentperceptionsonqualityassessment?QualityAssuranceinEducation,21(1),96–117.doi:10.1108/09684881311293089

Abottom-upapproachtobuildingapublicationindicatorfortheSSHRamos,A.,Carrondo,M.A.,&Sarrico,C.(2014-today)

RomaniaSSHscholars’perceptiononqualityresearchintheHumanitiesandSocialSciencesandqualityofhighereducationinRomaniaIonescu,A.,Baumgarten,A.,Ciuparu,D.,Funeriu,D.,Frangopol,P.,Tismăneanu,V.

(2013).ApologiadeMediocritate.WordandText:AJournalofLiteraryStudiesandLinguistics,3(1),65-86.

Maci,M.(2016).Anatomiauneiimposturi.Oșcoalăincapabilăsăînvețe.PrefațădeH.-R.Patapievici,Bucharest:EdituraTrei.

Mesaroș,C.(ed.)(2017).Filosofiaînuniversitateacontemporană.Timişoara:EdituraUniversitățiideVest.

SloveniaSocialmechanismsforestablishingandmaintainingofscientificcollaborationCugmas,M.,Ferligoj,A.,Kronegger,L.(2016).Thestabilityofco-authorship

structures.Scientometrics,106(1),163-186.Kronegger,L.,Mali,F.,Ferligoj,A.,Doreian,P.(2015).Classifyingscientific

disciplinesinSlovenia:astudyoftheevolutionofcollaborationstructures.JournaloftheAssociationforInformationScienceandTechnology,66(2),321-339.


19

CooperationnetworksinSlovenescienceFerligoj,A.,Kronegger,L.,Mali,F.,Snijders,T.A.B.,Doreian,P.(2015).Scientific

collaborationdynamicsinanationalscientificsystem.Scientometrics,104(3),987-1012.

Cerinšek,M.,Batagelj,V.(2015).Generalizedtwo-modecores.SocialNetworks,42,80-87.

Gómez-Núñez,A.J.,Batagelj,V.,Vargas-Quesada,B.,deMoya-Anegón,F.,&Chinchilla-Rodríguez,Z.(2014).OptimizingSCImagoJournal&CountryRankclassificationbycommunitydetection.JournalofInformetrics,8(2),369-383.

SpainAssessmentofscientificpublishersandbooksonHumanitiesandSocialSciences:qualitativeandquantitativeindicatorsGiménez-Toledo,E.(2016).AssessmentofJournal&BookPublishersinthe

HumanitiesandSocialSciencesinSpain.InM.Ochsner,S.E.Hug&H.-D.Daniel(eds.)ResearchAssessmentintheHumanities.TowardsCriteriaandProcedures(pp.91-102).Cham:SpringerOpen.

CitationsinSpanishjournalsofModernandContemporaryHistorypublishedbetween2000and2005:methodsofextraction,normalizationanddiffusion;Bibliometricindices;Socialandthematicnetworks.[LascitasenrevistasespañolasdeHistoriaModernayContemporáneapublicadasentre2000y2005:métodosdeextracción,homologaciónydifusión;índicesbibliométricos;redessocialesytemáticas]FernandezIzquierdo,F.,RomanRoman,A.,Rubio,M.C.,MorenoDiazDelCampo,F.J.,

Martin,C.,Garcia-Zorita,C.,Lascurain,M.L.,Efrain-Garcia,P.,Povedano,E.M.,&SanzCasado,E.(2007).BibliometricStudyofEarlyModernHistoryinSpainBasedonBibliographicReferencesinNationalScientificJournalsandConferenceProceedings.ProceedingsofISSI2007-the11thinternationalconferenceoftheInternationalSocietyforScientometricsandInformetrics.Madrid,2007.

CreatingandstudiesoftheCAAC(collectionsandarchivesofcontemporaryart)CuencaasamethodologicalmodelforresearchexcellenceinFineArtsRamónAlcaláMellado,J.(2014-2016).DevelopmentofanObservatoryofresearchactivityofSpanishuniversitiesintheperiod2002-2009,basedonR&D&iindicators.[Desarrollodeunobservatoriodelaactividadinvestigadoradelasuniversidadesespañolasenelperíodo2002-2009,apartirdeindicadoresdeI+D+i]DeFilippo,D.,García-Zorita,C.,Marugan,S.,&SanzCasado,E.(2013).Profilesof

production,impact,visibilityandcollaborationoftheSpanishuniversitysysteminsocialsciencesandhumanities.Proceedingsofthe14thInternationalSocietyofScientometricsandInformetricsConference(ISSI).Vienna,2013.


20

InRecs:ImpactindexesforSpanishjournalsinSocialSciencesandHumanities2007-2014,EC³EvaluacióndelaCienciaydelaComunicaciónCientíficaWebsite:http://ec3.ugr.es/in-recs/Knowledgeorganizationsystemsinresearchevaluation.ImplicationsforSocialSciencesandHumanitiesLópezPiñeiro,C.,&GiménezToledo,E.(2011).KnowledgeClassification:AProblem

forScientificAssessmentinSpain?KnowledgeOrganization,38(5),367-380.MultidimensionalanalysisofspecializationinpublicationsofSocialSciencesandHumanitiesMañana-Rodríguez,J.(2016).Disciplinarityandinterdisciplinarityincitationand

referencedimensions.Scientometrics,110(2),617-642.doi:10.1007/s11192-016-2190-0

Mañana-Rodríguez,J.,&Giménez-Toledo,E.(2012).Scholarlypublishinginsocialsciencesandhumanities,associatedprobabilitiesofbelonginganditsspectrum:aquantitativeapproachfortheSpanishcase.Scientometrics,94(3),893-910.

Mañana-Rodríguez,J.,&Giménez-Toledo,E.(2011).CoverageofSpanishsocialsciencesandhumanitiesjournalsbynationalandinternationaldatabases.InformationResearch,16(4),paper506.

RESH/DICE:PublicinformationsystemprovidingquantitativeandqualitativeindicatorsforSpanishjournalsinSocialSciencesandHumanitiesGiménez-Toledo,E.,Román-Román,A.,&Alcain-Partearroyo,M.D.(2007).From

experimentationtocoordinationintheevaluationofSpanishscientificjournalsinthehumanitiesandsocialsciences.ResearchEvaluation,16(2),137-148.

ScholarlyPublishersIndicatorsGiménez-Toledo,E.,Mañana-Rodríguez,J.,&Tejada-Artigas,C.M.(2015).Scholarly

publishers'indicators:Prestige,specialization,andreviewsystemsofscholarlybookpublishers.Elprofesionaldelainformación,24(6),855-860.

Giménez-Toledo,E.,Tejada-Artigas,C.,&Mañana-Rodríguez,J.(2013).Evaluationofscientificbooks’publishersinsocialsciencesandhumanities:resultsofasurvey.ResearchEvaluation,22(1),64-77.doi:10.1093/reseval/rvs036

Spanishscientific-technicalebookfromtheperspectiveoffourrelevantagentsinthebookpublishingsectorRomero-Otero,I.-S.,&Giménez-Toledo,E.(2012).Thee-bookandSpanishScientific

PublishersinSocialandHumanSciences.JournalofScholarlyPublishing,43(4),395-420.doi:10.3138/jsp.43.4.395


21

Spain/theNetherlandsLibrarycataloganalysisasatoolinstudiesofsocialsciencesandhumanitiesTorres-Salinas,D.,Moed,H.F.(2009).Librarycataloganalysisasatoolinstudiesof

socialsciencesandhumanities:AnexploratorystudyofpublishedbooktitlesinEconomics.JournalofInformetrics,3(1),9-26.doi:10.1016/j.joi.2008.10.002

SwitzerlandMeasuringResearchOutputinCommunicationSciencesandEducationalSciencesbetweeninternationalbenchmarks,culturaldifferencesandsocialrelevanceBuhmann,A.,Ingenhoff,D.,Leopori,B.,&Wise,M.(2015).Analyzingresearch

patternsinmediaandcommunication.Communication-TheEuropeanJournalofcommunicationResearch,40(3),267-293.

Probst,C.,Lepori,B.,Filippo,D.,&Ingenhoff,D.(2011).Profilesandbeyond:constructingconsensusonmeasuringresearchoutputincommunicationsciences.ResearchEvaluation,20(1),73–88.

Décrireetmesurerlaféconditédelarechercheenscienceshumainesetsocialesàpartird'étudesdecasPerret,J-F.,Sormani,P.,Bovet,A.,Kohler,A.,&Poglia,E.(2011).Décrireetmesurer

la«fécondité»desrecherchesenscienceshumainesetsociales.Aperçud’unprojet.BulletindelaSAGW,2,40-42.

Scientometrics2.0:WissenschaftlicheReputationundVernetzungHoffmann,C-P.,Lutz,C.,&Meckel,M.(2016).ArelationalAltmetric?Network

CentralityonResearchGateasanIndicatorofScientificImpact.JournaloftheAmericanSocietyofInformationScience&Technology,67(4),765-775.

Switzerland/theNetherlandsAssessmentoflegalresearchinaEuropeancomparativeperspectiveLienhard,A.,Byland,K.,Schmid,M.,&vanGeestel,R.(2015-2016)Website:http://www.rechtswissenschaft.unibe.ch/ueber_uns/qualitaetssicherung/qualitaet_in_der_forschung/index_ger.html


22

PARTIII:Conclusion

ThereisnotalotofcurrentresearchontheSSHscholars’notionsofresearchqualityin participating countries (see figure 1). There is a focus on Western Europeancountries,inwhichprojectsontheinvestigationofscholars’notionsofqualitywerefunded: Spain, France, UK, Belgium, Germany, Switzerland, Italy build a block ofneighbouringcountrieswithCroatiaalmostsharingaborderwiththisblock.

Figure1:ProjectsonSSHscholars’notionsofqualityinEurope.

Collectingtheprojects,wefoundthatthereisagrowinginterestinstudiesonSSHresearchevaluationanditsmethodsandthusanincreasingnumberofprojectsonsuchtopics.However,thequestionwhatexactlyresearchqualitymeansintheeyesof the scholars regarding their every-daypractice remainsunder-researchedeventhough it stands at the centre of research evaluation. Therefore, Work Group 1should fill this gap of knowledge and focus on this important topic, investigatingnotionsofresearchqualityofSSHscholars inEurope, thushelpingto increasethevalidityofresearchevaluations.Regarding the fact that there are several projects inmanyEuropean countries onhow SSH research is disseminated, on how SSH research is evaluated and onresearch practices in general from a descriptive point of view, this reportwill beexpandedduring the life-timeof theActionby includingawiderrangeofprojectsthatrelatetoresearchqualityintheSSHanditsrelationtoresearchevaluation.


23

Appendix:ProjectsonSSHscholars’notionsofqualitythatstartedin2016afterthelaunchofENRESSH

CzechRepublicInsearchoflosttime:TemporalpressureanditsepistemicimplicationsincontemporaryCzechacademiaFilipVostalPeriod:2016-2018Thereisanapparentlypervasivesenseinthecontemporaryscientificworldthatthingsarespeeding-upincessantly.Scientistsreportchronicbusynessandinsufficienttimeforresearch,theyneedtopublishmorepapers,meetdeadlinesandpressahead.Engagingwithsociologicalanalysesoftimethisprojectscrutinizesthetextureandimplicationsofsuchexperience.ItwilldosobyexaminingcausesandmanifestationoftemporalpressureinthelivesofearlycareerscientistsintheCzechRepublic.Itaimstoexplorethepersonal,andparticularly,epistemicimplicationsofchangingacademictemporalitiesastheyrelatetobroadershiftsincontemporaryorganizationandmanagementofscience.Theethnographicinvestigationwillfocusonthewaysinwhichtemporalpressureaffectsknowledgeproductionwithinselecteddisciplines(physicsandeconomics)ofCzechscientificinstitutions.Overall,theprojectaimstotacklethequestionofwhetherscientifichyper-productivityanditsassociatedtemporaldemandshaveprogressiveconservativeconsequencesforscience.

NorwayCentreforResearchQualityandPolicyImpactStudies(R-QUEST)atNIFUNIFU,OslowithsixotherinstitutionsfromDenmark,Sweden,theNetherlandsandUKPeriod:2016-2024R-QUESTconstitutesan8-yearcommitmenttoexplorethenatureandmechanismsofresearchquality–fundedbytheRCNFORINNPOLinitiative.Thecentrewilladdressthreecloselyrelatedquestions:Whatisresearchquality?Howarenotionsofresearchqualitynegotiated,establishedandpracticed,andwhatarethemechanismsthroughwhichthesenotionsaffectpolicy?Whatarethedriversofhighqualityresearch,andwhatistheroleofpolicyindevelopingoutstandingresearch?Whataretheeffectsofhighqualityresearchonthesociety?Websites,Furtherinformation:http://www.r-quest.no/research-partners/

projects on ssh scholars’ notions of research...

Documents