interim evaluation & assessment of future options …...interim evaluation & assessment of...

December 2012

Interim evaluation & assessment of future options for Science in Society Actions

Assessment of future options

Final Report - Annexes

Interim evaluation & assessment of future options for Science in Society Actions i

Table of Contents

Annex A - The Public Consultation Process: The Questionnaire 1

Annex B - The Public Consultation Process: Experts for the Pretest 6

Annex C - The Public Consultation Survey: The Results 7

Annex D - The Public Consultation Process: Workshop 1: The Programme 66

Annex E - The Public Consultation Process: Workshop 1: The Participants 67

Annex F – The Impact Assessment Workshop 2: The Programme 70

Annex G – The Impact Assessment Workshop 2: The Participants 71

Annex H - Indicators for future evaluation and assessment exercises for Science in society actions 73

Interim evaluation & assessment of future options for Science in Society Actions 1

Annex A - The Public Consultation Process: The Questionnaire

A.1 Statements developed for the different SiS topics

A.1.1 Ethics

1. The notion that “responsible development” (i.e. considering ELSA in product development) is also a way to improve the coproduction of innovations and contributes to market success is widely accepted among EU companies.

2. The awareness of ethical issues among EU researchers will rise across all scientific fields (incl. SSH) and these will be mandatorily demonstrated in all EU applications.

3. The provisions in publicly funded research projects to conduct ethically sound science will

increase.

4. The majority of EU researchers considers Ethical, Legal and Social Aspects (ELSA) not to be a “harassment” or an external control over research activities but as fundamental means of conducting research.

5. An EU Ethics Framework for responsible research and innovation, based on fundamental ethical principles and European values will be implemented in coordination with relevant international organizations.

6. Religious organisations, nongovernmental organisations and social media platforms that address human rights will become central actors in decision making processes and design of future science policy and research.

7. An EU wide charter on ethical and social considerations will be drafted that takes into account the different issues among scientific fields (e.g. issues in SSH are different to those in Biochemistry).

8. An effective structure of local/national/EU ethical committees to assess and to approve research projects by transparent and commonly agreed ethical standards will be implemented.

9. The notion of Ethics in European research will be substituted by a broader concept of “Responsible Research and Innovation” that emphasises wider societal issues (such as equity, sustainability, etc.).

A.1.2 Gender

1. All relevant actors in science (councils, research institutes, etc. ) will increase in their decision-making bodies the proportion of women to at least 40%.

2. The recruitment processes for jobs in science are completely transparent in how merit is assessed and selection process functions.

3. It is a common standard at EU and member state level to request information about gender aspects in research content as part of project selection process.

4. All universities and research organisations in Europe dispose of gender action plans in order to recruit and retain more women for science.

5. The total share of female researchers in the EU will be raised to 45 percent

6. All members of project assessment panels are given training to raise their awareness of possible gender bias in selection process


7. The early integration of gender aspects in research leads to more innovative approaches and improved quality of methods and outcomes, enabling also companies in Europe to realize a competitive advantage.

8. The working conditions in research do not longer require all-embracing full time engagement but allow alternative career paths too.

9. Regulatory organisations developing research related standards, e.g. CONSORT, take a lead role in establishing criteria for integration of sex/gender analysis in research, including when reporting results.

A.1.3 Governance

1. All stakeholders (not only sciences-related communities, but also the general public, NGOs, industry, etc.) are involved in a meaningful way in the development of research priorities, including broadly based consensus-building processes.

2. SiS aspects are firmly integrated into the selection process of funding programmes of the EC (and not separated from technology development programmes).

3. Society’s interest in and knowledge of techno-scientific issues has increased significantly, which is a prerequisite for broad public participation.

4. Common guidelines including the formulation of good practices regarding the dimensions of SiS are adopted in the ERA ensuring responsible research and innovation.

5. An integrated European Research Area has been established and its principles of international cooperation and exchange with third countries are actively followed.

6. Scientific advice and expertise in policy-making processes aiming to strengthen the European science system are taken into account in a transparent and inclusive way.

7. The number of European universities entertaining institutionalised relationships with the business sector (e.g., adequate representation of businesses on advisory boards of universities) has doubled.

8. Knowledge exchange between universities and the society at large is an integral element of universities’ duties.

A.1.4 Open Access

1. 100% of the publications of publicly funded research in Europe has been made available via Open Access (OA).

2. OA is a policy priority and corresponding actions have been implemented in all Member States.

3. The majority of researchers is aware of what OA means and they can make the choice whether to use it or not for themselves.

4. All existing decentral repositories can be accessed via a unified interoperable European search portal (prerequisites are unified meta-data and document standards).

5. The use of creative common licenses or similar mechanisms is broadly established.

6. The idea to publish on paper is an idea of the past. Printed journals do not exist anymore, all publications are digital (whether Open Access or not).

7. Research data, related to a concrete publication or as raw data, is mostly made accessible via Open Access


A.1.5 Public Engagement

1. The general public will take an active part in decision making processes regarding research topics (direction) and appraisal of publicly funded projects (appropriateness).

2. Public engagement will become a key aspect for evaluating individual researchers and research organizations and it will have a similar importance as publications, conference papers etc.

3. A governance model for "upstream public engagement" will emerge, that provides for universal public engagement.

4. Every publicly funded research project is obliged to publish its results in a citizen oriented way.

5. The rate of EU citizens participating in the activities of a CSO dealing with science and technology related issues will rise to 15% (2010: 7%).

6. "Responsible Development" that takes into account economic circumstances, social perceptions, political and cultural frameworks and ethical compatibility will become a key concept in R&D processes.

7. Social platforms will be promoted and consolidated to broaden the scientific interest of the civil society and to increase input for the development of responsible strategic research agendas.

8. The competencies of the general public for science and technology appraisal (understanding political/ power and ethical, environmental, legal and societal aspects) will increase.

A.1.6 Science Communication

1. All research outputs will be required to include a lay summary describing the research rationale, methodology, results, any lessons learned and a more prospective statement about the significance and possible implications of the work.

2. Science Communication will be an integral part of the duties of all European scientists.

3. Collaborative forms of public – science interaction through web based technologies (blogs, wikis, text editing, etc.) will become standard procedures before and after publication of research results.

4. The great majority of European research funders will have made science communication obligatory and a condition of funding.

5. European research will be evaluated, in part at least, in terms of its success in science communication.

6. Professional and target group specific dissemination activities will take up significant parts of the budget in publicly funded research projects.

7. New tools, guidelines and training courses assure that most researchers are successful science communicators and most research reaches a mass audience.

8. A much greater proportion of European citizens will routinely interacting with the scientific community, with the traditional separation of professional and public eliminated by the explosion in new social media.

A.1.7 Science Education

1. The number of young people entering careers in science, research and technology will increase significantly due to government actions concerning science education (e.g. National Action Plans).


2. Due to a focus on secondary school’s science-teaching pedagogy towards Inquiry Based Science Education (IBSE), the number of young people choosing science subjects at universities will increase significantly.

3. Inquiry-based teaching will be implemented as the standard science teaching and learning method in the majority of secondary schools in Europe.

4. (Almost) all European universities offer children universities to provide scientific lectures for children.

5. The interest for science and its practice at a young age will be raised due to more science activities such as science centers, science museums or class room projects.

6. The number of companies offering partnerships between industry, schools, and research organisations to bridge the gap between science education and science careers will increase significantly.

7. The use of new media and social networks will lead to a positive image of working in science.

8. International co-operation and exchange in the field of Science Education research will improve science teaching practice in Europe significantly.

9. Systemic reforms that support science education practices sensitive to students’ gender, ethnic, culture and social class variations will be implemented

A.2 For each of the statements developed for the different thematic domains a series of questions should be answered by the respondents

a.) Please rate your level of expertise regarding the topic:

• Expert knowledge

• Common knowledge

b.) Which socio economic impacts do you mainly associate with the statement?

• Advance responsible research and innovation

• Support social inclusion

• Solution of grand challenges*

• Scientific excellence

• Economic competitiveness

*Annotation: The Lund Declaration on Grand Challenges states that:" Challenges must turn into sustainable solutions in areas such as global warming, tightening supplies of energy, water and food, ageing societies, public health, pandemics and security."

c.) When do you think the statement will become reality?

• Until 2014

• 2015-2020

• after 2020

• will not occur

d.) Which political level is suited best for political intervention?:

• Mainly European level

• Mainly national level

• Shared responsibilities between EU and National level

• Mainly regional level


e.) What is the most important policy measure in order to reach the goals formulated in the statement?

• Support of sustainable institutional structures

• Research funding programmes

• Change in regulation (standards, guidelines, etc.)

• Dissemination activities addressing a broader public

f.) What is the critical success factor in order to reach the goals formulated in the statement?

• Active participation of the private sector

• Empirical evidence for the tangible benefits

• Interconnectedness between major stakeholders

• Suitable structures that enable policy learning

g.) What potential risk do you associate with the statement?

• Enlargement of regional and social disparities

• Migration of risk technologies

• Loss of economic competitiveness

• Decrease of scientific excellence

• Overburdening costs for all stakeholders

h.) Do you consider the goal formulated in the statement desirable?

• Yes

• No

i.) Which is your preferred policy option for future EU SiS actions?

• Baseline Option - Continuity incl. Lessons learned

• Redistribution of the Programme - segmentation, place certain parts where most appropriate

• SiS Actions with a strong policy pillar - horizontal activity aiming at coordination and coherence

• Embedding approach - no further separate SiS actions but integration within other Funding programmes

• No further SiS actions beyond FP7 - Complete dismissal

j.) Standard demographics

• Country of residence (DROP DOWN LISTE)

• Institutional background (University, Research organisation, Civil Society Organization, Political Institution, other (TEXT)

• Gender (male/ female) (CHECKBOX)

• Age (<30, 30-50, >50) (CHECKBOX)

• Scientific background (CHECKBOX)

k.) Final page • Do you have comments on the Science in Society Programme or any other additional comments?

(TEXT)

• If you are interested to receive the final results of the project please leave your Email address (TEXT)


Annex B - The Public Consultation Process: Experts for the Pretest

B.1 Experts contacted for the qualitative pretest

Field Experts contacted for the pretest Ethics Andrzej Gorski, Medial University of Warsaw, PL

Linda Nielsen, Faculty of Law a the Uni Copenhagen, DK Roger Strand, Bergen University, NO Federico Rajola, UCSC, IT

Gender Martina Schraudner, Fraunhofer Society, Germany Teresa Rees, Cardiff University, UK Elizabeth Pollitzer, Portia Ltd. UK Sabine Mayer, FFG, AT Klara Sekanina, KTI, Switzerland

Governance Stefan Kuhlann, University of Twente, NL Phillipe Laredo, MBS, UK Bernd Carsten Stahl, De Montfort University Leicester, UK Peter Biegelbauer, Institute of Advanced Studies, Austria Kjetil Rommetveit, University of Bergen, NO

Open Access Ulrich Herb, SULB Saarbrücken Hans Pfeiffenberger ,Helmholtz-Society ,Germany Barbara Kalumenos, STM Publishing Carlinie Sutton, OASPA Salvatore Mele, CERN, FR

Public Engagement Rinie van Est, Rathnau Institute, NL Katarina Prpic, Institute of Social Research Zagreb, CR Nicolas Pidgeon, Cardiff University, UK Michel Callon, Centre de Sociologie de l’Innovation, FR

Science Education Monika Réti HU, Semmelweis University, HU Harriet Wallberg-Henriksson, Karolinska Institute Stockholm, SE Adja Kahveci, Canakkale University, TK Manuela Welzel-Breuer, PH Heidelberg DE Vincentas Lamanauskas, University of Siauliai, LI Kristina Zoldosova, University of Trnava, Slovakia


Annex C - The Public Consultation Survey: The Results

C.1 Characteristics of the respondents

64

31

1

1

1

2

2

2

2

3

4

4

4

4

5

6

6

6

7

9

9

12

13

15

20

24

34

0 10 20 30 40 50 60 70

Not specified

Other country

Poland

Luxembourg

Denmark

Slovakia

Finland

Czech Republic

Bulgaria

Slovenia

Romania

Lithuania

Ireland

Estonia

Malta

Sweden

Hungary

Greece

Belgium

Portugal

Austria

Italy

France

Spain

United Kingdom

Netherlands

Germany

Residence (N = 291)


15

36

49

0 10 20 30 40 50 60

Not specified

Male

Female

%

Gender (n=291)

14

35

48

4

0 10 20 30 40 50 60

Not specified

50 or older

30 to 50

Less than 30

%

Age (n=291)


19

3

4

6

19

48

0 10 20 30 40 50 60

Not specified/Others

Public Authority

Civil Society Organisation

Private Sector

Research Organisation

University/University of Applied Sciences

%

Institutional Background (n=291)


C.2 Ethics

C.2.1 Expertise (n=127)

17

34

44

0 10 20 30 40 50

Not specified

Expert knowledge

Common knowledge

%


C.2.2 Time Frame

21

23

22

22

24

23

25

24

24

11

6

9

6

11

8

10

13

9

6

3

42

15

15

5

5

3

9

12

5

13

20

23

9

21

19

24

43

49

10

25

21

44

31

34

26

7

14

4

12

6

12

7

7

6

0 10 20 30 40 50 60

An EU Ethics Framework for responsible research and innovation, based on fundamental ethical principles and

European values will be implemented in coordination with relevant international organisations.

The provisions in publicly funded research projects to conduct ethically sound science will increase.

Religious organisations, non-governmental organisations and social media platforms that address

human rights will become central actors in decision making processes and design of future science policy

and research.

The majority of EU researchers considers Ethical, Legal and Social Aspects (ELSA) not to be a "harassment" or

an external control over research activities but as fundamental means of conducting research.

The notion that 'responsible development' (i.e. considering ELSA in product development) is also a way

to improve the coproduction of innovations and contributes to market success is widely accepted among

EU companies.

The awareness of ethical issues among EU researchers will rise across all scientific fields (incl. SSH) and these

will be mandatorily demonstrated in all EU applications.

The notion of Ethics in European research will be substituted by a broader concept of "Responsible Research and Innovation" that emphasises wider societal issues (such as equity, sustainability, etc.)

An EU wide charter on ethical and social considerations will be drafted that takes into account the different issues among scientific fields (e.g. issues in SSH are

different to those in Biochemistry)

An effective structure of local/national/EU ethical committees to assess and to approve research projects by transparent and commonly agreed ethical standards

will be implemented.

Until 2014 2015-2020 After 2020 Will not occur Don't know Not specified


C.2.3 Political Level

27

26

29

27

30

26

28

26

27

6

6

19

9

9

6

8

11

7

1

2

4

5

3

1

1

1

2

32

40

23

32

36

38

41

29

47

6

13

16

9

5

12

4

4

7

29

13

9

19

18

17

18

30

11

0 10 20 30 40 50






and research.





EU companies.








Mainly european level Mainly national level

Shared responsibilities EU & national level Mainly regional level

Don't know Not specified


C.2.4 Impacts

5

7

19

11

8

9

8

10

9

11

10

5

8

27

7

10

9

8

20

24

8

22

12

21

15

21

19

22

19

35

16

12

14

26

20

19

55

54

27

46

41

48

45

42

45

0 10 20 30 40 50 60






and research.





EU companies.








Advance responsible research & innovation Support social inclusion

Solution of grand challenges Scientific excellence

Economic competitiveness Don't know


C.2.5 Policy Measures

7

7

26

13

13

11

11

9

9

17

19

29

26

24

24

24

19

19

42

29

16

30

31

33

30

38

37

28

45

13

27

22

34

28

29

26

22

22

12

17

17

16

20

20

24

0 10 20 30 40 50






and research.





EU companies.








Support of sustainable institutional structures Research funding programmes

Change in regulation (standards, guidelines, etc.) Dissemination activities addressing a broader public

Don't know


C.2.6 Success Factors

7

12

21

13

7

9

8

11

8

28

26

17

26

19

32

35

25

37

40

26

27

26

33

34

26

29

37

19

27

11

24

32

24

22

22

18

12

13

9

13

38

14

19

15

9

0 10 20 30 40 50






and research.





EU companies.








Active participation of the private sector Empirical evidence for the tangible benefits

Interconnectedness between major stakeholders Suitable structures that enable policy learning

Don't know


C.2.7 Risks

21

28

24

28

29

29

30

28

25

19

16

15

13

11

16

9

15

16

6

6

13

5

4

5

4

4

8

6

8

11

5

6

6

5

5

6

11

6

11

10

9

5

4

10

10

5

5

13

4

5

3

8

5

6

0 10 20 30 40

An EU Ethics Framework for responsible research and innovation, based on fundamental ethical principles

and European values will be implemented in coordination with relevant international organisations.




and research.





EU companies.








Enlargement of regional and social disparities Migration of risk technologies

Loss of economic competitiveness Decrease of scientific excellence

Overburdening costs for all stakeholders Don't know


C.2.8 Desirability

31

30

31

30

31

30

31

30

31

3

4

8

6

6

4

8

10

8

5

5

36

4

8

10

11

13

13

61

61

25

60

55

56

50

48

49

0 10 20 30 40 50 60 70

An EU Ethics Framework for responsible research and innovation, based on fundamental ethical principles

and European values will be implemented in coordination with relevant international organisations.




and research.





EU companies.








Yes No Don't know Not specified


C.3 Gender


20

46

32

0 10 20 30 40 50

Not specified

Expert knowledge

Common knowledge

%


C.3.2 Time Frame

22

22

22

22

23

23

23

23

24

6

8

7

6

8

4

13

7

21

14

23

9

6

12

16

7

13

5

36

26

19

28

34

21

26

31

21

19

16

22

28

21

25

23

19

21

3

6

21

10

1

11

8

8

8

0 10 20 30 40

All relevant actors in science (councils, research institutes, etc. ) will increase in their decision-making

bodies the proportion of women to at least 40%.

The recruitment processes for jobs in science are completely transparent in how merit is assessed and

selection process functions.

It is a common standard at EU and member state level to request information about gender aspects in research

content as part of project selection process.

All universities and research organisations in Europe dispose of gender action plans in order to recruit and

retain more women for science.

The total share of female researchers in the EU will be raised to 45 percent

All members of project assessment panels are given training to raise their awareness of possible gender bias

in selection process

The early integration of gender aspects in research leads to more innovative approaches and improved quality of

methods and outcomes, enabling also companies in Europe to realize a competitive advantage

The working conditions in research do not longer require all-embracing full time engagement but allow

alternative career paths too.

Regulatory organisations developing research related standards, e.g. CONSORT, take a lead role in

establishing criteria for integration of sex/gender analysis in research, including when reporting results.




2

3

2

4

5

2

5

5

11

2

7

2

5

5

9

3

6

2

54

40

53

40

50

41

46

42

44

11

20

5

16

7

13

7

19

3

7

7

14

11

9

9

13

4

16

0 10 20 30 40 50 60






















C.3.4 Impacts

2

1

4

4

2

3

6

4

19

30

30

15

25

34

22

47

34

15

46

47

39

43

47

33

42

25

22

25

15

17

16

25

12

21

15

11

48

46

39

54

43

50

30

57

33

48

37

49

39

42

43

43

29

32

0 10 20 30 40 50 60























2

2

2

4

3

4

7

4

17

20

20

20

22

25

20

24

22

15

53

57

52

55

45

46

38

49

42

30

24

36

28

37

30

35

26

20

0 10 20 30 40 50 60




















Don't know



6

4

5

6

7

6

6

6

18

37

32

28

33

30

39

20

26

30

37

31

33

36

37

32

34

32

32

36

34

32

34

34

26

43

30

22

17

23

16

14

27

14

36

31

13

0 10 20 30 40 50




















Don't know


C.3.7 Risks

38

38

39

39

43

36

42

35

44

5

8

9

8

4

17

6

8

11

4

2

4

4

7

1

2

4

5

4

2

3

1

2

2

4

9

4

2

4

1

1

0

0

1

3

2

10

7

5

6

6

2

7

7

2

0 10 20 30 40 50


















Enlargement of regional & social disparities Migration of risk technologies




C.3.8 Desirability

17

18

18

19

18

18

20

19

18

6

2

2

5

8

3

6

7

24

10

2

9

5

10

8

6

0

5

68

78

71

71

64

71

69

74

53

0 10 20 30 40 50 60 70 80 90




















C.4 Governance


28

36

30

0 10 20 30 40

Not specified

Expert knowledge

Common knowledge

%


C.4.2 Time Frame

24

27

28

29

28

28

29

28

3

8

5

11

6

8

12

5

31

11

11

5

4

12

10

7

27

21

28

19

36

23

20

16

10

29

20

33

21

22

27

34

6

4

8

3

5

6

2

10

0 10 20 30 40

All stakeholders (not only sciences-related communities, but also the general public, NGOs,

industry, etc.) are involved in a meaningful way in the development of research priorities, including broadly

based consensus-building processes.

SiS aspects are firmly integrated into the selection process of funding programmes of the EC (and not

separated from technology development programmes).

Society’s interest in and knowledge of techno-scientific issues has increased significantly, which is a prerequisite for broad public participation.

Common guidelines including the formulation of good practices regarding the dimensions of SiS are adopted in the ERA ensuring responsible research and innovation.

An integrated European Research Area has been established and its principles of international

cooperation and exchange with third countries are actively followed.

Scientific advice and expertise in policy-making processes aiming to strengthen the European science

system are taken into account in a transparent and inclusive way.

The number of European universities entertaining institutionalised relationships with the business sector

(e.g., adequate representation of businesses on advisory boards of universities) has doubled.

Knowledge exchange between universities and the society at large is an integral element of universities’

duties.




2

3

4

4

2

3

10

2

3

0

9

0

1

2

14

19

40

33

30

36

36

36

24

22

15

4

18

1

2

8

16

20

9

29

6

26

26

18

6

7

0 10 20 30 40 50

All stakeholders (not only sciences-related communities, but also the general public, NGOs, industry, etc.) are involved in a meaningful way in the development of

research priorities, including broadly based consensus-building processes.



Society’s interest in and knowledge of techno-scientific issues has increased significantly, which is a prerequisite

for broad public participation.









duties.





C.4.4 Impacts

3

3

3

5

4

3

12

4

20

13

14

11

17

12

33

19

21

22

18

23

24

24

16

18

29

24

22

16

24

25

15

24

29

21

31

17

21

19

11

32

38

41

26

44

29

38

13

32

0 10 20 30 40 50















duties.






2

3

2

3

3

3

10

3

29

11

49

11

10

23

15

38

20

34

10

33

21

32

22

23

29

46

23

30

33

24

26

27

0 10 20 30 40 50 60















duties.



Don't know



6

9

6

9

7

7

10

9

34

33

29

29

31

33

16

28

42

36

32

34

34

31

30

38

28

22

26

21

17

19

23

20

23

7

14

8

10

13

33

14

0 10 20 30 40 50















duties.



Don't know


C.4.7 Risks

14

20

23

17

24

25

16

22

24

18

9

18

12

12

8

16

16

14

7

9

5

7

20

14

10

9

3

10

6

3

6

3

7

6

3

8

5

3

5

1

9

2

9

5

8

6

15

3

0 10 20 30















duties.





C.4.8 Desirability

27

29

28

29

28

29

28

28

1

8

6

9

10

6

14

3

21

12

3

5

8

3

30

6

51

51

63

57

53

62

28

63

0 10 20 30 40 50 60 70

All stakeholders (not only sciences-related communities, but also the general public, NGOs,

industry, etc.) are involved in a meaningful way in the development of research priorities, including broadly

based consensus-building processes.



Society’s interest in and knowledge of techno-scientific issues has increased significantly, which is a prerequisite for broad public participation.









duties.



C.5 Open Access


13

34

50

0 10 20 30 40 50 60

Not specified

Expert knowledge

Common knowledge

%


C.5.2 Time Frame

11.1

14.2

12.8

12.4

12.8

11.9

12.8

9

11

6

7

8

6

5

18

10

4

16

8

32

22

31

18

12

34

24

29

37

26

40

41

28

38

19

21

5

7

25

3

9

2

2

0 10 20 30 40 50

100% of the publications of publicly funded research in Europe has been made available via Open Access (OA).

OA is a policy priority and corresponding actions have been implemented in all Member States.

The majority of researchers is aware of what OA means and they can make the choice whether to use it or not

for themselves.

All existing decentral repositories can be accessed via a unified interoperable European search portal

(prerequisites are unified meta-data and document standards).

The use of creative common licenses or similar mechanisms is broadly established.

The idea to publish on paper is an idea of the past. Printed journals do not exist anymore, all publications

are digital (whether Open Access or not).

Research data, related to a concrete publication or as raw data, is mostly made accessible via Open Access.




5

5

7

7

10

32

12

0

1

10

2

5

2

4

50

45

34

31

36

27

38

8

19

26

5

15

8

11

25

18

11

41

20

15

20

0 10 20 30 40 50 60




for themselves.











C.5.4 Impacts

5

9

9

10

16

32

8

33

30

16

23

30

26

30

41

30

43

41

27

22

47

28

26

20

23

16

12

30

46

35

25

32

36

17

36

59

48

54

45

41

23

52

0 10 20 30 40 50 60 70




for themselves.












3

5

6

6

10

25

4

28

22

33

25

31

25

23

52

50

41

38

44

28

47

51

43

44

30

28

25

49

0 10 20 30 40 50 60




for themselves.









Don't know



11

9

10

9

14

28

9

35

42

30

32

29

16

30

39

43

32

44

24

18

43

43

33

46

30

29

22

49

19

8

8

18

22

23

19

0 10 20 30 40 50 60




for themselves.









Don't know


C.5.7 Risks

23

30

35

29

37

30

28

18

13

8

20

9

9

15

12

9

9

7

6

10

7

11

10

5

3

5

7

7

15

9

5

17

7

11

15

5

6

7

4

4

9

3

0 10 20 30 40




for themselves.











C.5.8 Desirability

17

18

19

19

19

19

18

4

6

6

10

13

17

11

11

8

5

10

6

27

15

69

68

71

60

63

37

56

0 10 20 30 40 50 60 70 80




for themselves.









C.6 Public Engagement


18

44

35

0 10 20 30 40 50

Not specified

Expert knowledge

Common knowledge

%


C.6.2 Time Frame

16

16

18

16

18

18

17

17

5

7

13

12

8

5

12

7

33

36

25

6

1

8

10

11

16

23

27

30

19

28

21

22

27

16

16

33

45

35

34

36

3

2

1

2

8

5

5

7

0 10 20 30 40 50

Public engagement will become a key aspect for evaluating individual researchers and research

organizations and it will have a similar importance as publications, conference papers etc.

The general public will take an active part in decision making processes regarding research topics (direction) and appraisal of publicly funded projects (appropriate-

ness)

A governance model for "upstream public engagement" will emerge, that provides for universal public

engagement

The rate of EU citizens participating in the activities of a CSO dealing with science and technology related issues

will rise to 15% (2010: 7%)

Social platforms will be promoted and consolidated to broaden the scientific interest of the civil society and to

increase input for the development of responsible strategic research agendas

"Responsible Development" that takes into account economic circumstances, so-cial perceptions, political and cultural frameworks and ethical compatibility will

be-come a key concept in R&D processes

The competencies of the general public for science and technology appraisal (un-derstanding political/ power

and ethical, environmental, legal and soci(et)al aspects) will increase

Every publicly funded research project is obliged to publish its results in a citizen oriented way




21

21

21

21

22

21

21

23

6

7

11

9

8

7

8

6

8

11

6

13

8

4

18

2

39

30

34

33

41

37

25

41

15

21

14

9

11

13

19

14

11

10

13

15

10

18

9

14

0 10 20 30 40 50




ness)


engagement


will rise to 15% (2010: 7%)












C.6.4 Impacts

3

3

7

7

3

3

4

3

14

20

11

12

15

28

20

12

19

11

12

11

12

16

17

19

27

29

31

33

28

35

27

19

42

47

44

55

52

34

50

40

46

36

36

29

38

47

33

44

0 10 20 30 40 50 60




ness)


engagement


will rise to 15% (2010: 7%)













3

3

8

9

5

5

9

5

38

50

35

52

50

26

56

39

31

24

27

18

17

36

19

38

35

28

27

25

31

44

22

36

23

31

34

24

33

30

26

18

0 10 20 30 40 50 60




ness)


engagement


will rise to 15% (2010: 7%)










Don't know



11

8

14

17

10

9

12

16

34

36

37

30

35

39

35

28

29

25

20

23

20

38

19

21

17

14

12

15

15

28

16

15

0 10 20 30 40 50




ness)


engagement


will rise to 15% (2010: 7%)










Don't know


C.6.7 Risks

15

11

25

32

26

22

30

29 20

23

22

7

4

6

8

7

7

8

12

6

2

6

5

5

5

8

13

8

7

8

13

6

4

10

12

8

8

9

5

11

4

0 10 20 30 40




ness)


engagement


will rise to 15% (2010: 7%)












C.6.8 Desirability

23

23

23

24

24

23

23

23

3

5

17

11

9

7

3

7

26

32

13

3

3

8

4

14

48

40

47

62

64

62

70

55

0 10 20 30 40 50 60 70 80




ness)


engagement


will rise to 15% (2010: 7%)










C.7 Science Communication


24

49

25

0 10 20 30 40 50 60

Not specified

Expert knowledge

Common knowledge

%


C.7.2 Time Frame

20

23

20

20

21

23

22

22

4

7

6

4

5

7

2

7

11

9

12

5

12

17

15

13

17

16

16

20

15

20

25

26

36

35

36

43

40

28

31

27

12

10

9

9

7

6

5

6

0 10 20 30 40 50

All research outputs will be required to include a lay summary describing research rationale, methodology,

results, any lessons learned and a more prospective statement about significance and possible implications

of the work.

Science Communication will be an integral part of the duties of all European scientists.

Collaborative forms of public – science interaction through web based technologies (blogs, wikis, text

editing, etc.) will become standard procedures before and after publication of research results.

The great majority of European research funders will have made science communication obligatory and a

condition of funding.

European research will be evaluated, in part at least, in terms of its success in science communication.

Professional and target group specific dissemination activities will take up significant parts of the budget in

publicly funded research projects.

New tools, guidelines and training courses assure that most researchers are successful science communicators

and most research reaches a mass audience

A much greater proportion of European citizens will routinely interacting with the scientific community, with the traditional separation of professional and

public eliminated by the explosion in new social media.




4

5

8

5

4

11

6

7

0

1

5

1

2

2

7

9

48

52

39

44

43

43

40

34

10

6

14

7

7

11

12

12

15

13

11

20

20

9

11

15

0 10 20 30 40 50 60



of the work.











A much greater proportion of European citizens will routinely interacting with the scientific community, with

the traditional separation of professional and public eliminated by the explosion in new social media.





C.7.4 Impacts

3

3

5

4

9

13

5

8

21

15

17

20

15

14

21

13

28

29

26

20

22

17

25

20

21

22

18

16

14

13

20

20

48

43

45

37

34

30

42

47

48

48

46

43

40

34

36

37

0 10 20 30 40 50 60



of the work.


















5

4

7

5

8

12

10

8

34

36

39

24

24

22

30

50

37

35

27

35

35

35

29

15

40

40

37

52

43

44

32

24

0 10 20 30 40 50 60



of the work.















Don't know



6

6

10

7

12

16

6

10

28

31

26

35

28

21

27

26

37

38

38

38

27

25

35

33

33

33

27

31

38

39

35

27

14

12

14

15

7

10

16

20

0 10 20 30 40 50



of the work.















Don't know


C.7.7 Risks

30

26

24

25

26

25

32

30

18

22

20

27

16

28

20

17

6

10

11

9

14

12

11

8

4

5

8

4

8

11

5

4

7

4

5

4

7

4

4

4

8

6

10

6

7

5

7

13

0 10 20 30 40



of the work.

















C.7.8 Desirability

18

18

19

18

18

19

19

19

6

4

12

7

10

13

10

7

13

15

18

15

17

28

11

12

63

64

51

60

56

40

61

62

0 10 20 30 40 50 60 70



of the work.











A much greater proportion of European citizens will routinely interacting with the scientific community, with the traditional separation of professional and

public eliminated by the explosion in new social media.



C.8 Science Education


19

64

17

0 10 20 30 40 50 60 70

Not specified

Expert knowledge

Common knowledge

%


C.8.2 Time Frame

17

17

19

18

19

19

19

19

19

9

9

4

15

4

8

10

8

11

21

20

7

17

7

10

16

4

11

26

23

19

18

19

21

13

22

28

23

27

39

24

39

36

31

37

26

4

4

12

7

12

6

11

10

6

0 10 20 30 40 50

The number of young people entering careers in science, research and technology will increase significantly due to government actions concerning science education

(e.g. National Action Plans)

Due to a focus on secondary school’s science-teaching pedagogy towards Inquiry Based Science Education

(IBSE), the number of young people choosing science subjects at universities will increase significantly

Inquiry-based teaching will be implemented as the standard science teaching and learning method in the

majority of secondary schools in Europe.

(Almost) all European universities offer children universities to provide scientific lectures for children.

The interest for science and its practice at a young age will be raised due to more science activities such as

science centres, science museums or class room projects.

The number of companies offering partnerships between industry, schools, and research organisations to

bridge the gap between science education and science careers, will increase significantly.

The use of new media and social networks will lead to a positive image of working in science.

International co-operation and exchange in the field of science education research will improve science teaching

practice in Europe significantly.

Systemic reforms that support science education practices sensitive to students’ gender, ethnic, culture

and social class variations will be implemented.




3

5

5

6

3

5

12

2

7

8

12

7

14

24

21

11

3

6

36

33

35

28

24

26

32

46

40

27

24

25

18

20

22

13

4

15

7

7

8

14

8

6

13

25

12

0 10 20 30 40 50

The number of young people entering careers in science, research and technology will increase significantly due to

government actions concerning science education (e.g. National Action Plans)








The number of companies offering partnerships between industry, schools, and research organisations to bridge the gap between science education and science careers,

will increase significantly.










C.8.4 Impacts

5

9

12

10

6

6

10

7

7

47

36

21

15

20

49

18

23

21

35

35

29

22

25

26

18

39

19

31

24

20

10

15

17

15

26

16

32

34

32

40

47

22

38

30

49

31

25

26

19

20

30

24

32

24

0 10 20 30 40 50 60






















4

6

7

8

4

7

9

4

7

37

34

29

37

53

35

50

25

23

30

37

43

13

14

19

10

22

31

34

33

28

24

26

32

18

51

40

0 10 20 30 40 50 60



















Don't know



3

4

6

15

6

4

12

4

5

45

39

39

21

24

17

15

36

46

42

42

34

34

41

37

30

48

36

34

43

44

25

29

25

28

39

32

22

14

7

15

36

55

23

16

12

0 10 20 30 40 50 60



















Don't know


C.8.7 Risks

31

31

32

32

28

27

40

33

33

12

13

13

17

16

15

5

10

9

10

10

9

7

2

5

6

5

7

6

6

5

3

4

10

4

6

6

7

5

5

2

1

6

7

4

3

13

10

10

10

19

16

6

10

14

0 10 20 30 40 50





















C.8.8 Desirability

20

20

20

20

21

21

21

20

22

3

4

10

14

1

8

10

3

6

7

11

10

16

6

7

7

4

9

71

65

60

49

72

64

62

72

64

0 10 20 30 40 50 60 70 80

The number of young people entering careers in science, research and technology will increase significantly due to government actions concerning science education

(e.g. National Action Plans)








The number of companies offering partnerships between industry, schools, and research organisations

to bridge the gap between science education and science careers, will increase significantly.


International co-operation and exchange in the field of science education research will improve science

teaching practice in Europe significantly.





Annex D - The Public Consultation Process: Workshop 1: The Programme

Workshop: Future of the Science in Society Programme

Panel/ Subgroups Topic

10:00 – 10:30 all participants Welcome and Introduction (DG Research)

10:30 – 10.45 all participants Introduction to Part 1 (Technopolis)

10.45 – 11.15 all participants Introduction to Part 2 (Fraunhofer ISI), introduction to the workshop concept and schedule

11.15 – 11.30 Coffee break

11:30 – 12.30 all participants Interactive Part: Identification of future needs of SiS actions

12:30 – 13:30 Lunch break

13:30 – 15:15 Panel Sessions/ Break out groups

In depth discussion of the results of the public consultation, development of policy implications

small groups (max. 10)

Panel 1: Open access

Panel 2: Science education

Panel 3: Public engagement / Ethics

Panel 4: Gender

Panel 5: Science Communication

15:15 –15.30 Coffee break

15:30 – 16:30 all participants Presentations of Panel Sessions (5-10 Minutes each), Open discussion on future policy options

16:30 – 16:45 all participants Wrap up


Annex E - The Public Consultation Process: Workshop 1: The Participants

Participants

Name Institution

EU Commission

Isabelle Dupont DG Research & Innovation Belgium

Sivasegaram Manimaaran DG Research & Innovation Belgium

Evaluation Team

Neil Brown Technopolis Group Limited United Kingdom

Christien Enzing Technopolis Group Limited United Kingdom

Flora Giarracca

Technopolis Group Limited United Kingdom

Andrej Horvath Technopolis Group Limited United Kingdom

Sabeen Sidiqui

Technopolis Group Limited United Kingdom

Paul Simmonds Technopolis Group Limited United Kingdom

Marko Stermsek Technopolis Group Limited United Kingdom

James Stroyan Technopolis Group Limited United Kingdom

Bernd Beckert Fraunhofer Institute for Systems and Innovation Research ISI Germany

Susanne Bührer Fraunhofer Institute for Systems and Innovation Research ISI Germany

Kerstin Goos Fraunhofer Institute for Systems and Innovation Research ISI Germany

Kerstin Kripp Fraunhofer Institute for Systems and Innovation Research ISI Germany

Ralf Lindner Fraunhofer Institute for Systems and Innovation Research ISI Germany

Thomas Stehnken Fraunhofer Institute for Systems and Innovation Research ISI Germany

Ethics

Miltos Ladikas Centre for Professional Ethics, University of Central Lancashire United Kingdom

Ruud ter Meulen Bristol University United Kingdom


Name Institution

Gender

Michèle Baron Ministère de la Recherche France

Claudine Hermann European Platform of Women Scientists France

Teresa Pinto Portuguese Association on Women's Studies Portugal

Elizabeth Pollitzer Portia Ltd. United Kingdom

Martina Schraudner Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Germany

Corinna Wilken Die Österreichische Forschungsförderungsgesellschaft FFG Austria

Skaidre Zickiene Siauliai University, Faculty of Social Sciences, Deparment of Management Lithuania

Open Access

Lars Bjørnshauge SPARC Europe Netherlands

Marie Farge CNRS France

Ulrich Herb SULB Saarbrücken Germany

Barbara Kalumenos STM - International Association of Scientific, Technical and Medical Publishers, Public Affairs United Kingdom

Hans Pfeiffenberger Open Access Working Group of the Helmholtz Association Germany

Public Engagement

Maximilian Fochler Universität Wien Austria

Sonja van der Arend TU Delft The Netherlands

Science Communication

Joachim Allgeier Forschungszentrum Jülich, Ethics in the Neurosciences Germany

Michel Claessens ITER Belgium

Anne-Catherine Hauglustaine-Robert

Jardin des Sciences, EUSCEA network France

Cees Midden Eindhoven University of Technology The Netherlands

Gemma Revuelta Science Communication Observatory Spain

Halil Sykja University of Tirana


Name Institution

Albany

Science Education

Gabriele Anzellotti University of Trento, Faculty of Mathematical, Physical and Natural Sciences Italy

Nuria de Salvador Xtec Spain

Alexa Joyce European Schoolnet Brussels Belgium

Ajda Kahveci Çanakkale Onsekiz Mart University, Çanakkale Turkey

Monika Réti Institute of Education of Development Hungary

Ulrike Steiger EU-Forschungsreferat der Pädagogischen Hochschulen in Baden-Württemberg Germany

Manuela Wenzel-Breuer Pädagogische Hochschule Heidelberg Germany


Annex F – The Impact Assessment Workshop 2: The Programme

Registration and refreshments from 09.30

10.00 – 10.10 Plenary Welcome presentation (Fraunhofer ISI)

Purpose, Programme, Ground rules, Groups

10:10 – 10:30 Plenary Presentation by DG Research and Innovation on current context

10:30 – 11.00 Plenary Presentation of key messages from interim evaluation (Technopolis)

11:00 – 12:30 Thematic Groups Policy objectives, Policy Options, Impact Assessment

12:30 – 13:30 Lunch break

13:30 – 15:00 Thematic Groups Continuation of Impact Assessment with a special view on RRI requirements

15:00 – 15.15 Coffee break

15:15 – 16:45 Plenary Presentation of main results from the groups and plenary discussion about the RRI approach as a whole and how to communicate this in an integrated way (ISI)

16:45 – 16:50 Plenary Closing remarks and farewell (ISI)


Annex G – The Impact Assessment Workshop 2: The Participants

Expert workshop: Future of the Science in Society Programme Brussels, 22 June 2012

Participants

Name Institution

Jean-Pierre Alix CNRS France

Bernd Beckert Fraunhofer Institute for Systems and Innovation Research ISI Germany

Lars Bjørnshauge SPARC Europe the Netherlands

Franz Bogner Z-MNU (Centre of Math & Science Education), University of Bayreuth Germany

Susanne Bührer Fraunhofer Institute for Systems and Innovation Research ISI Germany

Nuria de Salvador Xtec Spain

Peter Fisch DG RTD A6, Ex Post Evaluation Unit EU

Flora Giaracca Technolopis Group Limited United Kingdom

Erich Griessler Institute for Advanced Studies Austria

Alexa Joyce European Schoolnet Brussels Belgium

Kerstin Junge Tavistock Institute United Kingdom

Murat Kahveci University of Hamburg Turkey

Kerstin Kripp Fraunhofer Institute for Systems and Innovation Research ISI Germany

Henning Kroll Fraunhofer Institute for Systems and Innovation Research ISI Germany

Les Levidow The Open University United Kingdom

Ralf Lindner Fraunhofer Institute for Systems and Innovation Research ISI Germany

Niclas Meyer Fraunhofer Institute for Systems and Innovation Research ISI Germany

Margaret Mort Lancaster University United Kingdom


Name Institution

Christophe Morvan Fondations Sciences Citoyennes France

Herbert Münder EUSEA Germany

Bhanu Neupane UNESCO France

Carole Paleco European Platform of Women Scientists-EPWS Belgium

Elizabeth Pollitzer Portia Ltd. United Kingdom

Peter Rebernik EUSEA Austria

Monika Reti Institute of Education of Development Hungary

Kjetil Rommetveit University of Bergen Norway

Jens Schneider Fraunhofer Institute for Systems and Innovation Research ISI Germany

Martina Schraudner Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Germany

Paul Simmonds Technolopis Group Limited United Kingdom

Marko Stermsek Technolopis Group Limited United Kingdom

James Stroyan Technolopis Group Limited United Kingdom

Alma Swan Key Perspectives Ltd. United Kingdom

Ruud ter Meulen Bristol University United Kingdom

Sybille van den Hove Median SCP Spain

Jeroen van den Hoven TU Delft the Netherlands

Guido van Steendam International Forum for Biophilosophy Belgium

Marjolein Zweekhorst Athena institute FALW, Uni Amsterdam the Netherlands


Annex H - Indicators for future evaluation and assessment exercises for Science in society actions

H.1 Ethics

Increase the provisions in publicly funded research projects to conduct ethically sound science / Implement an EU Ethics Framework for responsible research and innovation

To what extent do you believe ethical issues are relevant to your research project (tick box)?1

• Not relevant

• Minor relevance

• Significant relevance

• Critical

Number of Ethicists or others with considerable ethics experience involved in the project2

Did your project undergo an Ethics Review (and/or Screening)?3

Please indicate whether your project involved any of the following issues (tick box)4:

• RESEARCH ON HUMANS

• RESEARCH ON HUMAN EMBRYO/FOETUS

• PRIVACY

• RESEARCH ON ANIMALS

• RESEARCH INVOLVING DEVELOPING COUNTRIES

• DUAL USE

Increase awareness of ethical issues among EU researchers and companies

• Increased acceptance of ethical issues among the employees of the participating organisations (research institutes, universities, enterprises, others)

• Deepened competencies, knowledge and skills on ethical issues within the participating organisations

• Increased awareness among the employees for the benefits associated with ethical issues • Implementation of training courses on ethical issues • Institutionalisation of responsibilities and dedicated resources for ethical issues (central unit,

decentral representatives) within the participation organisations

1 Science and society reporting questionnaire 2 Science and society reporting questionnaire 3 See Template Project Final Report, part 4.3: Report on societal implications 4 See Template Project Final Report, part 4.3: Report on societal implications


H.2 Gender

Workforce statistics / alternative career paths and recruitment processes in research

Workforce statistics: Number of people (woman/men) who worked on the project (on a headcount basis)5

• Scientific Coordinator

• Work package leaders

• Experienced researchers (i.e. PhD holders)

• PhD Students

• Other

Long-term increase / decrease in women’s access to key roles in research and innovation within the participating organisations: total number and shares compared to the overall distribution in staff

Kind of career paths existing within the participating organisations: expert career, careers for executive positions, others (please specify)

Use of anonymised recruitment processes within the participating organisations [Yes, please specify the kind of positions for which such a procdure is used / No]

Any changes already implemented or planned regarding the career paths and recruitment processes within the participating organisations?

Provide all universities and research organisations in Europe with gender action plans in order to recruit and retain more women for science

Existence of gender actions plans within the project partners’ organisations

• Yes, all partners

• No, not all partners, but ... % disposing of a GAP

• No participating organisation disposing of a GAP yet

Type of actions mentioned within the GAPs6

• Design and implementation of an equal opportunity policy

• Clear defined targets to achieve a gender balance in the workforce

• Organisation of conferences and workshops on gender

• Actions to improve work-life balance

• Other [text box]

Norms, values and stereotypes shaping the institutional environment

• Existence, quantity and quality of measures and actions promoting gender equality

• Existence of a monitoring system to evaluate progress regarding gender equality

• Creation of an enabling environment for the advancement of women in S&T: gender sensibility of the existing reward and career mechanisms

5 See Template Project Final Report, part 4.3: Report on societal implications 6 See Template Project Final Report, part 4.3: Report on societal implications


Impacts of participation in gender projects on the participating organisations

• Increased acceptance gender issues among the employees of the participating organisations (research institutes, universities, enterprises, others)

• Increased sensibilities about existing stereotypes and discriminations among the workforce of the participating organisations

• Increased awareness for the benefits associated with am improved gender policy among the staff • Implementation of training courses on gender issues • Institutionalisation of responsibilities and dedicated resources for gender issues (central unit,

decentral representatives) • Implementation of monitoring and evaluation systems / Integration of gender issues in the existing

KPI systems

Make the early integration of gender aspects in research part of the project selection process and common standard at EU and MS level

Was there a gender dimension associated with the research content [Yes, please specify / No] 7

Distribution of resources (time, money) according to gender

• Gender bias regarding the distribution of resources within the organisation

• Genderisation of science and technology: number (total number and share) of research and development projects considering gender aspects in research

Contribution of the project regarding the development of research-related standards [Yes, please specify / No] 8

Specific training for all members of project assessment panels to raise their awareness of possible gender bias

7 Science and society reporting questionnaire 8 Science and society reporting questionnaire


H.3 Governance

Consider scientific advice and expertise in policy-making processes in a transparent and inclusive way

Policy impacts, particularly at EU level9:

• Working documents

• Regulations

• Improved problem-solving • Scientific advice and decision-making processes at different geographical levels10

• Policy (development) in SiS at different geographical levels or multiple levels

• Innovation in policy-making11

• Adoption and / or establishment of common guidelines including the formulation of good practices regarding the dimensions of SiS in the ERA ensuring responsible research and innovation

Encourage knowledge exchange with the society

Network structure and cooperation behaviour

• Share of new and existing partnerships within the project12

• New / improved interactions with the wider environment (direct interactions, indirect interactions, financial interactions13)

• Change in attitudes and behaviour regarding collaboration • New ways of stakeholder involvement and participatory processes • Formation of networks to structure research and debates • Structured partnerships with business14 • Establishment of structures promoting knowledge sharing between universities / research

organisations and society

Wider impacts on the respective systems:

• Main intended users and / or beneficiaries of the projects and its results (Universities and research organisation outside the project; private companies; public administration; policymakers; CSOs, associations / societies / federations, trade unions, consumer associations etc.)15

• Impact on these groups (in which way?) • Diffusion of innovation in products, services and processes16 • Strengthened competitive position of industry

9 See Manfred Horvat, Reflections on Evaluating FP7 Activities. Presentation during the workshop „Planning the future Knowledge Base for Evaluation“, Conf-Shop, Brussels, 30 September 2011, particularly issues 6-9

10 Questionnaire for SaS/SiS project participants 11 EPEC (2011): Understanding the Long Term Impact of the Framework Programme Final Report to the European

Commission, DG Research 12 Questionnaire for SaS/SiS project participants 13 Spaapen, Jack; van Drooge, Leonie (2011): Introduction „productive interactions“ in social impact assessment. In:

Research Evaluation, September 2011, pp. 211-218. 14 Questionnaire for SaS/SiS project participants 15 Questionnaire for SaS/SiS project participants 16 EPEC (2011): Understanding the Long Term Impact of the Framework Programme Final Report to the European

Commission, DG Research


Improve policy coordination across all levels as well as the management of research institutions

Knowledge gain17:

• Improved understanding of issues / problems • Improved internal knowledge and capabilities • Improved ability or capacity to conduct R&D • Enhanced reputation and image • Improved competitive position • Increased understanding between science and wider audiences • Increased discussion of societal aspects within the research community • Raised awareness among scientists of the concerns / interests of citizens

Structural impacts on ERA and European added value like18

• The formation of durable research communities • Spaces for mutual learning and benchmarking • Changes in research capabilities and behaviour, critical mass, • Better Integration of research • Improved Cohesion of Europe

17 Questionnaire for SaS/SiS project participants 18 See Manfred Horvat, Reflections on Evaluating FP7 Activitites. Presentation durnig the workshop „Planning the future

Knowledge Base for Evaluation“, Conf-Shop, Brussels, 30 September 2011, particularly issues 6-9


H.4 Open Access

Increase the availability of research data and publications of publicly funded research in Europe via Open Access

How many Articles were published / accepted for publication in peer-reviewed journals?

• How many of these are published in open access journals?

• How many of these are published in open repositories?

Strengthen the use of creative common licenses or similar mechanisms

• Number of common licenses used

• Other mechanisms used

Increase awareness of the importance of Open Access among researchers)

• Increased acceptance of open access among the employees of the participating organisations (research institutes, universities, enterprises, others)

• Deepened competencies, knowledge and skills on open access among the workforce of the participating organisation

• Increased awareness for the benefits associated with open access among the staff

Support the access of all existing decentral repositories via a unified interoperable European search portal

• Increased knowledge about unified metadata and document standards necessary for a unified interoperable European search portal

• Use of the unified metadata and document standards


H.5 Public Engagement

Increase the involvement of stakeholders in the development of research priorities, including broadly based consensus-building processes

Did your project engage with societal actors beyond the research community? [Yes/No]

If yes, did you engage with citizens (citizens' panels / juries) or organised civil society (NGOs, patients' groups etc.)?

• No

• Yes, in determining what research should be performed

• Yes, in implementing the research

• Yes, in communicating /disseminating / using the results of the project In doing so, did your project involve actors whose role is mainly to organise the dialogue with citizens and organised civil society (e.g. professional mediator; communication company, science museums)? [Yes/No] 19

Did you engage with government / public bodies or policy makers (including international organisations)

• No

• Yes, in framing the research agenda

• Yes, in implementing the research agenda

• Yes, in communicating /disseminating / using the results of the project 20

Will the project generate outputs (expertise or scientific advice) which could be used by policy makers?

• Yes, as a primary objective (please indicate areas below- multiple answers possible)

• Yes, as a secondary objective (please indicate areas below - multiple answer possible)

• No If Yes, in which fields?

If Yes, at which level?

• Local / regional levels

• National level

• European level

• International level 21

Support the publication of the results of every publicly funded research project in a citizen-oriented way

Kind and number of items covering the project in the non-academic press (print, radio, television)22:

19 See Template Project Final Report, part 4.3: Report on societal implications 20 See Template Project Final Report, part 4.3: Report on societal implications 21 See Template Project Final Report, part 4.3: Report on societal implications 22 Questionnaire for SaS/SiS project participants


Creation or participation in an event in order to communicate with the public (i.e. not just other researchers in the press): If yes, please specify

Activities to disseminate the project results (specifically) to audiences beyond research / academia:

• No. of popular science publications (books, book chapters, articles in newspapers/ magazines

• No. of lectures to the public (open lectures at universities, guest lectures by researchers)

• No. of participations in TV/radio

• No. of open houses

• No. of active participations in science cafés, science festivals, researchers’ nights

Promote the integration of social perceptions, political and cultural frameworks, economic circumstances and ethical compatibility as a key concept in R&D processes

• Increased awareness about public engagement issues among the employees of the participating organisations (research institutes, universities, enterprises, others)

• Deepened competencies, knowledge and skills on public engagement among the staff of the participating organisation

• Increased awareness for the benefits associated with public engagement within the participating organisation

• Institutionalisation of responsibilities and dedicated resources for public engagement (central unit, decentral representatives)

• Implementation of monitoring and evaluation systems / Integration of public engagement issues in the existing KPI systems


H.6 Science Education

Increase government actions and implement systemic reforms (e.g. National Action Plans) that support science education practices in STEM to raise the number of young people entering careers in science, research and technology.

Indicators to measure young people’s attitudes and motivation in science and scientific literacy:

• Proportion of national student cohort studying STEM subjects at level 4 and at level 5, indexed against EU average and showing the three year trend;

• Share of students following second stage of tertiary education leading to an advanced research qualification;

• EU MS performance on OECD PISA scientific literacy stats;

• EU MS performance for STEM being taught in 7-11 age group by STEM graduates;

• Young people's (<11 years) attitudes to science and science careers and their inclination to go on to study science. 23

Indicators to measure effects of STEM –skills on labour market/labour market requirements:

• EU MS STEM job vacancies by broad economic sector (absolute, indexed, and 3-year trend);

• EU MS salary levels for STEM occupations indexed against salary levels for non-STEM

• EU MS employers views on STEM-related skills shortages and resulting labour market inefficiencies and costs; 24

Identifying the most successful IBSE projects and networks and building on / strengthening these (i.e. concentration of effort).

Indicators to measure the efficiency and effectivity of existing science teaching practices and innovative methods, i.e. IBSE

• proportion of STEM students taught by IBSE • proportion of STEM teacher adopt IBSE methods

Develop and promote a more diverse range of pedagogical approaches and other types of science activities.

• Does your project involve working with students and/or school pupils (e.g. open days, participation in science festivals and events, prizes/competitions or joint projects)? [Yes, please specify / No] 25

• Does the project generate any science education material (e.g. kits, websites, explanatory booklets, DVDs)? [Yes, please specify / No] 26

23 Public engagement: International review, analysis and proposals on indicators for measuring public engagement. VA Report 2011:2. Stockholm.


25 See Template Project Final Report, part 4.3: Report on societal implications 26 See Template Project Final Report, part 4.3: Report on societal implications


Support and strengthen international co-operation and exchange in science education research to improve science teaching practice.

Indicators to measure collaborations in science education as well as the level of these collaborations ( regional/national/EU-level)

• Partnerships schools/industry

• Partnerships schools/research institutes

• Collaboration of teachers in teacher learning communities

• Cooperations between teachers and researchers

• Cooperations between innovative SME and universities or other higher education institutions27

Encourage the offer of partnerships between industry, schools, and research organisations to bridge the gap between science education and science careers.

Does your project involve working with students and/or school pupils (e.g. open days, participation in science festivals and events, prizes/competitions or joint projects)? [Yes, please specify / No] 28

Does your partnership employ and train researchers? [Yes/No] 29

How much (including the value of time spent, as well as paid-out costs) do you estimate your project (when it is completed) will have spent on considering and dealing with Science Education, Training and Career Development issues? 30

Does this project anticipate having a direct impact on the local economy?

If Yes, is the project:

• Stimulating employment

• Retaining highly trained personnel

• Creating possible spin-out/start-up companies 31


28 See Template Project Final Report, part 4.3: Report on societal implications 29 Science and society reporting questionnaire 30 Science and society reporting questionnaire 31 Science and society reporting questionnaire

technopolis |group| United Kingdom 3 Pavilion Buildings Brighton BN1 1EE United Kingdom T +44 1273 204320 F +44 1273 747299 E [email protected] www.technopolis-group.com

interim evaluation & assessment of future options …...interim evaluation & assessment of...

Documents