space-16-tec-2018 access to space · space-16-tec-2018: access to space context † indispensable...

42
Florence BéroudUNIT B1 REA SPACE-16-TEC-2018 Access to space

Upload: others

Post on 19-Oct-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Florence BéroudUNIT B1

REA

SPACE-16-TEC-2018 Access to space

Page 2: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Purpose of this briefing

• To ensure common understanding of the WP's key elements in order to guarantee a single and harmonised approach in the evaluation, including what is addressed under which sub-criterion

• To review the evaluation criteria and provide guidelines for the interpretation

Page 3: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

General info

• During the briefing and remote evaluation, all evaluators must remain anonymous in order to avoid any bias

• Evaluators are invited to ask questions by email ([email protected]) Please specify Call and Action number. We will reply to these questions either during the briefing or afterwards by email.

• An independent observer is attending the topic briefing to monitor the process

• The slides and the FAQs will also be available at the Space 2018 Evaluation website at https://ec.europa.eu/info/h2020-space-2018_en

• You have received by email the IER Guidelines document that summarises the information on the evaluation criteria presented here

• Your reference is the Work programme SPACE-16-TEC-2018 topic description, p.42: http://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-2020/main/h2020-wp1820-leit-space_en.pdf

Page 4: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

SPACE-16-TEC-2018: Access to space

Context

• Indispensable element of the entire value chain of space, of strategic importance towards Europe’s non-dependence;

• A matter of security of supply, industry capability and technology readiness, and a condition of the modern space knowledge-based economies. Europe has achieved a remarkable position in this field amongst a handful of space fairing countries;

• The Space Strategy for Europe (COM(2016) 705 26/10/2016) has confirmed that Europe shall maintain autonomous reliable and cost-effective access to space. It underlined that it is crucial that Europe continues to have modern, efficient and flexible launch, production and test infrastructure facilities;

• Cost reduction and improving flexibility of European launch systems are the main challenges in order to foster European competitiveness on the global market.

Page 5: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

SPACE-16-TEC-2018: Access to spaceWhat?

- Research and Innovation action (RIA):

The specific challenge is to support research and innovation in technologies, in complementarity and full synergy with the on-going work undertaken by Member States and European initiatives, contributing to:

- a) Investigate the potential Launch system reusability technologies

- b) Launch system advanced manufacturing and modern ground infrastructures

How much?

- Total budget: 10 million EUR

- Estimated amount per proposal: 2 -3 million EUR

- Funding rate: 100%

- Maximum two proposals per sub-topic shall be selected for funding

Page 6: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Individual Evaluation

Report ( IER)

Page 7: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

A w a r d c r i t e r i a , s c o r e s a n d w e i g h t i n g

AW A R D C R I T E R I A

Excellence Impact Quality and efficiency of the implementation

Scoring: from 0 to 5*, weakness / shortcoming

0 — The proposal fails to address the criterion or cannot be assessed due to missing or incomplete information.

1 — Poor. The criterion is inadequately addressed, or there are serious inherent weaknesses.

2 — Fair. The proposal broadly addresses the criterion, but there are significant weaknesses.

3 — Good. The proposal addresses the criterion well, but a number of shortcomings are present.

4 — Very Good. The proposal addresses the criterion very well, but a small number of shortcomings are present.

5 — Excellent. The proposal successfully addresses all relevant aspects of the criterion. Any shortcomings are minor.

*You can award 0.5 points

Thresholds & weighting

Threshold for individual criteria = 3 + Overall threshold, applying to the sum of the three individual scores, =10.

Page 8: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Criter ion 1

- Excel lence

Page 9: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

E X C E L L E N C E

• HOW DOES THE PROPOSAL ADDRESS THE SPECIFICCHALLENGE AND SCOPE OF THE WORK PROGRAMMETOPIC?

• ARE THE OBJECTIVES CLEAR, MEASURABLE, REALISTIC AND ACHIEVABLE WITHIN PROJECTDURATION?

1 . 1 C l a r i t y a n d p e r t i n e n c e o f t h e o b j e c t i ve s

EXAMPLES

STRENGHTS

+ The objectives are clearly stated and are (fully) within the scope of the TEC-16-2018 topic.

+ The objectives are described clearly and appropriate quantifiable Key Performance Indicators have been identified to monitor xxxx.

WEAKNESSES/SHORTCOMINGS

- The specific objectives have not been presented with sufficient precision and have not been convincingly demonstrated as fully pertinent to the TEC-16-2018 topic.

- The objectives are formulated in too generic terms and as general problems to be solved rather than as solutions to be provided. They are also not focused adequately on business cases.

Page 10: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

1.1 Clarity and pertinence of the Objectives

Scope: Proposals shall address only one of two sub-topics

a) Investigate the potential launch system reusability technologies

New technologies relevant to reusability (advanced avionics and data management, structural parts and materials, health monitoring and control technologies, propulsion, disconnection systems, special ground systems)

b) Launch system advanced manufacturing an modern ground infrastructures

Technologies &processes for European launch systems(existing and planned including micro launch systems) development, production, testing and operations (automation, digital technologies to benefit quality and costs, digital assistance to operators, big data processing, advanced materials &low cost manufacturing, remote operations, ground infrastructures for European launch sites).

E X C E L L E N C E

Page 11: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

1 . 2 S o u n d n e s s o f t h e c o n c e p t , a n d c r e d i b i l i t y o f t h e p r o p o s e d m e t h o d o l o g y

• IS THE CONCEPT OF THE PROPOSAL SOUND (I.E. BASEDON MODELS OR ASSUMPTIONS WHICH ARE VALID ANDCONVINCING)?

• DOES THE PROPOSAL SHOW THAT THE PROPOSEDMETHODOLOGY IS APPROPRIATE AND EFFECTIVE FORACHIEVING THE STATED OBJECTIVES?

E X C E L L E N C E

• IS THE INITIAL AND TARGETED TECHNOLOGY READINESS LEVEL (TRL) PROPERLYDESCRIBED AND REALISTICALLY ACHIEVABLE WITH THE PROJECT PROPOSED?

Page 12: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

1.2 Soundness of concept, credibility of the methodology

E X C E L L E N C E

Sub-topic a)Reusability: Building block to prepare for in-flight demonstration

TRL 5/6Assessment of cost effectiveness for exploitationEstimation of investments and time schedule to reach TRL 8/9

Please use TRLs as they are defined in ISO 16290:2013

Sub-topic b) Advanced manufacturing & modern infrastructureTake account of possible evolutions of regulatory framework in particular REACHTake account of environmental impacts and health risks

For Both subtopics, activities need to be complementary with other European activities in the same domain.

Page 13: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

E X C E L L E N C E

Technology Readiness Level

Milestone achieved for the element

Work achievement (documented)

TRL 2: Technology concept and/or application formulated

Formulation of potential applications and preliminary element concept. No proof of concept yet.

• Formulation of potential applications. • Preliminary conceptual design of the element, providing understanding of how the basic principles would be used.

TRL 3: Analytical and experimental critical function and/or characteristic proof-of-concept

Element concept is elaborated and expected performance is demonstrated through analytical models supported by experimental data and characteristics.

• Preliminary performance requirements (can target several missions) including definition of functional performance requirements. • Conceptual design of the element. • Experimental data inputs, laboratory-based experiment definition and results. • Element analytical models for the proof-of-concept.

TRL 4: Component and/or breadboard functional verification in laboratory environment

Element functional performance is demonstrated by breadboard testing in laboratory environment.

• Preliminary performance requirements (can target several missions) with definition of functional performance requirements. • Conceptual design of the element. • Functional performance test plan. • Breadboard definition for the functional performance verification. • Breadboard test reports.

Page 14: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

E X C E L L E N C E

TRL 5: Component and/or breadboard critical function verification in a relevant environment

Critical functions of the element are identified and the associated relevant environment is defined. Breadboards not full-scale are built for verifying the performance through testing in the relevant environment, subject to scaling effects.

• Preliminary definition of performance requirements and of the relevant environment. • Identification and analysis of the element critical functions. • Preliminary design of the element, supported by appropriate models for the critical functions verification. • Critical function test plan. Analysis of scaling effects. • Breadboard definition for the critical function verification. • Breadboard test reports.

TRL 6: Model demonstrating the critical functions of the element in a relevant environment

Critical functions of the element are verified, performance is demonstrated in the relevant environment and representative model(s) in form, fit and function.

• Definition of performance requirements and of the relevant environment. • Identification and analysis of the element critical functions. • Design of the element, supported by appropriate models for the critical functions verification. • Critical function test plan. • Model definition for the critical function verifications. • Model test reports.

Page 15: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

E X C E L L E N C E

TRL 7: Model demonstrating the element performance for the operational environment

Performance is demonstrated for the operational environment, on the ground or if necessary in space. A representative model, fully reflective all aspects of the flight model design, is build and tested with adequate margins for demonstrating the performance in the operational environment.

• Definition of performance requirements, including definition of the operational environment.

• Model definition and realisation.• Model test plan.• Model test results.

TRL 8: Actual system completed and accepted for flight ("flight qualified")

Flight model is qualified and integrated in the final system ready for flight.

• Flight model is built and integrated into the final system.

• Flight acceptance of the final system.

TRL 9: Actual system "flight proven" through successful mission operations

Technology is mature. The element is successfully in service for the assigned mission in the actual operational environment.

• Commissioning in early operation phase.

• In-orbit operation report.

Page 16: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

1 . 3 E x t e n t t h a t p r o p o s e d w o r k i s b e yo n d t h e s t a t e o f t h e a r t , a n d d e m o n s t r a t e s i n n o va t i o n p o t e n t i a l

• IS THE STATE OF THE ART WELL RESEARCHED AND PRESENTED AND TO WHATEXTENT DOES THE PROPOSED WORK GO BEYOND THE STATE OF THE ART?

• IS THE PROPOSED WORK AMBITIOUS ENOUGH TO BE INNOVATIVE, WITHOUTBEING SO AMBITIOUS THAT IT IS NOT FEASIBLE?

• IS THE INNOVATION POTENTIAL DESCRIBED IN FUNCTION OF THE ADDRESSEDCRITICAL SPACE TECHNOLOGY:

E X C E L L E N C E

- WHICH ARE THEIR GROUND-BREAKING OBJECTIVES, NOVEL CONCEPTS AND APPROACHES PROPOSED ANDTO WHAT EXTENT ARE THEY INNOVATIVE?

- WHAT ARE THE NEW PRODUCTS PROPOSED?

Page 17: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

E X C E L L E N C E

Sub-topic a) The reusability technologies must be NEW

Sub-topic b) The launch system advanced manufacturing and modern infrastructure must be applicable to European space transportation systems including mini and micro launch systems.

For both subtopics, the work must be readily integrated with current and planned European launch, test and production infrastructures. A preliminary implementation plan shall be included in the proposal.

1.3 State of the Art and Innovation potential

Page 18: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

1 . 4 Ap p r o p r i a t e c o n s i d e r a t i o n o f i n t e r d i s c i p l i n a r y a p p r o a c h e s a n d , w h e r e r e l e v a n t , u s e o f s t a k e h o l d e r k n o w l e d g e a n d g e n d e r d i m e n s i o n i n r e s e a r c h a n d i n n o v a t i o n c o n t e n t

• ARE RELEVANT INTER-DISCIPLINARY APPROACHES (I.E. THE INTEGRATION OF DATA, TECHNIQUES, TOOLS ORCONCEPTS FROM TWO OR MORE DISCIPLINES E.G. NATURALSCIENCES, TECHNOLOGY, ENGINEERING, ECONOMICS, SOCIALSCIENCES) CONSIDERED IN THE PROPOSED CONCEPT?

• USE OF STAKEHOLDER KNOWLEDGE

WHO ARE THE STAKEHOLDERS (E.G. POTENTIAL USERS ORCUSTOMERS OR OTHER GROUPS OR ORGANISATIONS WITH ANINTEREST IN THE RESULTS OF THE PROPOSED PROJECT)?

HOW DOES THE PROPOSAL DEMONSTRATE THAT THE CONSORTIUMWILL GET RELEVANT KNOWLEDGE FROM THEM (E.G. ORGANIZINGWORKSHOPS WITH USERS, DISCUSSING WITH CUSTOMERS, PARTICIPATING IN RELEVANT WORKING GROUPS ETC.)?

HOW WILL THIS KNOWLEDGE BE USED?

E X C E L L E N C E

Page 19: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Criter ion 2

- Impact

Page 20: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

• HOW MUCH WOULD THE PROPOSAL CONTRIBUTE TO EACH OF THE EXPECTEDIMPACTS MENTIONED IN THE WORK PROGRAMME?

(Here you should base your evaluation on the assumption that the proposed concept and methodology would work in order to avoid penalising the proposal twice for the same shortcomings.)

I M P A C T

2 . 1 T h e e x t e n t t o w h i c h t h e o u t p u t s o f t h e p r o j e c t w o u l d c o n t r i b u t e t o e a c h o f t h e e x p e c t e d i m p a c t s m e n t i o n e d i n t h e w o r k p r o g r a m m e u n d e r t h e r e l e va n t t o p i c

Page 21: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

- Improved European competitiveness in cost-effective reusability technologies for space transportation systems complementary with other European activities in the same domain leading to launch system reusability solutions which could be demonstrated in flight within three to five years, including forecast of investment needed.

- Increase compliance with the evolution of the regulatory framework, in particular REACH

- Improved quality and safety control solutions which are absolutely fundamental in rocketry.`

- Cost reduction of space transportation system manufacturing, integration, testing and operations thanks to research and innovation in advanced manufacturing and modernisation of infrastructures taking into account the evolution of the normative framework and environmental impacts. This is aimed at improving the overall competitiveness of European solutions on the worldwide market and helping create new jobs.

- Integration of human factors in ground facilities ,such as improved work environment for operators, efficiency, quality of operations and human safety.

Note: the 1st impact is more relevant to proposals answering sub-topic a) and the 4th and 5th impacts are more relevant to proposals answering sub-topic b)

I M P A C T – 2 . 1 I m p a c t s l i s t e d i n t h e w o r k p r o g r a m m e

Page 22: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

2 . 2 An y s u b s t a n t i a l i m p a c t s n o t m e n t i o n e d i n t h e w o r k p r o g r a m m e , t h a t w o u l d e n h a n c e i n n o v a t i o n c a p a c i t y , c r e a t e n e w m a r k e t o p p o r t u n i t i e s , s t r e n g t h e n c o m p e t i t i v e n e s s a n d g r o w t h o f c o m p a n i e s , a d d r e s s i s s u e s r e l a t e d t o c l i m a t e c h a n g e o r t h e e n v i r o n m e n t , o r b r i n g o t h e r i m p o r t a n t b e n e f i t s f o r s o c i e t y

• ARE THERE ANY OTHER SUBSTANTIAL IMPACTS NOT MENTIONED IN THE WORKPROGRAMME WHICH ARE DESCRIBED IN THE PROPOSAL AND ARE RELEVANT TOTHE ISSUES LISTED ABOVE?

(THE PROPOSAL SHOULD NOT BE PENALISED IF IT DOES NOT HAVE ADDITIONALIMPACT.)

I M P A C T

Page 23: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

• PLEASE ASSESS THE QUALITY OF THE "DRAFT PLAN FOR THEEXPLOITATION AND DISSEMINATION OF THE PROJECT'SRESULTS (PEDR)" WITH RESPECT TO THE SPECIFIC MEASURESTO BE IMPLEMENTED BOTH DURING AND AFTER THE END OFTHE PROJECT.

• IS THE PLAN TO DISSEMINATE PROJECT RESULTS TOAUDIENCES WHO MIGHT USE THE RESULTS IN THEIR OWNWORK (E.G. PRESENTATIONS IN CONFERENCES, PUBLICATIONSIN PEER REVIEW JOURNALS ETC.) APPROPRIATE?

• ARE ACTIVITIES TO COMMUNICATE PROJECT ACTIVITIES TOMULTIPLE AUDIENCES BEYOND THE PROJECT'S OWNCOMMUNITY (MEDIA, PUBLIC ETC.) INCLUDED? HOW ARETHESE TAILORED TO THE NEEDS OF VARIOUS AUDIENCES?

I M P A C T

2 . 3 Q u a l i t y o f t h e p r o p o s e d m e a s u r e s t o :- e x p l o i t a n d d i s s e m i n a t e t h e p r o j e c t r e s u l t s ( i n c l u d i n g

m a n a g e m e n t o f I P R ) , a n d t o m a n a g e r e s e a r c h d a t a w h e r e r e l e v a n t

- c o m m u n i c a t e t h e p r o j e c t a c t i v i t i e s t o d i f f e r e n t t a r g e t a u d i e n c e s

Page 24: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

• IS THE STRATEGY FOR THE MANAGEMENT OF IPR APPROPRIATE? DOES ITDESCRIBE THE MAIN BACKGROUND IPR NEEDED TO CARRY OUT THE PROJECT, ITSOWNERSHIP AND ACCESSIBILITY, AUTHORISATION TO USE THIRD PARTY RIGHTS, AND THE OWNERSHIP PRINCIPLES THAT WILL GOVERN THE PROJECT RESULTS?

• IF RESEARCH DATA IS GENERATED, DOES THE PROPOSAL IDENTIFY THE TYPES OFDATA THAT WILL BE GENERATED, THE STANDARDS THAT WILL BE USED, HOW THEDATA WILL BE EXPLOITED AND/OR SHARED/MADE ACCESSIBLE FOR VERIFICATIONAND RE-USE?

I M P A C T

2 . 3 Q u a l i t y o f t h e p r o p o s e d m e a s u r e s t o :- e x p l o i t a n d d i s s e m i n a t e t h e p r o j e c t r e s u l t s ( i n c l u d i n g

m a n a g e m e n t o f I P R ) , a n d t o m a n a g e r e s e a r c h d a t a w h e r e r e l e v a n t

- c o m m u n i c a t e t h e p r o j e c t a c t i v i t i e s t o d i f f e r e n t t a r g e t a u d i e n c e s

Page 25: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

I M P A C T

As regards the draft plan for exploitation and dissemination of the project results; for sub-topic a) the assessment of the exploitation measures should be made in conjunction with sub-criterion 1.2 Excellence –soundness of concept and credibility of the proposed methodology which requires the following:

Assessment of cost effectiveness for exploitationEstimation of investments and time schedule to reach TRL 8/9

Please provide consistent assessment with sub-criterion 1.2, and avoid penalising the proposal twice.

2.3 Quality of the proposed measures to exploit and disseminate the project results..

Page 26: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Criter ion 3 - Qual i ty

and eff ic iency of the implementat ion

Page 27: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

3 . 1 Q u a l i t y a n d e f f e c t i v e n e s s o f t h e w o r k p l a n , i n c l u d i n g e x t e n t t o w h i c h t h e r e s o u r c e s a s s i g n e d t o w o r k p a c k a g e s a r e i n l i n e w i t h t h e i r o b j e c t i v e s a n d d e l i v e r a b l e s

• HOW GOOD IS THE OVERALL STRUCTURE OF THE WORK PLAN?

• HAVE THE DIFFERENT WORK PACKAGES BEEN SCHEDULED AND LINKEDAPPROPRIATELY? ARE THEIR INTER-DEPENDENCIES CLEAR?

• ARE MILESTONES AND DELIVERABLES WELL DEFINED AND APPROPRIATELYTIMED?

• ARE THE WORK PACKAGES AND ASSOCIATED TASKS SUFFICIENTLY DESCRIBEDTO PROVIDE A GOOD UNDERSTANDING OF THE WORK INVOLVED AND JUSTIFYTHE PROPOSED RESOURCES TO BE ALLOCATED?

Q u a l i t y a n d e f f i c i e n c y o f t h e i m p l e m e n t a t i o n

Page 28: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Q u a l i t y a n d e f f i c i e n c y o f t h e i m p l e m e n t a t i o n

Does the proposal foresee activities to further detail its preliminary implementation plan for integrating the works with current and planned European launch, test and production infrastructure?(cf. sub-criterion 1.3 State of the art and innovation potential)

3.1 Quality and effectiveness of the work plan...

Page 29: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

3 . 2 Ap p r o p r i a t e n e s s o f t h e m a n a g e m e n t s t r u c t u r e s a n d p r o c e d u r e s , i n c l u d i n g r i s k a n d i n n o v a t i o n m a n a g e m e n t

• ARE THE DESCRIBED ORGANISATIONAL STRUCTURE AND DECISION MAKINGMECHANISMS APPROPRIATE TO THE SCALE AND COMPLEXITY OF THE PROJECT?

• ARE THE MANAGEMENT PROCEDURES, INCLUDING QUALITY MANAGEMENT ANDCONFLICT RESOLUTION, APPROPRIATE TO THE SCALE AND COMPLEXITY OF THEPROJECT?

• ARE CRITICAL INTERNAL AND EXTERNAL RISKS RELATED TO PROJECT IMPLEMENTATIONDESCRIBED WITH APPROPRIATE MITIGATION MEASURES AND SEVERITY/PROBABILITYESTIMATES?

• HAS INNOVATION MANAGEMENT BEEN APPROPRIATELY ADDRESSED IN THEMANAGEMENT STRUCTURE AND WORK PLAN (I.E. DOES THE PROPOSAL SHOW HOW ANYSCIENTIFIC OR TECHNOLOGICAL INVENTIONS OR ADVANCES WILL ACTUALLY BE TURNEDINTO RELEVANT CONCRETE INNOVATIONS WITH SPECIFIC PRACTICAL OR COMMERCIALADVANTAGES OVER EXISTING SOLUTIONS)?

Q u a l i t y a n d e f f i c i e n c y o f t h e i m p l e m e n t a t i o n

Page 30: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

3 . 3 C o m p l e m e n t a r i t y o f t h e p a r t i c i p a n t s a n d e x t e n t t o w h i c h t h e c o n s o r t i u m a s a w h o l e b r i n g s t o g e t h e r t h e n e c e s s a r y e x p e r t i s e

• HOW IS THE COMPLEMENTARITY OF THE PARTICIPANTS DESCRIBED?

• DOES THE CONSORTIUM BRING TOGETHER THE NECESSARY EXPERTISE FOR THEPROJECT?

Q u a l i t y a n d e f f i c i e n c y o f t h e i m p l e m e n t a t i o n

Page 31: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

3 . 4 Ap p r o p r i a t e n e s s o f t h e a l l o c a t i o n o f t a s k s , e n s u r i n g t h a t a l l p a r t i c i p a n t s h a v e a v a l i d r o l e a n d a d e q u a t e r e s o u r c e s i n t h e p r o j e c t t o f u l f i l t h a t r o l e

• DOES EACH PARTNER HAVE A VALID ROLE AND ADEQUATE RESOURCES (BOTHEFFORT AND BUDGET) TO FULFIL THAT ROLE?

• IF "OTHER DIRECT COSTS" EXCEED 15% OF PERSONNEL COSTS:

HAVE THEY BEEN EXPLAINED ADEQUATELY?

ARE THESE COSTS "REASONABLE, JUSTIFIED AND COMPLY WITH THE PRINCIPLE OFSOUND FINANCIAL MANAGEMENT, IN PARTICULAR REGARDING ECONOMY ANDEFFICIENCY"(ART. 6.1.)?

PLEASE, PARTICULAR ATTENTION SHOULD BE GIVEN FOR PROPOSAL WITH VERY HIGHOTHER DIRECT COSTS.

• IF ANY OF THE WORK WILL BE SUBCONTRACTED TO THIRD PARTIES, HASTHIS BEEN JUSTIFIED CONVINCINGLY?

Q u a l i t y a n d e f f i c i e n c y o f t h e i m p l e m e n t a t i o n

Page 32: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Quali ty of

the reports

Page 33: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

‘If this was my proposal,would I find this report fair, accurate, clear and

complete?’

… even if it brings bad news …

Quality of the reports

Page 34: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Avoid:Comments not related to the criterion in question.

Double counting of same weakness under different criteria.

Contradicting statements relative to strengths and weaknesses.

Comments too short / too long / inappropriate.

Categorical statements not properly verified and quoting texte.g. “The proposal doesn’t mention user requirements” – when there is a short reference…

Scores that don’t match the comments.

Making recommendations.

Evaluating on potential, proposals need to be evaluated as they are.

Discrimination/political incorrectness.

Quality of the reports

Page 35: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

As Evaluator you have to ensure that…

Comments in IERs and CRs are:• Specific to the relevant criterion• Clear and substantial• Facts, not opinions: "We think that…" "This proposal is… "• Consistent with the score, balancing strengths and

weaknesses• Of adequate length: not just one sentence, not a booklet!

Comments judge the proposal, they do not summarise it.

Do not provide advice on improving the proposal.

Good vs. poor comments

Quality of the reports

Page 36: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Poor comments merely echo the score

“The innovative aspects of the research programme are poor”.

Good comments explain it

“This proposal is not innovative in X or Y, and it does not take Z into account”.

Good comments are clear

“The resources in WP 4 and 6 are seriously underestimated given the complexity of the activity proposed”.

Poor comments are ambiguous

“The resources for the project are unrealistic”

Quality of the reports

Page 37: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Poor comments are vague, subject to interpretation

“We think the management plan is probably inadequate given the duration of the project and the number of partners”.

Good comments are precise and final

“The management plan is inadequate. It does not include clear risk assessment; it doesn't foresee a problem-solving mechanism in the event of disputes between partners”.

Quality of the reports

Page 38: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Poor comments are inaccurate and provide an opening for a complaint

“There is no discussion of a dissemination strategy.”

“There is only one industrial partner in the consortium.”

“The coordinator is not adequately experienced."

Good comments close the question

“Dissemination activities are listed but the proposal doesn't have a clear dissemination strategy”.

“The consortium doesn't include a sufficient industrial participation.”

“The coordinator does not demonstrate in the proposal an adequate level of experience in this field.”

Quality of the reports

Page 39: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Poor comments include words like…

PerhapsThinkSeems

AssumeProbably

Good comments include words like…

BecausePercent

SpecificallyFor example

Quality of the reports

Page 40: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Useful vocabulary

Insufficient, minimal, fails to describe, unacceptable, inadequate, very generic, not evident, unfocused, very weak, bad, does not meet the requirements, inappropriate, limited, unclear, not sound enough, not specified, no significant impact, unjustified, overestimated …

Extremely relevant, credible, very clear, precisely specified, realistic, very innovative, extremely well suited, timely, convincing, comprehensive, high quality, justified, very well identified, strong, highly effective, thoughtful, very promising, evidence, well-formulated, carefully prepared, very professionally prepared, fully in line, very profound, sound, very convincingly integrated, clearly articulated, coherent, well balanced, very plausible, ambitious, clear advances, well above average …

Quality of the reports

Page 41: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Process reminders

• Evaluators: accept the proposals assigned in SEP (by EOB Monday 23.4.)

- In case of non-acceptance provide a justification (e.g. CoI)

• Finalise and submit one individual evaluation report by Thursday 26.4.

- So that quality controllers can check the completeness of your report and give you feedback on whether issues were commented under appropriate subcriteria

- You will receive comments in your IER reports in SEP

- In some cases your report may be re-opened by REA staff so that you may complete it

• Complete the Remote Evaluation by Friday May 11th

- Necessary to allow the rapporteur to prepare a draft consensus report (CR) prior to the consensus meeting

• The draft CR will be available in the system a few days before the consensus meeting

- Read the draft CR prior to the meeting to prepare for the consensus discussion

Page 42: SPACE-16-TEC-2018 Access to space · SPACE-16-TEC-2018: Access to space Context † Indispensable element of the entire value chain of space, of strategic importance towards Europe’s

Thank you! In case of questions or problemsduring the individual evaluation,

do not hesitate to contactthe topic coordinator

(for technical issues: IT Helpdesk)

Disclaimer:This presentation is intended for information purposes only within the framework of the evaluation of the call H2020-SPACE-2018.Please refrain from sharing it with other parties not involved in this process or from making it publicly available.