Evaluation BBMRI - (preliminary) results

BBMRI Stakeholder’s Forum: Patient Working GroupParis15 December 2009

2

Introduction

• Project for WP1/WP7...1. to carry out an ex-ante impact assessment of BBMRI2. to set up a coherent monitoring and evaluation strategy for BBMRI

• Approach of the studyAd 1: Literature reviewAd 1: 10 Case studies - combinedAd1: Mapping and clustering of biobanks - SNA• Preliminary conclusionsAd 2: Logical Framework analysisAd 2: Indicator development• Validation by stakeholders

• Timeline: January - December 2009

3

Case studies - selection

• Estonia BioBank - population based• DeCode - population based• Biobank Castilla-Léon - case/control oncology• TransBig - case/control oncology (Mammaprint)• Telethon - case/control rare disease network• EuroBioBank - case/control rare disease network• UDBN - case/control network• ENGAGE - data network• Graz Biobank - population based• Huddinge Brain bank - case/control neurology

4

Case studies results combined (1) - Time/maturity

• Majority started 5-7 years ago • Majority is nationally organised

< 2000 >2000<2005 >2005

3 (DeCode, Graz, Huddinge)

4 (EGF, UDBN, EBB, TBCyL)

3 (Telethon, Engage,

TransBig)

Regional National European/Inter-national

1 (TBCyL) 5 (DeCode, EGF, UDBN, Graz, Huddinge)

3 (EBB, Engage, TransBig)

5

Case studies results combined (2) - Numbers

• Collection - big spread: 1,100 - 4,400,000• Distribution - mainly unclear, only when central access is

available

Collection < 10,000 10,000-100,000 >100,000

3 (Huddinge, TransBig, TBCyL)

3 (UDBN, EGF, Telethon)

4 (Graz, EBB, Engage, DeCode

Distribution ? 0-1000 >1000

5 (DeCode, EGF, Graz, Engage,

Huddinge)

2 (TransBig, TBCyL)

3 (UDBN, Telethon, EBB)

6

Case studies results combined (3) - Organisation

• Centralised repositories vs distributed• Research - networks - infrastructure

Central Distributed Mixed

6 (DeCode, EGF, Huddinge, Graz, Transbig, TBCyL)

3 (Telethon, EBB, Engage)

1 (UDBN)

Research Soft - networks Infrastructure

3 (TransBig, TBCyL,

Huddinge)

3 (Telethon, EBB, Engage)

4 (EGF, DeCode, Graz, UDBN)

7

Case studies results combined (4) - Outreach activities

Outreach

Score 1 Score 2 Score 3 Industry 5 (EGF, BTCyL,

EBB, Engage, Huddinge)

2 (Telethon, UDBN)

3 (deCODE, TransBig, Graz)

Society 3 (deCODE, Engage, Huddinge)

4 (BTCyL, TransBig, UDBN,

Graz)

3 (EGF, EBB, Telethon)

• Majority not very active towards industry• To society it is slightly better

8

Case studies results combined (5) - Funding patterns • Mixed funding models

- personnel/infrastructure: public (institution)- research: public (grants)

• Biobanking long-term investment in infrastructure• The start-up phase most expensive• Personnel highest costs, both at the start and in maintenance• Hidden costs that are not made explicit

• Networking reduces costs (Telethon) - saved 2/3• Funding from industry doesn't work at the start (objectives

diverge), but neither in the long run • Not until public funding is received the biobank activities are

getting sustainable (Graz, Estonia)• Difficult to find core (sustainable) funding• Difficult to find public funding since it is time-limited

9

Social Network analysis – Why and how

• Interconnectedness crucial element in the development of European biobanking

• How can we identify the nature and evolution of links?: “Social Network Analysis” techniques : Inter-organisational links

• Source: projects identified as biobanking related projects by the "Networking meeting for EU-Funded Biobanking Projects" Report (2008) and the Wellcome Trust report "From Biobanks to biomarkers" (2006); Data collected from CORDIS • Red square is an institution that is not BBMRI• Green triangle is BBMRI member• Round green is BBMRI associate• Size of node is proportional to number of projects an institution has

participated in• The thickness of the line is proportional to the number of projects they

have co-participated• FP6: only institutions that are participating in more than one project

10

Social Network analysis - result FP5

11

Social Network analysis - result FP6

12

Social Network analysis - organisations

BBMRI org FP5 BBMRI org FP6 INSERM INSERM INSERM INSERM

KI KAROLINSKA INSTITUTET KI KAROLINSKA INSTITUTET

APHP ASSISTANCE PUBLIQUE - HOPITAUX DE PARIS LEIDENMC LEIDEN UNIVERSITY MEDICAL CENTER MTCC

MUNICH UNIVERSITY NIEH FODOR JOZSEF NATIONAL CENTER

FOR PUBLIC HEALTH BESTA ISTITUTO NAZIONALE NEUROLOGICO CARLO BESTA ERASMUSMC ERASMUS MEDICAL CENTER HGDP-CEPH FONDATION JEAN DAUSSET MRC MEDICAL RESARCH COUNCIL GENETHON GENETHON LEICESTER UNIVERSITY OF LEICESTER CNBBSV ISTITUTO SUPERIORE DI SANITA LUNDU LUND UNIVERSITY LJUBLJANA UNIVERSITY OF LJUBLJANA UTARTU UNIVERSITY OF TARTU LUNDU LUND UNIVERSITY UU UPPSALA UNIVERSITET CHARITE CHARITE MTCC MUNICH UNIVERSITY NAPLES2 SECONDA UNIVERSITA DEGLI STUDI DI NAPOLI IMPCOL IMPERIAL COLLEGE Non-BBMRI Non-BBMRI GERCAN GERMAN CANCER RESEARCH CENTER OXFORDU UNIVERSITY OF OXFORD FINNISHCR FINNISH CANCER REGISTRY UCL UNIVERSITY COLLEGE LONDON KARLS EBERHARD KARLS UNIVERSITY OF TÜBINGEN UHELSINKI UNIVERSITY OF HELSINKI DANISHCS DANISH CANCER SOCIETY CNRS UHELSINKI UNIVERSITY OF HELSINKI KING'S COLLEGE LONDON

13

Overall (preliminary) conclusions (1)

Time/maturity• Before 2000, biobanking wasn’t really recognized as a concept • The time it takes before a biobank becomes fully operational is

decreasing (speed increases)

Networking• There is a tendency towards bigger, international networks, and

therefore bigger collections/number of samples• From the SNA, you can see that the core BBMRI members already

had a central position in FP5• In FP6 you can see a boost in biobanking activities by the number of

projects and number of institutions involved: periphery is growing• From FP6 SNA, you can see that there are still a considerable

number of institutions involved in biobanking that need to be included in BBMRI

14

Overall (preliminary) conclusions (2)

Organisation• All biobanks display elements of research, networking and

infrastructure;• Research is an ‘early’ activity, • Networking and infrastructure need time to establish• Networks, as a ‘soft’ infrastructure, are designed to connect

distributed repositories• 4 of the central repositories have evolved into mature, ‘hard’

infrastructures• Distribution is related to the issue of access

• The fact that half the cases don’t know how many samples are transferred, can be taken as a sign of internal orientation - are they really opening up?

• When we find carefully managed distribution strategies, we can link it to funding cultures

15

Overall (preliminary) conclusions (3)

Funding• You need at least one funding period to build a biobank• In the transition from research to infrastructure, the funders should

address the issue of funding maintenance of infrastructure• Funding mechanisms in Europe are not suitable for maintenance of

biobanks

Outreach• In general, the external activities are not overwhelming; biobanks are

usually more geared towards research or operational affairs, rather than towards socio-economic engagement

• Having said that, outreach activities are particularly noticeable in some types of biobanks• Population based biobanks > society• Case control biobanks > industry• Rare disease oriented biobanks > society• Immature or young biobanks > little external activities as yet

16

Indicator development

Short term outputs - internal and operational side of biobanks/BBMRI• Quality management procedures, IT infrastructures, ELSI

procedures, User access agreements, Search engines, Common tools and services

• Adoption of ERI concept

Middle term outcomes - establishment of hard infrastructure/networks, • Increased availability of samples; increased research

collaboration; increased use of biobanks and the payment of fees related to this use;

• ERI contracts

• Start of outreach activities to comply with external strategy

17

Indicator development

Long term impacts - sustainable funding/adequate business model• Medical applications in diagnostics, biomarkers, drugs, and the

economic and health benefits of these applications, including new therapies, new diagnostics, personalized medicine:

• Outreach activities to the appropriate professionals, patients and policy makers

• Long term collaborations with industry; New Life Sciences companies providing biobank related services

Contextual impacts - not so easy to make them explicit: you only miss them when you don’t do it

• Sustainable savings in the execution of biomedical research - funders

• Increased speed in building networks - USA

• A clear governance structure without which it isn’t possible to comply with future regulatory and technological frameworks

18

Thank you

Pauline MattssonIngeborg MeijerJordi Molas GallartAnke Nooyen

Technopolis Group has offices in Amsterdam, Ankara, Brighton, Brussels, Paris, Stockholm, Tallinn and Vienna.