enfin an experimental network for functional · pdf filean experimental network for functional...

Project No. LSHG-CT-2005-518254

ENFIN

An Experimental Network for Functional Integration

Instrument: Network of Excellence

Thematic Priority: LSH-2004-1.1.4-1

D7.2

Report on the across-analysis workshop

Due date of deliverable: 15.05.08

Actual submission date: 15.05.08

Start date of project: 13.11.2005 Duration: 60 months

Organisation name of lead contractor for this deliverable: EMBL-BIRNEY

Project co-funded by the European Commission within the Sixth Framework Programme (2002-2006)

Dissemination Level

PU Public X

PP Restricted to other programme participants (including the Commission Services)

RE Restricted to a group specified by the consortium (including the Commission Services)

CO Confidential, only for members of the consortium (including the Commission Services)

Project deliverable: ENFIN

2

Contributors

EMBL-EBI, CNIO, SPRI

INTRODUCTION

D7.2: Report on the across-analysis workshop

To pursue our efforts in assessing computational methods in systems biology, we have organized a workshop to present and discuss

the experience gained within different initiatives such as the ENFIN Consortium, CASP (Critical Assessment of Techniques for

Protein Structure Prediction) and DREAM (Dialogue for Reverse Engineering Assessments and Methods) groups. By far this is not an

exclusive list of groups and we are pursuing the identification of potential partners to share our experience, success and failures with.

Methods

The workshop called “ENFIN-DREAM” was hosted by Alfonso Valencia at the CNIO (Madrid, Spain) on April 28th - 29

th, and the

program Committee included the organizers of the previous DREAM2 meeting, G. Stolovitzky (IBM T.J. Watson Research Center,

New York, USA) and A. Califano (Columbia University Medical Center, New York, USA), as well as ENFIN partners Ioannis

Xenarios (SPRI / Swiss Institute of Bioinformatics), Alfonso Valencia (CNIO) and Pascal Kahlem (EMBL-EBI).

The proceedings of this workshop will be published with those of the DREAM2 meeting in an upcoming volume of the Annals of the

New York Academy of Sciences.

Most of the presentations of the meeting can be found on the ENFIN website, in the section meetings:

http://www.enfin.org/page.php?page=workshops

The program and the list of abstracts can be found in Annex.

DREAM: Dialogue for Reverse Engineering Assessments and Methods. http://wiki.c2b2.columbia.edu/dream/index.php/The_DREAM_Project DREAM is sponsored by Columbia University MAGNet Center, the NIH Roadmap Initiative and the IBM Computational Biology Center.

Results (if applicable, interactions with other workpackages)

ENFIN collaborations of computational predictions and experimental validations were presented, raising a discussion on the technical

limitations of experimental approaches to significantly validate the computational approaches. It was shown that an increase of the

confidence level of the prediction could be gained by the integration of several computational approaches (e.g. mitotic spindle protein

predictions).

Experiences from the DREAM2 challenges were presented by Gustavo Stolovitzky, including the gold standard used to compare the

results for each challenge.

Challenges were aimed at reconstructing gene networks. One of the challenges, for example, had consisted in retrieving the correct set

of genes being targets of the transcription factor BCL6, after adding this set of genes amongst a large number of other genes (namely

decoy genes).

The presentation showed that in most challenges, with a few exceptions, most computational methods had worked rather poorly

(prediction close to random), which raised concerns with regard to the standard use of these methods in research. For confidentiality

reasons, the authors of the methods were not divulgated.

The organizers of the DREAM initiative plan to increase the scope of the challenges by adding new types of data, such as

phosphorylation states or proteomics profiles, and will request the competitors to predict not only qualitative results, but also

quantitative. One additional experimental datasets are measurement of perturbed biological systems, which is similar in approach to

the TGF-beta and the diauxic shift ENFIN modelling projects. This should be closely monitored so that synergies could be identified. An overview of the last CASP challenge was presented by Ana Tramontano. The presentation showed the methods used to assess the predictions of protein structure in comparison to the true structure obtained by crystallography. The last CASP challenged included the possibility to predict, beyond the protein structure, the function of the protein. Unfortunately, the too few predictions per target did not allow deriving any sensible conclusion.

Project deliverable: ENFIN

3

Perspectives To continue the collaboration with the DREAM initiative, ENFIN intends to provide a dataset for the challenge DREAM 2009.

Publications (if applicable)

Proceedings of the meeting will be published in the Annals of the New York Academy of Sciences in the course of this year 2008.

The ENFIN project is funded by the European Commission within its FP6 Programme, under the thematic area

"Life sciences, genomics and biotechnology for health”; Contract number LSHG-CT-2005-518254

DREAM is sponsored by Columbia University MAGNet Center, the NIH Roadmap Initiative and the IBM Computational Biology Center.

ENFIN – DREAM Conference

Assessment of Computational Methods in Systems Biology April 28 – 29, 2008

www.enfin.org

Spanish National Cancer Research Centre (CNIO) (Auditorium)

C/ Melchor Fernández Almagro, 3, E-28029 Madrid + (34) 917 328 000 + (34) 912 246 900

Madrid, Spain

Organisers: Alfonso Valencia - [email protected]

Ana Rojas Mendoza [email protected]

Pascal Kahlem - [email protected] (Mob. +49 15209854747)

Scope:

The European Network of Excellence ENFIN develops infrastructure, tools and methods to enhance

Integrative Systems Biology in Europe. The project addresses three fields of research i) discrete function

prediction, ii) network reconstruction, iii) systems level modeling. One concept of the Network is the

strong collaboration between dry and wet laboratories, which cycle data between computational predictions

and experimental validations.

Although wet experiments are used to validate chosen computational predictions, they often do not allow

assessing the quality of computational methods, because of limited scale, technology and resources.

Because of the growing number of bioinformatics tools available in Systems Biology, strategies are needed

to assess the accuracy of the computational predictions.

In collaboration with the DREAM project, we organize the first European ENFIN-DREAM Conference.

Participants of ENFIN along with other researchers will present strategies to assess methods in the field of

Systems Biology.

ENFIN: Experimental Network for Functional Integration. http://www.enfin.org

DREAM: Dialogue for Reverse Engineering Assessments and Methods.

http://wiki.c2b2.columbia.edu/dream/index.php/The_DREAM_Project

Important dates:

Jan 31, 2008: Papers submission deadline

March 1, 2008: Papers acceptance issued

March 15, 2008: Registration deadline




Speakers

EMBL-European Bioinformatics Institute, Hinxton, UK

Ewan Birney ([email protected]) Technical University of Denmark, Lyngby, Denmark

Soren Brunak ([email protected])

University College London, London, UK

Christine Orengo ([email protected]) QureTec, Tartu, Estonia

Jaak Vilo ([email protected])

CERTH, Thessaloniki, Greece

Christos Ouzounis ([email protected]) Vital-IT group, Swiss Institute of Bioinformatics, Lausanne, Switzerland

Ioannis Xenarios ([email protected])

Genoscope, Evry, France

Vincent Schachter ([email protected]) Spanish National Cancer Research Centre, Madrid, Spain

Alfonso Valencia ([email protected])

IBM T.J. Watson Research Center, New York, USA

Gustavo A. Stolovitzky ([email protected]) Columbia University Medical Center, New York, USA

Andrea Califano ([email protected]) Telethon Institute of Genetics and Medicine (TIGEM), Naples, Italy

Diego di Bernardo ([email protected])

University of Rome "La Sapienza" - Department of Biochemical Sciences "Rossi Fanelli", Italy

Anna Tramontano ([email protected])

Department Biochemistry & Molecular Biology (Procel lab), University of Malaga, Spain

Ian Morilla ([email protected])

University of Padova, Italy

Alberto Corradin ([email protected])

INSERM ERM 206, Marseille, France

Denis Thieffry ([email protected])

Programme Committee

Gustavo A. Stolovitzky ([email protected])

Andrea Califano ([email protected])

Alfonso Valencia ([email protected]) Ioannis Xenarios ([email protected])

Pascal Kahlem ([email protected])




Day 1: April 28 12:00 – 13:00 Arrivals - Registrations

13:00 – 13:30 E. Birney: Introduction

SESSION 1: Protein function prediction

13:30 – 14:00 C. Orengo “Gene3D: Integrating complex data to reveal protein networks”

14:00 – 14:30 I. Morilla Dominguez “Biomathematical improvement of protein high-throughput

functional prediction”

14:30 – 15:00 V. Schächter "Assessment of metabolic models"

Coffee Break

SESSION 2: Network reconstruction

15:30 – 16:00 J. Vilo “Usage of gene expression data for pathway reconstruction”

16:00 – 16:30 A. Corradin “In silico assessment of four reverse engineering algorithms: role of network

complexity and multi-experiment design in network reconstruction and hub detection”

16:30 – 17:00 C. Ouzounis "Evolutionary analysis of biological pathways: implications for curation and

inference"

Coffee Break

SESSION 3: Systems-level modeling

17:30 – 18:00 I. Xenarios “A qualitative modeling approach of TNF-alpha / TGF-beta regulatory

network: Advantages and Limitations”

18:00 – 18:30 D. Thieffry “Logical modelling and analysis of biological regulatory networks:

identification of stable states and feedback circuit analysis”

18:30 – 19:00 A. Valencia "Text mining and assessment of computational methods in Systems Biology"

Day 2: April 29

SESSION 4: Learning from DREAM and CASP

9:00 – 9:30 Andrea Califano “Integrated, biochemically validated molecular interaction networks

reveal master regulators of human malignancies”

9:30 – 10:00 Gustavo Stolovitzky “Dialogue on Reverse Engineering Assessment and Methods: the

DREAM of high throughput pathway inference”

10:00 – 10:30 Diego Di Bernardo “Reverse engineering gene network in genetic diseased and drug

discovery”

Coffee Break

11:00 – 11:30 Anna Tramontano “The CASP experiment: opportunities and pitfalls”

11:30 – 12:00 S. Brunak “Prediction of Protein Categories from Sequence“

12:00 End of meeting




Abstracts

Gustavo Stolovitzky

“Dialogue on Reverse Engineering Assessment and Methods: the DREAM of high throughput pathway

inference. “

The biotechnological advances of the last decade have confronted us with an explosion data that need to be

organized and structured before they may provide a coherent biological picture. To accomplish this task,

the availability of an accurate map of the physical interactions in the cell that are responsible for cellular

behavior and function would be exceedingly helpful, as these data are ultimately the result of such

molecular interactions. However, all we have at this time is partially correct representation of the

interactions between genes, their byproducts, and other cellular entities. DREAM, the Dialogue on Reverse

Engineering Assessment and Methods, is fostering a concerted effort by computational and experimental

biologists to understand the limitations and enhance the strengths of the efforts to reverse engineering

cellular networks from high throughput data. In this talk I will discuss the salient arguments of the recent

DREAM2 conference, where we challenged the community to blindly infer networks known to the

organizers from high throughput data. I will highlight the strategies that have achieved the better inference

results and discuss the state of the art in Reverse Engineering, as well as some of the challenges and

opportunities awaiting us.

Anna Tramontano

“The CASP experiment: opportunities and pitfalls”

In 1994, John Moult proposed a world-wide experiment named CASP aimed at establishing the current

state of the art in protein structure prediction, identifying what progress has been made, and highlighting

where future effort may be most productively focused.

Experimental structural biologists who are about to solve a protein structure are asked to make the

sequence of the protein available, together with a tentative date for the release of the final coordinates. In

the recent past, structural genomics consortia have significantly contributed to the set of CASP targets.

Predictors produce and deposit models for these proteins before the structures are made public. Finally, a

panel of three assessors compares the models with the structures as soon as they are available and tries to

evaluate the quality of the models and to draw some conclusions about the state of the art of the different

methods. The experiment is run blindly, that is, the assessors do not know who the predictors are until the

very end of the experiment.

Each of the routes to the prediction of a protein structure commonly used has traditionally been mirrored by

a CASP “category”, evaluated by one of the three assessors. The results of the comparison between the

models and the target structures are discussed in a meeting where assessors and predictors convene. The

conclusions are made available to the whole scientific community through the World Wide Web and

through the publication of a special issue of the journal “Proteins: Structure, Function, and Bioinformatics”.

The method for assessing protein structure predictions in CASP has developed throughout the years

becoming very professional and somewhat standardized, however several other categories have been

introduced in CASP throughout the years, such as prediction of function, domain boundaries, disordered

regions, and model quality and each of them has introduced novel problems that required "ad hoc"

solutions that I will discuss.

A. Di Cara1, L. Mendoza2, A. Garg3, G. Di Michieli3, I. Xenarios4

1 Merck-Serono, Geneva, Switzerland




2 UNAM, Mexico City, Mexico

3 EPFL, Lausanne, Switzerland

4 Swiss Institute of Bioinformatics /Vital-IT, Lausanne, Switzerland

“A qualitative modeling approach of TNF-alpha / TGF-beta regulatory network: Advantages and

Limitations”

The understanding of the dynamical behavior of any biological system is a holy grail of systems biology.

Within the ENFIN network (www.enfin.org) our aim is to provide methodologies to a wide variety of

“wet” and “dry” scientists to tackle that important challenge.

As a test case we started to study the interplay between TNFa!and TGFb!pathways, by modeling these

networks and identifying key molecular regulators. We use a qualitative boolean modeling approach. This

modeling technique only requires the topology of the interactions and their net effect defined as activation

or inhibition. There is no need to accumulate vast amount of kinetic data (at this stage) and then fit these

onto the model.

Our aim is to use this model as a guide to identify key “wet” experiments to perform. The drug discovery

process could benefit from using this model for biomarker discovery and mode of action studies.

Our approach to study the TNFa!and TGFb!pathway interactions comprised four steps: (I) Model building of

the two interconnected pathways which consists of currently 26 components, extracted from experimental

literature with emphasis on the identification of feedback loops. (II) Using a generalized logical analysis1

method we identified steady state(s) of our network. (III) dynamical simulations where we perturbed the

steady states with TNFa, TGFb!or both ligands. (IV) Validation of the model in which TNFa!and TGFb!are

added simultaneously. Here we observe a dominance of the TGFb!pathway, which is in accordance with

experimentally derived data in a dendritic cell context.

Altogether our results show that using this modeling approach we are able to recapitulate the crosstalk

between the TNFa!and TGFb!pathways and identify key components involved in the functional behavior of

these two signaling networks. The final step of our modeling is to experimentally test some of our

predictions that shed some light on novel cellular behavior.

1L. Mendoza, I. Xenarios, Theor Biol Med Model, 2006 Mar, 16;3:13

2 F. Geissmann, P. Revy et al., Jour. Immun., 1999, 162: 4567-4575

Christos Ouzounis

"Evolutionary analysis of biological pathways: implications for curation and inference".

The analysis of metabolic and signaling pathways will be presented in a few case studies, covering a wide

range of phylogenetic distances, from archaea and bacteria to vertebrates. These studies have revealed both

the remarkable conservation of core metabolism and the surprising diversification of signaling cascades.

Methods that accurately detect enzyme specificity will also be presented and the limitations of similar

approaches for effector specificity will be examined. Finally, possible implications for curation and

inference of biological pathways will be discussed.

Audit, B., Levy, E.D., Gilks, W.R., Goldovsky, L. and Ouzounis, C.A. (2007) CORRIE: enzyme sequence

annotation with confidence estimates. BMC Bioinformatics, 8 Suppl 4, S3.

Ouzounis, C.A. and Karp, P.D. (2000) Global properties of the metabolic map of Escherichia coli. Genome

Res., 10, 568-576.

Peregrin-Alvarez, J.M., Tsoka, S. and Ouzounis, C.A. (2003) The phylogenetic extent of metabolic enzymes

and pathways. Genome Res., 13, 422-427.

Tsoka, S. and Ouzounis, C.A. (2001) Functional versatility and molecular diversity of the metabolic map of

Escherichia coli. Genome Res., 11, 1503-1510.

Tsoka, S., Simon, D. and Ouzounis, C.A. (2003) Automated metabolic reconstruction for Methanococcus

jannaschii. Archaea, 1, 223-229.

von Mering, C., Zdobnov, E.M., Tsoka, S., Ciccarelli, F.D., Pereira-Leal, J.B., Ouzounis, C.A. and Bork, P.

(2003) Genome evolution reveals biochemical networks and functional modules. Proc. Natl. Acad. Sci.




USA, 100, 15428-15433.

Diego Di Bernardo

“Reverse engineering gene network in genetic diseased and drug discovery”

We will present a reverse engineering method to identify gene networks from multiple measurements of

gene expression at steady-state following single-gene perturbation, or from time-series data following a

single perturbation experiment. We will then present their application to a synthetic gene network we built

in the yeast S. cerevisiae. We will conclude with a brief comparison we made between the performance of

different reverse engineering methods, and how we measured the performance.

Hedi Peterson, Jaak Vilo

“Usage of gene expression data for pathway reconstruction”

We will describe a study that investigates the suitability of expression data for pathway reconstruction,

using a variety of pathways. In light of these experiments we can hypothesize on the chances of discovering

new "missing" members of the pathways. We are currently exploring any significant relationships between

the datasets picked out by our method and their biological significance to the pathway in question; for

example, if the pathway in question is related to apoptosis, are the cancer datasets the best descriptors for

this pathway?

We validate the effectiveness of our methodology on Reactome pathways by leave-one-out cross-validation

experiments. During validation a gene from the known pathway is deliberately excluded during the

development of our model, and then used as a test point to validate the model. We will use our results to

determine those pathways for which it is easier to predict new members using gene expression. Finally we

will suggest the possible biological reasons behind the given conclusions.

Ian Morilla (1), Adam Reid (2), Corin Yeats (2), Jonathan Lees (2), Christine Orengo (2), Juan A. G.

Ranea (1)

“BIOMATHEMATICAL IMPROVEMENT OF PROTEIN HIGH-THROUGHPUT FUNCTIONAL

PREDICTION”

(1) Department Biochemistry & Molecular Biology (Procel lab), Faculty of Sciences, University of

Malaga, UMA, Spain

(2) Department of Biochemistry and Molecular Biology, University College London, London WC1E 6BT,

UK

The aim of the most high-throughput experiments is to discover new molecules functionally associated to

particular biological systems or processes. So a large of biological datasets are generated from any of the

current proteomic or genomic experiments. The individual sequences are usually identified and functionally

annotated by single homology searches run on the extant available sequence databases. But this is just a

first simple approach for allowing to validate the experiments (true or false positives). Unfortunately, in

many cases, these high-throughput experiments show a poor overall performance with high rates of false

positive and false negative hits. Therefore a complementary bioinformatics treatment is required to obtain a

more reliable functional prediction.

Systems biology is provided with various high-throughput technologies in order to analyze global

biological datasets. Each of these technologies has different levels of accuracy and system coverage

(statistical power) and reflect error rates when are used individually. These error rates can be reduced by

integrating multiple datasets from different high throughput technologies and so capture complementary

information. In this work we are right integrating the Fused Domain (FD), Inherited Protein-Protein

interaction data (hiPPI), GEC(gene expression profile comparison) and Semantic Similarity (SS)

bioinformatics methods. We have chosen Fisherís weighted and non weighted methods and Shannon

information theory [2] for integrating predictions. In this way, we deal effectively with the difference in




statistical power, thus reducing the error rates and improving predictions benchmarked on the yeast

proteome annotated by GO (Gene Ontology database [3,4]).

[1] Hwang D. et al. (2005) A data integration methodology for systems biology. PNAS 102(48), 17296-

17301.

[2] Shannon C. E. (1948) A Mathematical Theory of Communication. The Bell System Technical Journal,

Vol. 27, pp. 379-423, 623-656.

[3] Zeeberg, B.R., Feng, W., Wang, G., Wang, M.D., Fojo, A.T., Sunshine, M., Narasimhan, S., Kane,

D.W., Reinhold, W.C., Lababidi, S., et al. (2003) GoMiner: a resource for biological interpretation of

genomic and proteomic data. Genome Biol., 4, R28.

[4] The Gene Ontology Consortium. (2000) Gene Ontology: tool for the unification of biology. Nature

Genet, 25, 25-29.

Alberto Corradin, Barbara Di Camillo, Gianna Maria Toffolo, Claudio Cobelli.

“In silico assessment of four reverse engineering algorithms: role of network complexity and multi-

experiment design in network reconstruction and hub detection”.

Background.

An important problem in systems biology is the inference of regulatory networks from gene expression

data. Several reverse engineering methods have been proposed in the literature, among them linear and

nonlinear Dynamic Bayesian Networks, DBNs, (Ferrazzi et al., 2007), ARACNe (Margolin et al., 2006)

and Graphical Gaussian Models, GGMs, (Schafer and Strimmer, 2005). These algorithms use different

approaches: linear and nonlinear DBNs are model-based methodologies, whereas ARACNe and GGMs

exploit pair-wise profile comparison based on mutual information and partial correlation, respectively.

Since no biological network is understood well enough to serve as a standard, reverse engineering methods

are usually assessed in silico. Recently a novel simulator, which takes into account topological properties

(e.g. scale-free connectivity), interactions among regulators and complex dynamics, has been developed

(Di Camillo et al., 2006). The purpose here is to assess the ability of the four reverse engineering methods

to reconstruct network topology and to detect hubs.

Methods.

300 different network topologies were generated with number of genes N=12, 20 or 100. For each of them,

gene expression datasets were simulated starting from S= 10, 3 or 1 different initial conditions, each

corresponding to an experiment. For each condition, M=50, 10, 5 or 2 time samples were collected.

Algorithms were tested both with and without gaussian noise (mean=0, SD=1 corresponding to

CV≥10%), but reported results only refer to noisy data.

Algorithms were scored using F-measure (Ferrazzi et al., 2007), which combines sensitivity S and positive

predictive value PPV: F=2*S*PPV/(S+PPV).

Statistically significant differences were assessed using exact Wilcoxon tests.

Results.

All methods perform poorly with networks having N=100 genes (F<40%). With lower N, linear DBNs

perform best in data-rich situations (S=10, M=50), e.g. with N=12, F=0.62± 0.11 (mean±SD). However,

DBNs performance deteriorates with lower M (e.g. with N=12: F=0.54± 0.12, F=0.39± 0.11, F=0.23±0.09

for M=10, 5, 2, respectively), whereas ARACNe is less sensitive to M (N=12: F=0.53±0.10, F=0.52±0.07,

F=0.51± 0.11, F=0.43±0.16 for M=50, 10, 5, 2, respectively) and performs best in data-poor situations.

In our simulations, GGMs and nonlinear DBNs are always overcome by the other algorithms.

All methodsí results deteriorate with lower S (e.g. with N=12, M=50, S=3: F= 0.47±0.13 with linear DBNs,

which scores highest), reaching poor performance (F<40%) when S=1. The four methods perform similarly

in hub detection: the F-measure of the connections of identified hubs is reasonable (F=0.63±0.14 with

ARACNe, which performs best when N=20, S=10, M=50), but the number of hubsí connections is

underestimated.

Conclusions.




Reverse Engineering methods are able to formulate reliable hypothesis about networks with a limited

number of genes if their expression is adequately monitored during multiple experiments, so to excite

different states of the system.

The method of choice depends on the number of samples: linear DBNs outperform in data-rich situations

while ARACNe in data-poor ones.

Denis Thieffry

“Logical modelling and analysis of biological regulatory networks: identification of stable states and

feedback circuit analysis”

The complexity of biological regulatory networks calls for the development of proper mathematical

methods to model their structures and to obtain insight in their dynamical behaviours. One qualitative

approach consists in modelling regulatory networks in terms of logical equations, using Boolean or multi-

level variables.

Recently, we have proposed a novel implementation of the multi-level logical modelling approach by

means of Multi-valued Decision Diagrams This representation enabled the development of two efficient

algorithms for the dynamical analysis of parameterised regulatory graphs. A first algorithm allows the

identification of all stable states without generating the state transition graph. A second algorithm assesses

the conditions insuring the functionality of the feedback circuits found in the regulatory graph.

These algorithms have been implemented into a novel development version of our logical modelling

software GINsim. Their application to logical models of T cell activation and differentiation will be briefly

presented.




Local Information

To find hotels near venue, please visit:

http://www.cnio.es/eventos/listhotels.asp

For directions, please visit: http://www.cnio.es/ing/comollegar.asp

enfin an experimental network for functional · pdf filean experimental network for functional...

Documents