Genetic Programming Theory and Practice I1

GENETIC PROGRAMMING SERIES

Series Editor

John Koza Stanford University

Also in the series:

GENETIC PROGRAMMING AND DATA STRUCTURES: Genetic Programming + Data Structures = Automatic Programming! William B. Langdon; ISBN: 0-7923-8 135- 1

AUTOMATIC RE-ENGINEERING OF SOFTWARE USING GENETIC PROGRAMMING, Conor Ryan; ISBN: 0-7923-8653-1

DATA MINING USING GRAMMAR BASED GENETIC PROGRAMMING AND APPLICATIONS, Man Leung Wong and Kwong Sak Leung; ISBN: 0-7923-7746-X

GRAMMATICAL EVOLUTION: Evolutionary Automatic Programming in an Arbitrary Language, Michael O'Neill and Conor Ryan; ISBN: 1-4020- 7444- 1

GENETIC PROGRAMMING IV: Routine Human-Computer Machine Intelligence, John R. Koza, Martin A. Keane, Matthew J. Streeter, William Mydlowec, Jessen Yu, Guido Lanza; ISBN: 1-4020-7446-8

GENETIC PROGRAMMING THEORY AND PRACTICE, edited by Rick Riolo, Bill Worzel; ISBN: 1-4020-758 1-2

AUTOMATIC QUANTUM COMPUTER PROGRAMMING: A Genetic Programming Approach, by Lee Spector; ISBN: 1-4020-7894-3

The cover art was created by Leslie Sobel in Photoshop from an original photomicrograph of plant cells and genetic programming code. More of Sobel's artwork can be seen at www.lesliesobel.com.

Genetic Programming Theory and Practice I1

Edited by

Una-May 0' Reilly Massachusetts Institute of Technology

Tina Yu Chevron Texaco Information Technology Group

Rick Riolo University of Michigan

Bill Worzel Genetics Squared, Inc.

Ql - Springer

Library of Congress Cataloging-in-Publication Data

A C.I.P. Catalogue record for this book is available from the Library of Congress.

O'Reilly, U. Genetic Programming Theory and Practice I1 I edited by Una-May O'Reilly, Tina Yu, Rick Riolo,

Bill Worzel p.cm.

ISBN 0-387-23253-2 e-book 0-387-23254-0 Printed on acid-free paper.

O 2005 Springer Science+Business Media, Inc. All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, Inc., 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now know or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if the are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.

Printed in the United States of America.

9 8 7 6 5 4 3 2 1 SPIN 11312086 (HC)

Contents

Contributing Authors

Preface

Foreword

1 Genetic Programming: Theory and Practice Una-May O'Reilly, Tina Yu, Rick Riolo and Bill Worzel

2 Discovering Financial Technical Trading Rules Using Genetic Program-

ming with Lambda Abstraction Tina Yu, Shu-Heng Chen and Tzu- Wen Kuo

3 Using Genetic Programming in Industrial Statistical Model Building Flor Castillo, Arthur Kordon, Jeff Sweeney and Wayne Zirk

4 Population Sizing for Genetic Programming Based On Decision-Making Kumara Sastry, Una-May O'Reilly and David E. Goldberg

5 Considering the Roles of Structure in Problem Solving by Computer Jason Daida

6 Lessons Learned using Genetic Programming in a Stock Picking Context Michael Caplan and Ying Becker

7 Favourable Biasing of Function Sets Conor Ryan, Maarten Keijzel; and Mike Cattolico

8 Toward Automated Design of Industrial-Strength Analog Circuits by Means

of Genetic Programming J. R. Koza, L. W Jones, M. A. Keane, M. J. Streeter and S. H. Al-Sakran

vii ...

X l l l

xv

vi GENETIC PROGRAMMING THEORY AND PRACTICE I1 9 Topological Synthesis of Robust Dynamic Systems by Sustainable Ge- 143

netic Programming Jianjun Hu and Erik Goodman

10 Does Genetic Programming Inherently Adopt Structured Design Techniques? John M. Hall and Terence Soule

11 Genetic Programming of an Algorithmic Chemistry W Banzhaf and C. Lasarczyk

12 ACGP: Adaptable Constrained Genetic Programming Cezary 2. Janikow

13 Using Genetic Programming to Search for Supply Chain Reordering

Policies Scott A. Moore and Kurt DeMaagd

14 Cartesian Genetic Programming and the Post Docking Filtering Problem A. Beatriz Garmendia-Doval, Julian E Millel; and S. David Morley

15 Listening to Data: Tuning a Genetic Programming System Duncan MacLean, Eric A. Wollesen and Bill Worzel

16 Incident Detection on Highways Daniel Howard and Simon C. Roberts

17 Pareto-Front Exploitation in Symbolic Regression Guido E Smits and Mark Kotanchek

18 An Evolved Antenna for Deployment on NASA's Space Technology 5

Mission Jason D. Lohn, Gregory S. Hornby, and Derek S. Linden

Index

Contributing Authors

Sameer H. Al-Sakran is a Systems Research Programmer at Genetic Program- ming Inc. in Mountain View, California (al-sakran@sccm.Stanford.edu).

Wolfgang Banzhaf is Professor and Head of the Department of Computer Science at Memorial University of Newfoundland, St. John's, Canada (banzhaf@cs.mun.ca).

Ying Becker is a Principal, Advanced Research Center at State Street Global Advisors, State Street Corp (ying-becker@ssga.com).

Michael Capla11 is a Principal, IJS Quantitative Active Equity at State Street Global Advisors, State Street Corp (michael~caplan@ssga.com).

Flor Castillo is a Research Specialist in the Statistics and Applied Math and Physics Groups within the Physical Sciences Research and Development of the Dow Chemical Company (Facastillo@dow.com).

Mike Cattolico is a consultant at Tiger Mountain Scientific, Inc. (mike @TigerScience.com).

Shu-Heng Chen is Director of AI-ECON Research Center and Professor of Economics at National Chengchi University in Taiwan (chchen@nccu.edu.tw).

Jason M. Daida is an Associate Research Scientist in the Space Physics Re- search Laboratory, Department of Atmospheric, Oceanic and Space Sciences and is affiliated with the Center for the Study of Complex Systems at The University of Michigan, Ann Arbor (daida@umich.edu).

... Vl l l GENETIC PROGRAMMING THEORYAND PRACTICE 11

Kurt DeMaagd is a STIET Fellow and Ph.D. student in the Business Infor- mation Systems department at the Michigan Business School in Ann Arbor, Michigan (demaagdk@umich.edu).

A. Beatriz Garmendia-Doval is a Software engine'er at Quality Objects Ltd., Madrid, Spain (beatrizagd @ yahoo.co.uk).

David E. Goldberg is the Jerry S. Dobrovolny Distinguished Professor in En- trepreneurial Engineering and Director of the Illinois Genetic Algorithms Lab- oratory at the University of Illinois at Urbana-Champaign (deg@uiuc.edu).

Erik D. Goodman is Professor of Electrical and Computer Engineering and of Mechanical Engineering at Michigan State University (goodman @ egr.msu.edu).

John Hall is a computer scientist in the Digital Send Technology group at Hewlett-Packard Company (gpdesign@johnmhall.net).

Gregory S. Hornby is a computer scientist with QSS Group Inc., working as a member of the Evolvable Systems Group in the Computational Sciences Division at NASA Ames Research Center (hornby@email.arc.nasa.gov).

Daniel Howard is a QinetiQ Fellow and heads the Software Evolution Centre at QinetiQ in Malvern, UK (dhoward@qinetiq.com).

Jianjun Hu is a Ph.D. student in Computer Science and a member of Genetic Algorithm Research and Application Group (GARAGe) at Michigan State Uni- versity (hujianju@msu.edu).

Cezary Z. Janikow is an Associate Professor of Computer Science at the Uni- versity of Missouri, St. Louis (janikow@umsl.edu).

Lee W. Jones is a Systems Research Programmer at Genetic Programming Inc. in Mountain View, California (Ij25@pacbell.net).

Martin A. Keane is a consultant to the gaming industry and works with Genetic Programming, Inc. (martinkeane @ ameritech.net).

Contributing Authors

Maarten Keijzer is research scientist for KiQ Ltd, Amsterdam and researcher for the Strategic Research and Development Group at WL I Delft Hydraulics, Delft. He operates the researchlconsultancy company PrognoSys, Utrecht, The Netherlands (mkeijzer @ xs4all.nl).

Arthur K. Kordon is a Research and Development Leader in the Applied Math and Physics Group within the Physical Sciences Research and Development of the Dow Chemical Company (akordon@dow.com).

Mark Kotanchek is the group leader for Applied Math and Physics within Physical Sciences Research and Development of the Dow Chemical Company, Midland, MI, USA (mkotanchek@dow.com).

John R. Koza is Consulting Professor at Stanford University in the Biomedical Informatics Program in the Department of Medicine and in the Department of Electrical Engineering (koza@stanford.edu).

Tzu-Wen Kuo is aPh.D. student of Economics at National Chengchi University, Taiwan (kuo@aiecon.org).

Christian Lasarczyk is Research Assistant in the Department of Computer Science at the University of Dortmund, Germany (christian.lasarczyk@uni- dortmund.de).

Derek Linden is the Chief Technical Officer of Linden Innovation Research LLC, a company which specializes in the automated design and optimization of antennas and electromagnetic devices (dlinden@lindenir.com).

Jason D. Lohn leads the Evolvable Systems Group in the Computational Sci- ences Division at NASA Ames Research Center Cjlohn@email.arc.nasa.gov).

Duncan MacLean is co-founder of Genetics Squared, Inc., a computational dis- covery company working in the pharmaceutical industry (dmaclean@acm.org).

Julian Francis Miller is a Lecturer in the Department of Electronics at the University of York, England (jfm@ohm.york.ac.uk).

GENETIC PROGRAMMING THEORYAND PRACTICE II

Scott A. Moore is the Arthur F. Thurnau Professor and BBA Program Director at the Michigan Business School in Ann Arbor (samoore@umich.edu).

David Morley is the founder and Principal Consultant of Enspiral Discovery Ltd, and was previously the head of Computational Technology Development at Vernalis, Cambridge, UK (d.morley @enspiral-discovery.com).

Una-May O'Reilly is a research scientist in the Living Machines and Humanoid Robotics group in the Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (unamay@csail.mit.edu).

Rick Riolo is Director of the Computer Lab and Associate Research Scientist in the Center for the Study of Complex Systems at the University of Michigan (rlriolo@umich.edu).

Simon C. Roberts is a senior engineer of the Software Evolution Centre at QinetiQ in Malvern, UK (scroberts@btinternet.com).

Conor Ryan is a senior lecturer and University Fellow at the University of Limerick, Ireland (conor.ryan@ul.ie).

Kumara Sastry is a member of Illinois Genetic Algorithms Laboratory and a graduate student of Material Science and Engineering at the University of Illinois at Urbana-Champaign (kumara@illigal.ge.uiuc.edu).

Guido Smits is a Research and Development Leader in the Applied Math and Physics group within Physical Sciences Research and Development of Dow Benelux, Terneuzen, Netherlands (GFSMITS @dow.com).

Terence Soule is a Professor at the University of Idaho, where is he a member of the Computer Science Department and of the Bioinformatics and Computational Biology Program (tsoule@cs.uidaho.edu).

Matthew W. Streeter is a Ph.D. student at Carnegie Mellon University and was formerly a Systems Research Programmer at Genetic Programming Inc. in Mountain View, California (matts@cs.cmu.edu).

Contributing Authors

Jeff Sweeney is a Senior Statistician within the Physical Sciences Research & Development Group of the Dow Chemical Company (Jsweeney@dow.com).

Eric A. Wollesen is a gradute of the University of Michigan. He is currently employed as a software developer by Genetics Squared, Inc., a computational discovery company working in the pharmaceutical industry (ericw @genetics2.com).

Bill Worzel' is the Chief Technology Officer and co-founder of Genetics Squared, Inc., a computational discovery company working in the pharma- ceutical industry (billw @ arroyosoft.com).

Tina Yu is a computer scientist in the Mathematical Modeling Team at Chevron- Texaco Information Technology Company (Tina.Yu@chevrontexaco.com).

Wayne Zirk is a Senior Statistician within the Physical Sciences Research & Development Group of the Dow Chemical Company (ZirkWE@dow.com).

Preface

The work described in this book was first presented at the Second Workshop on Genetic Programming, Theory and Practice, organized by the Center for the Study of Complex Systems at the University of Michigan, Ann Arbor, 13-15 May 2004. The goal of this workshop series is to promote the exchange of research results and ideas between those who focus on Genetic Programming (GP) theory and those who focus on the application of GP to various real- world problems. In order to facilitate these interactions, the number of talks and participants was small and the time for discussion was large. Further, participants were asked to review each other's chapters before the workshop. Those reviewer comments, as well as discussion at the workshop, are reflected in the chapters presented in this book. Additional information about the workshop, addendums to chapters, and a site for continuing discussions by participants and by others can be found at http://cscs.umich.edu:8000/GPTP-20041.

We thank all the workshop participants for making the workshop an exciting and productive three days. In particular we thank all the authors, without whose hard work and creative talents, neither the workshop nor the book would be possible. We also thank our keynote speakers Lawrence ("Dave") Davis of NuTech Solutions, Inc., Jordan Pollack of Brandeis University, and Richard Lenski of Michigan State University, who delivered three thought-provoking speeches that inspired a great deal of discussion among the participants.

The workshop received support from these sources:

rn The Center for the Study of Complex Systems (CSCS);

rn Third Millennium Venture Capital Limited;

rn State Street Global Advisors, Boston, MA;

rn Biocomputing and Developmental Systems Group, Computer Science and Information Systems, University of Limerick;

rn Christopher T. May, RedQueen Capital Management; and

rn Dow Chemical, Core R&D/Physical Sciences.

xiv GENETIC PROGRAMMING THEORYAND PRACTICE II

We thank all of our sponsors for their kind and generous support for the work- shop and GP research in general.

A number of people made key contributions to running the workshop and assisting the attendees while they were in Ann Arbor. Foremost among them was Howard Oishi. Howard was assisted by Mike Charters. We also thank Bill Tozier for helping with reading and copy-editing chapters. Melissa Fearon's editorial efforts were invaluable from the initial plans for the book through its final publication. Thanks also to Deborah Doherty of Kluwer for helping with various technical publishing issues. Finally, we thank Carl Simon, Director of CSCS, for his support for this endeavor from its very inception.

Foreword

It was my good fortune to be invited to the 2004 Genetic Programming Workshop on Theory and Practice, held in May in Ann Arbor, Michigan. The goals of the workshop were unique, as was the blend of participants. To my knowledge, this workshop is alone in focusing on and promoting the interaction between theory and practice in the evolutionary computation world. There are many workshops and conference tracks that are oriented toward one or the other of these two, mostly disjoint, areas of evolutionary computation work. To participate in a workshop promoting interactions between the two subfields was a great joy.

The workshop organizers have summarized the various talks in the first chap- ter of this volume, and the reader can get a feel there for the talk I gave on the first day of the workshop. It is worth noting that a talk like mine - containing actual slides from training sessions for industrial practitioners of evolutionary computation, and containing a series of slides describing historically accurate but prickly interchanges between practitioners and theoreticians over the last twenty years - would most likely not have received a sympathetic hearing ten or twenty years ago. The attendees of this workshop, practitioners and theo- reticians in roughly equal numbers, were able to laugh at some points, consider others, and during the course of the workshop, openly discuss issues related to the integration of theoretical and practical work in evolutionary computation. Our field is maturing in both areas, and so are our approaches to promoting interactions between our field's practical and theoretical subfields.

There is a good deal to be gained by all of this in these types of interactions, and by the change in focus that they create. The papers in this year's workshop are very stimulating, and I look forward as well to reading next year's workshop volume, containing even more work lying on the frontiers between theory and application of evolutionary computation.

Dr. Dave Davis, Vice President of Product Research NuTech Solutions, Inc., Massachusetts, USA June, 2004