REFERENCES AND NOTES

1. 0. Diels and K. Alder, Justus Liebigs Ann. Chem.460, 98 (1928); 0. Diels, J. H. Blom, W. Koll, ibid.443, 242 (1925); B. M. Trost, I. Fleming, L. A.Paquette, Comprehensive Organic Synthesis(Pergamon, Oxford, 1991), vol. 5, pp. 316; F.Friguelli and A. Tatichi, Dienes in the Diels-AlderReaction (Wiley, New York, 1990), and referencestherein; W. Carruthers, Cycloaddition Reactions inOrganic Synthesis (Pergamon, Oxford, 1990),and references therein; G. Desimoni, G. Tacconi,A. Barco, G. P. Pollini, ACS Monograph No. 180(American Chemical Society, Washington, DC,1983), and references cited therein.

2. For recent reviews, see U. Pindur, G. Lutz, C. Otto,Chem. Rev. 93, 741 (1993); H. B. Kagan and 0.Riant, ibid. 92, 1007 (1992), and references therein.

3. D. Hilvert, K. W. Hill, K. D. Nared, M-T. M. Auditor,J. Am Chem. Soc. 111, 9261 (1989).

4. A. C. Braisted and P. G. Schultz, ibid. 112, 7430(1 990).

5. C. J. Suckling, M. C. Tedford, L. M. Bence, J. I.Irvine, W. H. Stimson, Biorg. Med. Chem. Lett. 2,49 (1992).

6. K. D. Janda, C. G. Sheviin, R. A. Lerner, Science259, 490 (1993).

7. L. E. Overman, G. F. Taylor, C. B. Petty, P. J.Jessup, J. Org. Chem. 43, 2164 (1978).

8. For example: d-pumiliotoxin, C. L. E. Overmanand P. J. Jessup, Tetrahedron Lett. 1253 (1977);J. Am. Chem. Soc. 100, 5179 (1978); d-perhydro-gephyrotoxin, L. E. Overman and C. Fukaya, ibid.102, 1454 (1980); dIisogabaculine, S. Danishef-sky and F. M. Hershenson, J. Org. Chem. 44,1180 (1979); tylidine, L. E. Overman, C. B. Petty,R. J. Doedens, ibid., p. 4183; amaryllidaceaealkaloids, L. E. Overman et al., J. Am. Chem. Soc.103, 2816 (1981); gephirotoxine, L. E. Overman,D. Lesuisse, M. Hashimoto, ibid. 105, 16, 5373(1983).

9. Diene 1b was prepared by neutralizing the corre-sponding carboxylic acid suspended in PBS (so-dium phosphate buffer) with one equivalent ofNaOH.

10. All new adducts were characterized by NMRand mass spectrometry.

11. In the crude mixture of the cycloaddition, wecould not detect any trace of the meta regioiso-mer by 1H NMR (>98% ortho regioselective).

12. M. J. Frisch et al., Gaussian 92 (Gaussian, Pitts-burgh, PA, 1992).

13. R. J. Loncharich, F. K. Brown, K. N. Houk, J. Org.Chem. 54, 1129 (1989). All structures were fullyoptimized with the 3-21G basis set. Reactantswere also fully optimized with the 6-31G* basisset. Vibrational frequencies for all transition struc-tures were calculated at the RHF/3-21 G level andshown to have one imaginary frequency. In addi-tion, the energies were evaluated by single-pointcalculations with the 6-31G* basis set on 3-21Ggeometries.

14. K. N. Houk, R. J. Loncharich, J. Blake, W. Jor-gensen, J. Am. Chem. Soc. 111, 9172 (1989).

15. L. E. Overman, G. F. Taylor, K. N. Houk, I. N.Domelsmith, ibid. 100, 3182 (1978).

16. M. I. Page and W. P. Jencks, Proc. Nati. Acad.Sci. U.S.A. 68, 1678 (1971).

17. F. Bernardi et al., J. Am. Chem. Soc. 110, 3050(1989); F. K. Brown and K. N. Houk, TetrahedronLett. 25, 4609 (1984).

18. This strategy to avoid product inhibition was firstdescribed by Braisted and Schultz in (4).

19. The stereochemistry was established on the basisof 'H NMR analysis. For example: 5-exo, BH6 =2.65 ppm and 4J6 7a= 3.5 Hz; 5-endo, &H6 = 3.12ppm, and 6 7=a 0 Hz.

20. G. Kohler and C. Milstein, Nature 256, 495 (1975);the KLH conjugate was prepared by slowly add-ing 2 mg of the hapten dissolved in 100 &l ofdimethylformamide to 2 mg of KLH in 900 gl of0.01 M sodium phosphate buffer, pH 7.4. Four8-week-old 129GIX+ mice each received 2 in-traperitoneal (IP) injections of 100 gg of the hap-ten conjugated to KLH and RIBI adjuvant (MPLand TDM emulsion) 2 weeks apart. A 50 >g IP

208

injection of KLH conjugate in alum was given 2weeks later. One month after the second injection,the mouse with the highest titer was injectedintravenously with 50 Ag of KLH-conjugate; 3days later, the spleen was taken for the prepara-tion of hybridomas. Spleen cells (1.0 x 108) werefused with 2.0 x 107 SP2/0 myeloma cells. Cellswere plated into 30 96-well plates; each wellcontained 150 gl of hypoxanthine, aminopterin,thymidine-Dulbecco's minimal essential medium(HAT-DMEM) containing 1% nutridoma, and 2%bovine serum albumin.

21. E. Engvall, Methods Enzymol. 70, 419 (1980).22. A Cl8 column, (VYDAC 201TP54) was used with

an isocratic program of 78/22; H20 (0.1% tri-

fluoroacetic acid)/acetonitrile, at 1.5 mI/min, 240nm. The retention time for the trans isomer is6.00 min and for the cis isomer is 6.60 min.

23. We thank J. Ashley for technical assistance inthe kinetic analysis; L. Ghosez, K. B. Sharpless,K. C. Nicolaou, E. Keinan, and D. Boger forhelpful discussions and comments on the manu-script; and R. K. Chadha for the x-ray crystallo-graphic study. Supported in part by the NationalScience Foundation (CHE-9116377) and the A.P. Sloan Foundation (K.D.J.). We also thankFundaci6n Ram6n-Areces (Spain) for a fellow-ship to B.P.T.

16 July 1993; accepted 3 September 1993

Detecting Subtle SequenceSignals:.A Gibbs Sampling

Strategy for Multiple AlignmentCharles E. Lawrence, Stephen F. Altschul, Mark S. Boguski,

Jun S. Liu, Andrew F. Neuwald, John C. WoottonA wealth of protein and DNA sequence data is being generated by genome projects andother sequencing efforts. A crucial barrier to deciphering these sequences and under-standing the relations among them is the difficulty of detecting subtle local residue patternscommon to multiple sequences. Such patterns frequently reflect similar molecular struc-tures and biological properties. A mathematical definition of this "local multiple alignment"problem suitable for full computer automation has been used to develop a new andsensitive algorithm, based on the statistical method of iterative sampling. This algorithmfinds an optimized local alignment model for N sequences in N-linear time, requiring onlyseconds on current workstations, and allows the simultaneous detection and optimizationof multiple patterns and pattern repeats. The method is illustrated as applied to helix-turn-helix proteins, lipocalins, and prenyltransferases.

Patterns shared by multiple protein ornucleic acid sequences shed light on molec-ular structure, function, and evolution.The recognition of such patterns generallyrelies upon aligning many sequences, acomplex, multifaceted research processwhose difficulty has long been appreciated.This problem may be divided into "globalmultiple alignment" (1, 2), whose goal is toalign complete sequences, and "local mul-tiple alignment" (2-11), whose aim is tolocate relatively short patterns shared byotherwise dissimilar sequences. We report anew algorithm for local multiple alignmentthat assumes no prior information on thepatterns or their locations within the se-quences; it determines these locations fromonly the information intrinsic to the se-quences themselves. We focus on subtle

C. E. Lawrence, S. F. Altschul, M. S. Boguski, A. F.Neuwald, and J. C. Wootton are with the NationalCenter for Biotechnology Information, National Libraryof Medicine, National Institutes of Health, Bethesda,MD 20894. J. S. Liu is with the Statistics Department,Harvard University, Cambridge, MA 02138. C. E.Lawrence is also affiliated with the Biometrics Labora-tory, Wadsworth Center for Labs and Research, NewYork State Department of Health, Albany, NY 12201.

SCIENCE * VOL. 262 * 8 OCTOBER 1993

amino acid sequence patterns that may varygreatly among different proteins.

Much research on the alignment of suchpatterns uses additional information to sup-plement algorithmic analyses of the actualsequences, including data on three-dimen-sional structure, chemical interactions ofresidues, effects of mutations, and interpre-tation of sequence database search results.However, such research, which has led tomany discoveries of sequence relations andstructure and function predictions [see (12)for a recent example], is laborious andrequires frequent input of expert knowl-edge. These approaches are becoming in-creasingly overwhelmed by the quantity ofsequence data.A number of automated local multiple

alignment algorithms have been developed(2-11), and some have proved valuable aspart of integrated software workbenches.Unfortunately, rigorous algorithms for find-ing optimal solutions have been so compu-tationally expensive as to limit their appli-cability to a very small number of se-quences, and heuristic approaches havegained speed by sacrificing sensitivity to

P1.1 1. 911.1 W11

on

Apr

il 13

, 200

9 w

ww

.sci

ence

mag

.org

Dow

nloa

ded

from

highly variable patterns.Our method is both fast and sensitive

and generally finds an optimized local align-ment model for N sequences in N-lineartime. This advantage is achieved by incor-porating some recent developments in sta-tistics and by using a formulation of theproblem that models well the underlyingbiology but avoids the explicit treatment ofgaps. We illustrate the application of thismethod with a diverse set of difficult butwell understood test cases.

Problem and methods. Our problem isto locate and describe a pattern thought tobe contained within a set of biopolymersequences. The model we use has threefundamental characteristics. First, we seeka relatively small number of sequence ele-ments or patterns, each consisting of oneungapped segment from each of the inputsequences. Second, a single pattern is de-scribed by a probabilistic model of residuefrequencies at each position. Third, thelocation of the pattern within the se-quences is described by a set of probabilis-tically inferred position variables. Thesefeatures are derived from well establishedprinciples of protein structure and knowl-edge of the sources of sequence patternvariation (13). These principles are valid ingeneral for globular protein families, al-though a few interesting counterexamplesare known.

First, homologous proteins or proteindomains typically are characterized by acore of common secondary structure ele-ments separated by intervening loops (13).Gaps in sequence alignments stem primarilyfrom variations in loop length, and loopsthat participate in active sites are con-strained to maintain their geometry, andthus frequently retain their length as well.Common sequence patterns therefore cangenerally be described by a relatively smallnumber of ungapped elements.

Second, physicochemical constraints in-fluence which particular residues may occurat each position in a sequence element.The similarities of closely related sequencesstem largely from recent common ancestryand are relatively easy to locate by variousmethods (2-11), including ours. In con-trast, our primary concern is to locate thecommon features of sequences that differgreatly. Here, similar local residue patternsreflect structural and functional constraintsthat arise from the energetic interactionsamong residues or between residue andligand, irrespective of evolutionary history.The relation between a state's energy andfrequency forms the basis of statistical me-chanics, and an analogous relation governsthe frequencies of residues subject to ran-dom point mutations (14). Residue fre-quency models are therefore natural in thepresent context.

Third, genomic rearrangements, as wellas insertions, deletions, and duplications ofsequence segments, result in the occurrenceof a common pattern at different positionswithin sequences. However, these muta-tional events are "unobserved" because nodata directly specify their effects on thepositions of the patterns (6). As recognizedby statisticians since the 1970s (15), manyproblems with unobserved data are mosteasily addressed by pretending that criticalmissing data are available. The key "miss-ing information principle" (15) is that theprobabilities for the unobserved positionsmay be inferred through the application ofBayes theorem to the observed sequencedata.

The optimization procedure we use isthe predictive update version (16) of theGibbs sampler (17). Strategies based oniterative sampling have been of great inter-est in statistics (18). The algorithm can beunderstood as a stochastic analog of expec-tation maximization (EM) methods previ-ously used for local multiple alignment (6,7). It yields a more robust optimizationprocedure and permits the integration ofinformation from multiple patterns. In ad-dition, a procedure for the automatic de-termination of pattern width has beendeveloped. For clarity, we first describethe identification of a single pattern offixed width within each input sequenceand then generalize to variable widths andmultiple patterns.

The basic algorithm. We assume thatwe are given a set ofN sequences S,, . . ..SN and that we seek within each sequencemutually similar segments of specifiedwidth W. The algorithm maintains twoevolving data structures. The first is thepattern description, in the form of a prob-abilistic model of residue frequencies foreach position i from 1 to W, and consist-ing of the variables qi,1, . . ., qi,20. Thispattern description is accompanied by ananalogous probabilistic description of the"background frequencies" Pi, . . -- P20with which residues occur in sites notdescribed by the pattern. The second datastructure, constituting the alignment, is aset of positions ak, for k from 1 to N., forthe common pattern within the se-quences. Our objective will be to identifythe "best," defined as the most probable,common pattern. This pattern is obtainedby locating the alignment that maximizesthe ratio of the corresponding patternprobability to background probability.

The algorithm is initialized by choosingrandom starting positions within the vari-ous sequences. It then proceeds throughmany iterations to execute the followingtwo steps of the Gibbs sampler:

1) Predictive update step. One of the Nsequences, z, is chosen either at random or

in specified order. The pattern descriptionqi j and background frequencies pj are thencalculated, as described in Eq. 1 below,from the current positions ak in all se-quences excluding z.

2) Sampling step. Every possible seg-ment of width W within sequence z isconsidered as a possible instance of thepattern. The probabilities Q, of generatingeach segment x according to the currentpattern probabilities qij are calculated, asare the probabilities Px of generating thesesegments by the background probabilitiesp,. The weight Ax = QX/Px is assigned tosegment x, and with each segment soweighted, a random one is selected (19). Itsposition then becomes the new a,.

This simple iterative procedure consti-tutes the basic algorithm. The central ideais that the more accurate the pattern de-scription constructed in step 1, the moreaccurate the determination of its locationin step 2, and vice versa. Given randompositions ak, in step 2 the pattern descrip-tion qij will tend to favor no particularsegment. Once some correct a, have beenselected by chance, however, the qij beginto reflect, albeit imperfectly, a pattern ex-tant within other sequences. This processtends to recruit further correct ak, which inturn improve the discriminating power ofthe evolving pattern.An aspect of the algorithm alluded to in

step 1 above concerns the calculation of theqij from the current set of ak. For the ithposition of the pattern we have N - 1observed amino acids, because sequence zhas been excluded; let ci i be the count ofamino acid j in this position. Bayesianstatistical analysis suggests that, for thepurpose of pattern estimation, these c1,should be supplemented with residue-de-pendent "pseudocounts" bj to yield patternprobabilities

cij + bj4ij=N- 1 +B (1)

where B is the sum of the 1,. The pj arecalculated analogously, with the corre-sponding counts taken over all nonpatternpositions (20).

After normalization, A, gives the prob-ability that the pattern in sequence z be-longs at position x. The algorithm finds themost probable alignment by selecting a setof ak's that maximizes the product of theseratios. Equivalently, one may maximize F,the sum of the logarithms of these ratios. Inthe notation developed above, F is given bythe formula

W 20

F = Xc jj log'

i = 1 j = 1

(2)

where the c.. and qij are calculated from thecomplete alignment (Fig. 1).

SCIENCE * VOL. 262 * 8 OCTOBER 1993

- --------------------------- ......i------------------------RMR - ...

on

Apr

il 13

, 200

9 w

ww

.sci

ence

mag

.org

Dow

nloa

ded

from

Phase shifts. One defect of the algo-rithm as just described is the "phase" prob-lem. The strongest pattern may begin, forexample, at positions 7, 19, 8, 23, and soforth within the various sequences. Howev-er, if the algorithm happens to choose al =9 and a2 = 21 in an early iteration, it willthen most likely proceed to choose a3 = 10and a4 = 25. In other words, the algorithmcan get locked into a nonoptimal "localmaximum" that is a shifted form of theoptimal pattern. This situation can beavoided by inserting another step into thealgorithm (16). After every Mth iteration,for example, one may compare the currentset of ak with sets shifted left and right by upto a certain number of letters. Probabilityratios may be calculated, as above, for allpossibilities, and a random selection ismade among them with appropriate corre-sponding weights.

Pattern width. The algorithm as so fardescribed requires the pattern width to beinput. It is possible, of course, to executethe algorithm with a range of plausiblewidths and then select the best result ac-cording to some criterion. One difficulty isthat the function F is not immediatelyuseful for this purpose, as its optimal valuealways increases with increasing width W.

The problem here corresponds to thewell-known issue of model selection en-countered in statistics. The difficulty stemsfrom the change in the dimensionality withthe additional freely adjustable parameters.Several criteria that incorporate the effectsof variable dimension have been useful inother applications (21). Unfortunately,these criteria did not perform well at select-ing those pattern widths that identifiedcorrect alignments in data sets with knownsolutions.A superior criterion proved to be one

based on the incomplete-data log-probabil-ity ratio G (22), which subtracts from thefunction F the information required to de-termine the location of the pattern in eachof the input sequences. We found thatdividing G by the number of free parame-ters needed to specify the pattern (19W inthe case of proteins) produced a statisticuseful for choosing pattern width. We callthis quantity the information per parame-ter. The use of this empirical criterion isdiscussed in the examples section below andis illustrated in Figs. 2 and 3.

Multiple patterns. As described above,a pattern within a set of sequences can bedescribed as consisting of several distinctelements separated by gaps. The Gibbssampler may easily maintain several distinctpatterns rather than a single one. Seekingseveral patterns simultaneously rather thansequentially allows information gainedabout one to aid the alignment of others.The relative positions of elements within

210

the sequences can be used to improve theirsimultaneous alignment. Because only oneelement in sequence z is altered at a time,the combinatorial problem of joint posi-tioning is circumvented. Nevertheless, be-cause no element's position is permanentlyfixed, the best joint location of all elementsmay be identified.

Incorporating models of element loca-tion that favor consistent ordering (colin-earity) and of element spacing that favorclose packing accommodates insertions anddeletions. Our implementation of a multi-element version of the Gibbs sampler (23)includes ordering probabilities (24). As il-lustrated below, this joint information im-proves the prediction of the correct align-ment of colinear elements. Constraints onloop length variation result in similarities inthe spacing of the elements of homologousproteins. Thus, inclusion of an elementspacing component in the model should

A

B

ArgLysGluAspGinHisAsnSerGlyAlaThrProcysValLeuIleMetTyrPheTrp

Sigma-37SpoIinCNahRAntennapediaNtrC (Brady.)DicAMerDFisMAT alLambda cIICrp (CAP)Lambda CroP22 CroAraCFnrHtpRNtrC (K.a.)CytRDeoRGalRLacITetRTrpRNifASpoIIGPinPurREbgRLexAP22 cI

improve alignment. However, we have notyet found it necessary to incorporate spac-ing effects into the algorithm (25).

Examples. To examine the algorithm,we have chosen three examples that presentdifferent classes of difficulties for automatedmultiple alignment. First is the helix-turn-helix (HTH) motif, which represents alarge class of sequence-specific DNA bind-ing structures involved in numerous cases ofgene regulation. Such HTH motifs gener-ally occur singly as local, isolated structuresin different sequence contexts. Detectionand alignment of HTH motifs is a well-recognized problem because of the greatsequence variation compatible with thesame basic structure. Second are the lipo-calins, a family of proteins that bind small,hydrophobic ligands for a wide range ofbiological purposes. These proteins showwidely spaced sequence motifs within high-ly variable sequences but share the same

223 IIDLTYIQNK SQKETGDILGISQMHVSR LQRKAVKKLR94 RFGLDLKKEK TQREIAKELGISRSWSR IEKRALMKMF22 VVFNQLLVDR RVSITAENLGLTQPAVSN ALKRLRTSLQ

326 FHFNRYLTRR RRIEIAHALCLTERQIKI WFQNRRMKWK449 LTAALAATRG NQIRAADLLGLNRNTLRK KIRDLDIQVY22 IRYRRMNLKH TQRSLAKALKISHVSVSQ WERGDSEPTG5 )MAY TVSRLALDAGVSVHIVRD YLLRGLLRPV

73 LDMVMQYTRG NQTRAAL4MINRGTLRK KLKKYGMN99 FRRKQSLNSK EKEEVAKKCGITPLQVRV WFINKRMRSK25 SALLNKIALM GTEKTAEAVGVDKSQISR WKRLMIPKFS

169 THPDGMQIKI TRQEIGQIVGCSRETVGR ILKMLEDQNL15 ITLKDYAMRF GQTKTAKDLGVYQSAINK AIHAGRKIFL12 YKKDVIDHFG TQRAVAKALGISDAAVSQ WKEVIPEKDA

196 ISDHLADSNF DIASVAQHVCLSPSRLSH LFRQQLGISV196 FSPREFRLTM TRGDIGNYLGLTVETISR LLGRFQKSGM252 ARWLDEDNKS TLQELADRYGVSAERVRQ LEKNANKKLR444 LTTALRHTQG HKQEAARLLGWGRNTLTR KLRELGME11 MKAKKQETAA TMKDVALKAKVSTATVSR ALMNPDKVSQ23 LQELRRSDRL HLKDAAALLGVSEMTIRR DLNNHSAPVV3 MA TIKDVARLAGVSVAWSR VINNSPRASE5 MKPV TLYDVAEYAGVSYQTVSR VVNQASHVSA

26 LLNEVGIEGL TTRKLAQKLGVEQPTLYW VKNKRALLD67 IVEELLRGEM SQRELKNELGAGIATITR GSNSLRAAPV

495 LIAALEKAGW VQAKAARLLGMTPRQVAY RIQIMDITMP205 RFGLVGEEEK TQKDVAIMGISQSYISR LEKRIIKRLR160 QAGRLIAAGT PRQKVAIIYDVGVSTLYK TFPAGDR

3 MA TIKDVAKRANVSTTTVSH VINKTRFVAE3 MA TLKDIAIEAGVSLATVSR VLNDDPTLNV

27 DHISQTGMPP TRAEIAQRLGFRSPNAAE EHLKALARKG25 SSILNRIAIR GQRRVADALGINESQISR WRGDFIPRMG

Position in site1 2 3 4 5 6 7 8 9 10 11 12

949

53679

240168117151

9915769

58999999

222133

99

60099999

13099

107121166104

999

265442969

22499

11756

112130

9999

1149

13699

137380401473

999

1179

4399999

619999

9 99 719 99 99 99 99 99 99 151

181 901251 9

9 99 9

500 9149 9323 9

9 99 99 99 9

137380140299224125168

99

439999

93114

9999

137194140125

91258999

1819999

149166198262

99

999999999

21599

295156458

9198262

99

9133

96799

899

1141999

5819999999

999999999

4399

295598149427104

9108366

529

536799

248819151

9311

999999

13699

24011139

343466392290

11642

1863229

213213269461284020224384

51222217720204442

A25944A28627A32837A23450B26499B24328 (BVECDA)C29010A32142 (LDNECFS)A90983 (JEBY1)A03579 (QCBP2L)A03553 (QRECC)A03577 (RCBPL)A25867 (RGBP22)A03554 (RGECA)A03552 (RGECF)A00700 (RGECH)A03564 (RGKBCP)A24963 (RPECCT)A24076 (RPECDO)A03559 (RPECG)A03558 (RPECL)A03576 (RPECTN)A03568 (RPECW)S02513S07337S07958S08477S09205S11945B25867 (11BPC2)

13 14 15

22271

14067

278125

9639

43130210

920537619

13699

949

1409

6312516838756

18170

2109

58379

198999

94999

27812589639

112855

9999

619

26299

16 17 18

999999999

43999

746177427

9999

265719999

898195678

130999999

26299

6062565367

17024089999999

589

619

1369

366

Fig. 1. Alignment and probability ratio model for the helix-turn-helix pattern common to 30 proteins(45). (A) The alignment. Columns from left to right are: sequence name; locations ak of the left endof the common pattern in each sequence; aligned sequences, including residues flanking the18-residue common pattern; right-end positions (ak + 17) of the common pattern; NBRF/PIRaccession number; and NBRF/PIR code name, if available. Asterisks (***) below the alignmentindicate the 20-residue segment previously described on the basis of structural superpositions (26,27). Almost equal values of information per parameter were given by pattern widths of 18 to 21residues (Fig. 2): the longer widths extended to the right the 18-residue pattern shown. (B)Probability ratios (100 x qj,/p) for each amino acid at each position in the pattern model.

SCIENCE * VOL. 262 * 8 OCTOBER 1993

on

Apr

il 13

, 200

9 w

ww

.sci

ence

mag

.org

Dow

nloa

ded

from

structural topology throughout the polypep-tide chain. We chose the five most diver-gent lipocalins with known 3D structure foranalysis because the correct alignment oftheir sequence motifs has previously de-pended on structural superposition. Thirdare isoprenyl-protein transferases, essentialcomponents of the cytoplasmic signal trans-duction network. The P3 subunits of theseenzymes contain multiple copies of multiplemotifs that have not previously been satis-factorily characterized by automated align-ment methods.

Single site: HTH proteins. The wide-spread DNA binding HTH structure com-prises -20 contiguous amino acids (26). Inour test set of 30 proteins (Fig. 1), thecorrect location of the motif is known (26,27) from x-ray and nuclear magnetic reso-nance structures, or from substitution mu-tation experiments, or both. The rest of the3D structure of these proteins, apart fromthe HTH structure itself, is completelydifferent in different subfamilies. Further-more, the element is found at positionsthroughout the polypeptide chain. Our testset represents a typically diverse cross sec-tion of HTH sequences. Close homologshave been excluded. The difficulty of detec-tion and alignment of the HTH motif fromsuch sequences is well recognized. Therehave been several attempts to develop po-sition-specific weight matrices and otherempirical pattern discriminators diagnosticfor this structure (28). These have achievedsome success in making several predictionsthat were later confirmed and that have alsoaroused controversy (29).We used this example to develop two

important features of the algorithm. First,the empirical criterion of information perparameter allowed for the automated deter-mination of element width (Fig. 2). Sec-ond, heuristic convergence criteria substan-tially shortened the time required to findthe best model (Fig. 3, legend). These twofeatures enabled the algorithm to identifyand align all 30 HTH motifs quickly andconsistently. Correct alignments were ob-tained with six pattern widths in the rangefrom 17 to 22 residues (Fig. 2), of which 21residues had the highest converged value ofinformation per parameter. These resultscompare favorably with the 20-residue viewbased previously on structural superposi-tions (26, 27). The criteria developed em-pirically with the HTH example haveworked consistently well in all of our sub-sequent applications.

Multiple sites: Lipocalins. The majorityof protein sequence families contain multi-ple colinear elements separated by variable-length gaps (13). We have successfullyaligned distantly related sequences for sever-al problems in this class, including proteinkinases, aspartyl proteinases, aminoacyl-

tRNA ligases and mammalian helix-loop-helix proteins. We report here on one of themost difficult of these test cases: in lipocalins(30, 31), two weak sequence motifs, cen-tered on the generally conserved residues-Gly-X-Trp- and -Thr-Asp-, are recognized

from structural comparisons (31, 32). Therest of the topologically conserved lipocalinfolds have very different sequences.

Conventional automated sequence align-ment methods, although successful for se-lected subsets of the data [such as (33)], fail

Fig. 2. Information per parameter as the criterion j18.of pattern width for helix-turn-helix (HTH) pro- Z + + * +teins. The points indicate the maximum values of +information per parameter found by the algo- 1.6 +rithm. The upper points (A and +) used the m

complete sequences of the 30 HTH proteins 11.4listed in Fig. 1A. (A) All of the sequences in thedata set were aligned in the correct register (as c 1.2in Fig. 1A). (+) One or more of the sequences in . . . . . .the data set were incorrectly aligned. All com- X1 0xpletely correct alignments in the width range ,from 17 to 22 residues gave greater values of £ 15 20 25 30information per parameter than any incorrect Pattern widthalignments outside this width range. (0) The"nonsites" sequence data of the 30 HTH proteins, constructed by deleting the 18 residues of theHTH pattern itself (Fig. 1A) from each of the sequences. (x) A shuffled data set (46) of the 30 HTHsequences. The alignments from the nonsites background of the HTH proteins give values slightlygreater than random expectation.

Fig. 3. Convergence behav-ior of the Gibbs sampling 1.8 'algorithm. Because theGibbs sampler, when run for i 1.6finite time, is a heuristic rath- = 12 3er than a rigorous optimiza- , 1.4tion procedure, one cannot Eguarantee the optimality of I.1X2the results it produces. IP1Therefore, the best solution 11.0found in a series of runs willbe called "maximal." A sin- 0gle pattern of width 18 resi- Edues was sought in the data i.set of 30 HTH proteins 0.6shown in Fig. 1A. Solid linesshow the course of three 04independent runs with dif- 0 1000 2000 3000 4000 5000 6000ferent random seeds. Evolv- Number of Iterationsing models in such runs rap-idly reach intermediate "background" information values (1.0 to 1.2 bits per parameter) and thensample different models in this plateau region for a widely variable number of iterations beforeconverging rapidly. Curve 1 is typical in showing a very short lag time on the plateau; longer lagsas in curves 2 and 3 are less common. Curves 1 and 3 illustrate the stochastic behavior of the Gibbssampler: once "converged," the model stays predominantly at the maximal value of 1.84 bits perparameter but is never permanently in this solution. In the infinite limit, the sampler will spend theplurality of its time on the pattern that maximizes Fand therefore the information per parameter (22).Curve 2 demonstrates persistence (after escape from the background plateau) in a submaximalstate (1.80 bits per parameter), which is a "phase-shifted" version of the best model. Whensufficiently large stochastic phase shifts are allowed (see text), such states do not normally trap theevolving model for many iterations. Curve 3 reaches exactly the same maximal value as curve 1,suggesting one possible strategy for detecting convergence, namely, recurrence of exactly thesame pattern with different seeds. The following heuristic approach was found to greatly reduce thetime required to find the apparently optimal alignment: (i) For a given random seed, repeat the basicalgorithm a fixed number of times (typically 10 times for each input sequence) beyond the lastiteration in which the best pattern observed (with this seed) improved; and (ii) try at most some fixednumber of seeds (usually 10 for a single element), but stop when the best pattern is reproduced bya specified number of different seeds (usually 2). The rationale underlying this approach is that it isunlikely for the identical suboptimal solution to be found on several independent trials before theoptimal solution is found once. The dotted line represents a run on the nonsite data set (Fig. 2). Suchruns never exceed 1.2 bits per parameter and thus are stuck permanently in states resembling thebackground plateaus from which the models of the HTH motif alignments eventually escape.Repeated runs in which shuffled sequences as input data are used can provide criteria for howstrong a pattern is required to be considered significant (38).

SCIENCE * VOL. 262 * 8 OCTOBER 1993 211

on

Apr

il 13

, 200

9 w

ww

.sci

ence

mag

.org

Dow

nloa

ded

from

to align these motifs for the full spectrum oflipocalin sequences. Challenged with fivesuch diverse sequences of known crystalstructure, our algorithm correctly alignedthese two regions and extended the width ofboth to 16 residues (Fig. 4), in agreementwith the structural evidence (31, 32).

Multiple copies of multiple sites: Pre-nyltransferases. Internal repeats in proteinsequences underlie many important struc-tures and functions and are more commonthan is generally recognized (34). Theserepeats are often obscured by sequence di-vergence following duplication, renderingtheir detection and characterization a chal-lenging problem. The analysis of repeats isoften labor-intensive, relying in part onvisual inspection of "dot plots" (10, 34)-aprocedure that limits searches and surveysof large databases.An example of recent interest involves

sequence repeats in the subunits of theheterodimeric protein-isoprenyltransferases(10, 35). These enzymes are responsible fortargeting and anchoring members of the rassuperfamily of small guanosine triphos-phatases to their sites of action on variouscellular membranes (36). The 1 subunits ofprenyltransferases contain a subtle internalrepeat of possible function significance(34). Although no direct structural infor-mation is yet available for these proteins,previous sequence analysis suggested thatthe 1 subunit repeat consists of three motifsseparated by variable-length gaps and thatthis entire tripartite structure is repeatedthree to five times in each of four proteins(10, 35).

The challenge here is therefore to iden-tify a relatively large number of weak pat-terns covering up to 80 percent of thelength of the sequences. The resultingcrowding of elements increases interele-ment dependencies and the complexity ofthe joint probability surface over which thealgorithm must find the most probablealignment.

The previous analysis was subjective andtime-consuming, relying on the combined useof several different multiple alignment meth-ods. In contrast, the Gibbs sampling algo-rithm quickly and objectively reproduced andextended the previous results (Fig. 5).

Evaluation and comparison. The maindifficulties of automated local multiplealignment stem from the high dimensional-ity of the search space and the existence ofmany local optima. Here, the large searchspace is explored one dimension at a timeby comparing each sequence to an evolvingresidue frequency model. Stochastic sam-pling permits the algorithm to escape localoptima in which deterministic approachesmay get trapped. Including a phase shiftstep expedites convergence by permittingthe sampler to explore related local optima.

212

Tests showed the algorithm to be rela- the algorithm to seek a pattern in only atively insensitive to various numbers of specified number of input sequences.negative examples included among the in- The use of an appropriate model forput sequences. To cope with large numbers interelement spacing would improve theof negative examples, we have extended algorithm's sensitivity, but this feature has

Motif A

17 32ICYAIANSE .. GYCPDVKPVN DFDLSAFAGAWHEIAK LPLENENQGK ... FGQRVVNLVP

25 40LACBBOVIN .. QALIVTQTMK GLDIQKVAG'WYSLAM AASDISLLDA ... KIDALNEMV

16 31BBPYPIEBR .. GACPEVKPVD NFDWSNYHGRWWEVAK YPNSVEKYGK ... YGGVTKENVF

14 29RETBCBOVIN .. CRVSSFRVKE NFDKARFAGTWYAMAK KDPEGLFLQD ... SFLQKGNDDH

27 42MUP2-EOUSE .. HAEEASSTGR NFNVEKINGEWHTIIL ASDKREKIED ... SVTYDGFNTP

Motif B

104 119WVLATDYKNYAINYNC DYHPDKKAMS109 124LVLDTDYKKYLLFCME NSAEtEQSLA100 115NVLSTDNIWYIIGYYC KYDED KDGHQ105 120WIIDTDYETFAVQYSC RLLNLDGTCA . .

109 124TIPKTDYDNFLMAHLI NEKDGETFQL ..

Fig. 4. Two motifs located automatically in five lipocalins of known crystal structure. The sequences,defined by SwissProt database codes, are, from top to bottom: Manduca sexta insecticyanin,bovine 3-lactoglobulin, Pieris brassicae bilin-binding protein, bovine plasma retinol-binding protein,and mouse major urinary protein 2. Asterisks (***) below the alignment denote generally conservedresidues recognized from structural comparisons (30, 31). The criterion of information perparameter (0.66 and 0.65 bits for motifs A and B, respectively) suggested an extended width of 16residues for both motifs, in agreement with the superposable structures of the proteins in theseregions (31, 32).

A L BRami

109 MLYWIANSLKVM DRDWLSDD---129 TKRKIVVKLFTI SPSG-------------------- GPFGGGPGQLSH LA- STYAAINALSLC DNIDGCWDRID181 DRKGIYQWLISL KEPN-------------------- GGFRTCLEVGEV DTR GIYCALSIATLL NILTEEL----230 LTEGVLNYLKNC QNYE-------------------- GGFGSCPHVDEA HGG YTFCATASLAIL RSMDQIN----279 NVEKLLEWSSAR QLQEE------------------- RGFCGRSNKLVD GC- YSFWVGGSAAIL EAFGYGQCF--331 NXHALRDYILYC CQEKEQ------------------ PGLRDKPGAHSD FY- HTNYCLLGLAVA E----------

SSYSCTPNDSPH ... 27 aa415 NVRKIIHYFKSN LSSPS

FT-P74 QREKHFHYLKRG LRQLTDAYECLDAS99 SRPWLCYWILHS LELLDEPIPQIV

122 VATDVCQFLELC QSPD-------------------- GGFGGGPGQYPH LA PTYAAVNALCII GTEEAYNVIN173 NREKLLQYLYSL KQPD-------------------- GSFLMHVGGEVD VR SAYCAASVASLT NIITPDL---221 LFEGTAEWIARC QNWE-------------------- GGIGGVPGMEAH GG YTFCGLAALVIL KKERSLN---269 NLKSLLQWVTSR QMRFE------------------- GGFQGRCNKLVD GC YSFWQAGLLPLL ... 20 aa...331 HQQALQEYILMC CQCPA------------------- GGLLDKPGRSRD FY HTCYCLSGLSIA ...

Bet28 LKEKHIRYIESL DTKKHNFEYWLTEHLRLN------------------- -- GIYWGLTALCVL DSPETFV---

56 LKEEVISFVLSC WDDKY------------------- GAFAPFPRHDAH LL TTLSAVQILATY DALDVLGKDR108 RKVRLISFIRGN QLED-------------------- GSFQGDRFGEVD TR FVYTALSALSIL GELTSEV---156 VVDPAVDFVLKC YNFD-------------------- GGFGLCPNAESH AA QAFTCLGALAIA NKLDMLSDDQ207 QLEEIGWWLCER QLPE------------------- GGLNGRPSKLPD VC YSWWVLSSLAII GRLDWIN---255 NYEKLTEFILKC QDEKK------------------- GGISDRPENEVD VF HTVFGVAGLSLM ...

GGT-P19 LLEKHADYIASY GSKKDDYEYCMSEYLRMS--------------------- GVYWGLTVMDLM GQLHRM67 NKEEILVFIKSC QHEC-------------------- GGVSASIGHDPH LL YTLSAVQILTLY DSIHVI

115 NVDKVVAYVQSL QKED-------------------- GSFAGDIWGEID TR FSFCAVATLALL GKLDAI163 NVEKAIEFVLSC MNFD-------------------- GGFGCRPGSESH AG QIYCCTGFLAIT SQLHQV211 NSDLLGWWLCER QLPS-------------------- GGLNGRPEKLPD VC YSWWVLASLKII GRLHWI259 DREKLRSFILAC QDEET------------------- GGFADRPGDMVD PF HTLFGIAGLSLL ...

Cdc4312 VIKKHRKFFERH ... 103 aa...

127 DKRSLARFVSKC Q ... 52 aa...191 DTEKLLGYIMSQ QCYN-----------------G --;YTSCALSTLALL SSLEKLSDKF240 FKEDTITWLLHR QVSSHGCMKFESELNASYDQSDD GGFQGRENKFAD TC YAFWCLNSLHLL TKDWKMIC--309 QTELVTNYLLDR TQKTLT----------------- GFSKNEEDAD LY HSCLGSAALALI ...

Fig. 5. Repeating motifs in prenyltransferase subunits. Ram1 (Swiss-Prot, accession numberP22007) and FT-13 (Swiss-Prot, Q02293) are the p subunits of farnesyltransferase from the yeast,Saccharomyces cerevisiae, and rat brain, respectively. Bet2 (PIR International, S22843) is the Psubunit of type geranylgeranyltransferase from S. cerevisiae. GGT-P (GenBank, L10416) andCdc43 (Swiss-Prot, P18898) are the p subunits of type 11 geranylgeranyltransferase from S.cerevisiae and rat brain, respectively. The primary structures of these proteins have been shown tocontain a variable number of tripartite internal repeats, each of which is composed of "A" and "B"subdomains separated by a "linker region" containing multiple Gly and Pro residues (10, 35). Whenanalyzed by the Gibbs sampler, these previously defined motifs were identified and additionalcopies were also observed [compare with figure 1 in (35)]. The information per parameter for motifsA, L, and B was 2.3, 2.3, and 2.4 bits, respectively. Dashes indicate the locations and extents ofgaps between motifs; ellipses (... .) accompanied by a number and the abbreviation "aa" indicatethe locations and extents of larger gaps expressed as the number of amino acid residues. Thespacing between motifs L and B is only two or three residues, whereas that between motifs A andL is greater and more variable.

SCIENCE * VOL. 262 * 8 OCTOBER 1993

W1 1. -17M .. i..: ...:---I

on

Apr

il 13

, 200

9 w

ww

.sci

ence

mag

.org

Dow

nloa

ded

from

S £

not been needed to identify even the subtlepatterns described above. The problem ofhighly correlated input sequences can beaddressed by various weighting schemes(37), but we have yet to implement such afeature. Choosing an optimal number ofelements requires further study. We havefound that an additional element is notwarranted when multiple random seeds leadto many different alignments and when theresulting information per parameter consis-tently fails to exceed that obtained fromshuffled sequences (38). Prior knowledgeconcerning amino acid relations (39) hasbeen used profitably in pairwise proteinsequence alignment as well as in patternconstruction methods (8, 40). We havemodified the Gibbs sampler to use suchprior information, but in practice, for evenmoderate numbers of sequences (>5), wehave not found it to yield any improve-ment. However, an interesting new ap-proach to incorporating prior informationhas been described (41), and there is muchroom for further experimentation.

Some basic similarities between ourmethod and several earlier ones should benoted. Stormo and Hartzell and Hertz et al.(5) seek the pattern that maximizes a mea-sure similar to F. Their approach differsmainly in the heuristic optimization proce-dure used, which is an adaptation of analgorithm first proposed by Bacon and An-derson (4). We have implemented themethod of (5) and tested it on a variety ofexamples. This approach uses only a smallsubset of the data for the early sequencesexamined, and thus is easily misled. As aresult, the solution found was rarely as goodas that produced by the sampler. Further-more, the need to construct an alignment foreach possible segment in the initial sequencerequires on average more passes through theinput data than does the sampler (see be-low), resulting in greater execution times.

Both EM methods (42) and the Gibbssampler are built on a common statisticalfoundation. Two EM approaches for multi-ple alignment have been described, block-based methods (6, 7) and gap-based meth-ods in the form of hidden Markov models(43). For multielement problems, theGibbs sampler outperforms block-based EMmethods. Because EM methods are forcedto sum over all possibilities, the time com-plexity grows exponentially with additionalelements. In contrast, the Gibbs samplernever needs to consider more than oneelement at a time. The speed of the samplerstems partly from the fact that it alwaysdeals with a specific model alignment ratherthan a weighted average. Also, because EMmethods are deterministic, they tend to gettrapped by local optima which are avoidedby the sampler. Hidden Markov models,because they permit arbitrary gaps, have

great flexibility in modeling patterns, butsuffer the penalties of this added complexitydiscussed above.

Several other approaches to the localmultiple alignment problem bear a briefreview. Methods that seek a "consensus"word with the highest aggregate scoreagainst segments within the input sequenceshave been described (3). Their space re-quirements effectively limit them to proteinpatterns of six residues, and their time re-quirements effectively allow only closely re-lated words to contribute to a consensus.These constraints greatly decrease the sensi-tivity of these methods to weak patterns.

Algorithms that compare all input se-quences with one another and then coalesceconsistent pairwise local alignments havebeen described (8-10). The MACAW algo-rithm (8) has comparable speed to the Gibbssampler for a relatively small number ofinput sequences and can locate many dis-tinct patterns in a single run. Its time com-plexity, however, is at least quadratic in theaggregate length of the input sequences, andit tends to be less sensitive to weak sequencepatterns. The performance of methods thatmust compare all input sequences with oneanother may degrade as the number of se-quences increases. In contrast, the power ofthe Gibbs sampler and EM methods in-creases with additional sequences becausethe pattern model is improved by more data.As illustrated above, the Gibbs sampler issuccessful even with a relatively small num-ber of input sequences. A version of theGibbs sampling algorithm has been added tothe MACAW program (8), and the updatedprogram is available upon request.

The memory requirements for the Gibbssampler are negligible; storing the inputsequences is usually the dominant spacedemand. When flexible halting criteria,such as those described in Fig. 3, are used, itis difficult to analyze the worst-case timecomplexity of the method. However, fortypical protein sequence data sets, we havefound that, for a single pattern width, eachinput sequence needs to be sampled onaverage fewer than T z 100 times beforeconvergence. In the more time-consumingstep 2 of the basic algorithm, approximatelyLW multiplications are performed, where Lis the length of the sequence that has beenremoved from the model. Therefore, thetotal number of multiplications needed toexecute the Gibbs sampler is approximatelyTNLW, where L is the average length of theN input sequences (44). The factor T isexpected to grow with increasing L. Howev-er, experimentation suggests that T tends todecrease slowly with increasing N when thecommon pattern exists at roughly equalstrength within the input sequences. Thus,linear time complexity has been observed inapplications.

SCIENCE * VOL. 262 * 8 OCTOBER 1993

In conclusion, as illustrated by our ex-amples, the Gibbs sampler objectivelysolves difficult multiple sequence alignmentproblems in a matter of seconds in theabsence of any expert knowledge or ancil-lary information derived from three-dimen-sional structures or other sources. By adopt-ing a randomized optimization procedure inthe place of deterministic approaches, it isable to retain both speed and sensitivity toweak but biologically significant patterns.

REFERENCES AND NOTES

1. S. F. Altschul and D. J. Lipman, SIAM J. Appl.Math. 49, 197 (1989); W. Bains, Nucleic AcidsRes. 14, 159 (1986); G. J. Barton and M. J.Sternberg, J. Mol. Biol. 198, 327 (1987); M. P.Berger and P. J. Munson, Comput. Appl. Biosci.7, 479 (1991); H. Carrillo and D. Lipman, SIAM J.Appl. Math. 48, 1073 (1988); D. Feng and R. F.Doolittle, J. Mol. Evol. 25, 351 (1987); D. Gusfield,Bull. Math. Biol. 155,141 (1993); C. M. Henneke,Comput. Appl. Biosci. 5, 141 (1989); D. G. Hig-gins, A. J. Bleasby, R. Fuchs, ibid. 8,189 (1992);M. S. Johnson and R. F. Doolittle, J. Mol. Evol.123, 267 (1986); D. J. Lipman, S. F. Altschul, J. D.Kececioglu, Proc. Natl. Acad. Sci. U.S.A. 86, 4412(1989); M. Murata, J. S. Richardson, J. L. Suss-man, ibid. 82, 3073 (1985); R. B. Russell and G. J.Barton, Proteins 14, 309 (1992); D. Sankoff, SIAMJ. Appl. Math. 28, 35 (1975); S. Subbiah and S. C.Harrison, J. Mol. Biol. 209, 539 (1989); W. R.Taylor, Methods Enzymol. 183, 456 (1990); M.Vingron and P. Argos, Comput. Appl. Biusci. 5,115 (1989); M. S. Waterman and M. D. Perlwitz,Bull. Math. Biol. 46, 567 (1984).

2. G. J. Barton and M. J. Steinberg, J. Mo!. Biol. 212,389 (1990); C. Chappey, A. Danckaert, P. Dessen,S. Hazout, Comput. Appl. Biosci. 7,195 (1991); E.Depiereux and E. Feytmans, ibid. 8, 501 (1992); D.J. Parry-Smith and T. K. Attwood, ibid. 7,233 (1991);ibid. 8, 451 (1992); E. Sobel and H. Martinez, Nu-cleic Acids Res. 14, 363 (1986).

3. J. Posfai, A. S. Bhagwat, G. Posfai, R. J. Roberts,Nucleic Acids Res. 17, 2421 (1989); C. M. Queen,N. Wegman, L. J. Korn, ibid. 10, 449 (1982); H. 0.Smith, T. M. Annau, S. Chandrasegaran, Proc.Natl. Acad. Sci. U.S.A. 87, 826 (1990); R. Staden,Comput. Appl. Biosci. 5, 293 (1989); M. S. Water-man, R. Arratia, D. J. Galas, Bull. Math. Biol. 46,515 (1984).

4. D. J. Bacon and W. F. Anderson, J. Mol. Biol. 191,153 (1986).

5. G. D. Stormo and G. W. Hartzell Ill, Proc. Nat!.Acad. Sci. U.S.A. 86,1183 (1989); G. Z. Hertz, G.W. Hartzell Ill, G. D. Stormo, Comput. Appl. Bio-sci. 6, 81 (1990).

6. C. E. Lawrence and A. A. Reilly, Proteins 7, 41(1 990).

7. L. R. Cardon and G. D. Stormo, J. Mol. Biol. 223,159 (1992).

8. G. D. Schuler, S. F. Altschul, D. J. Lipman, Pro-teins 9, 180 (1991).

9. M. Vingron and P. Argos, J. Mol. Biol. 218, 33(1991).

10. M. S. Boguski, R. C. Hardison, S. Schwartz, W.Miller, New Biol. 4, 247 (1992).

11. N. N. Alexandrov, Comput. Appl. Biosci. 8, 339(1992); S. Henikoff and J. G. Henikoff, NucleicAcids Res. 19, 6565 (1991); S. Henikoff, New Biol.3, 1148 (1991); M. Y. Leung, B. E. Blaisdell, C.Burge, S. Karlin, J. Mol. Biol. 221, 1367 (1991); M.A. Roytberg, Comput. Appl. Biosci. 8, 57 (1992).

12. P. Bork, C. Sander, A. Valencia, Proc. Natl. Acad.Sci. U.S.A. 89, 7290 (1992).

13. J. S. Richardson, Adv. Protein Chem. 34, 167(1981); A. M. Lesk and C. Chothia, J. Mol. Biol.136, 225 (1980); C. Chothia, Annu. Rev. Biochem.53, 537 (1984); A. Sali and T. L. Blundell, J. Mol.Biol. 212, 403 (1990); R. F. Doolittle, Protein Sci. 1,191 (1992).

213

on

Apr

il 13

, 200

9 w

ww

.sci

ence

mag

.org

Dow

nloa

ded

from

14. F. M. Pohl, Nature New Biol. 234, 277 (1971); 0.G. Berg and P. H. von Hippel, J. Mol. Biol. 193,723 (1987); S. H. Bryant and C. E. Lawrence,Proteins 16, 92 (1993).

15. T. Orchard and M. A. Woodbury, Proceedings ofthe Sixth Berkeley Symposium on Mathematics,Statistics and Probability (Univ. of CaliforniaPress, Berkeley, 1972), vol. 1, pp. 697-715; L. A.Goodman, Biometrika 61, 215 (1974).

16. J. Liu, Department of Statistics Research ReportNo. R-426 (Harvard Univ. Press, Cambridge, MA,1992).

17. In the context of simulated annealing [N. Metrop-olis, A. Rosenbluth, M. Rosenbluth, A. Teller, E.Teller, J. Chem. Phys. 21, 1087 (1953); S. Kirk-patrick, C. D. Gelatt, M. P. Vecchi, Science 220,671 (1983)], D. Geman and D. Geman [IEEETrans. Pattern Anal. Mach. Intell. 6, 721 (1984)]introduced Gibbs sampling as a particular sto-chastic relaxation step. They chose its namebecause a key theorem from statistical physics(the Hammersley Cliffort Theorem) uses Gibbs/Boltzmann potentials. Because we use residuefrequencies based on Gibbs/Boltzmann-like freeenergies, the name is more appropriate here thanin many other applications.

18. M. Tanner and W. H. Wong, J. Am. Stat. Assoc.82, 528 (1987); A. E. Gelfand and A. F. M. Smith,ibid. 85, 389 (1990).

19. Segment x is chosen with probability A/YAi,where the sum is taken over all possible seg-ments.

20. One could choose qJ simply proportional to c,but this would imply a zero probability for anyamino acid not actually observed. This difficultymay be surmounted through the use of Bayesianpredictive inference (18). Bayesian analysismakes use of subjective "prior probabilities" forthe values of the parameters to be estimated. Acommon choice for such priors when multinomialmodels are involved is the Dirichlet distribution [J.Aitchison and 1. R. Dumsmore, Statistical Predic-tion Analysis (Cambridge Univ. Press, New York,1972)], which results in the simple addition of"pseudocounts" to the observed counts, as givenin Eq. 1. These pseudocounts should capture apriori expectations concerning the occurrence ofthe various letters in different pattern positionsand may vary from one application to another. Wehave found that letting bj = Bpj, where pJ is thefrequency of residue j in the complete data set, iseffective. We used this noninformative prior prob-ability in all of our applications. The total numberof pseudocounts B is also part of the prior spec-ification, for which there is no "correct" choice.The smaller B is taken, the greater the relianceplaced on current observation vis-a-vis a prioriexpectation. We have found that choosing BvrN generally works well, yielding pseudocountsb, applicable to the calculation of both q. and p1.

21. H. Akaike, IEEE Trans. Autom. Control 19, 716(1974); G. Schwarz, Ann. Stat. 6, 461 (1978).

22. G is equal toN L'i

F - £ logL'i + 2,Yj log Yij

where L', is the number of possible positions forthe pattern within sequence i, and Y.] is thenormalized weight of position j, that is the weight0a/P. divided by the sum of these weights withinsequence i. This adjustment accounts for the factthat the position of the pattern within each se-quence is not known [see (6) and R. J. A. Littleand D. B. Rubin, Statistical Analysis with Missing

Data (Wiley, New York, 1987)]. It can be shownthat G increases monotonically with increasing F,so an optimization algorithm for F remains appro-priate.

23. A version of the Gibbs sampling procedure forlocating patterns within multiple sequences hasbeen implemented in the C programming lan-guage, and is available from the authors uponrequest.

24. During each iteration, the orders of all elements inN - 1 of the input sequences are available.Counts of observed orders are combined withprior probabilities (taken as uniform in our appli-cations) to calculate "model" order probabilities,analogously to Eq. 1. When choosing an ele-ment's new position, the weights A, of all candi-date segments may be adjusted naturally bymultiplication with these posterior order probabil-ities. A probabilistic model of element locationbased jointly on the observed order and residuefrequency is produced. In our applications wehave set the total number of order pseudocountsto NST/k, where N is the number of sequences, Sis the number of sites per sequence, T is thenumber of types of element, and k has beenchosen as 20. Details of the ordering model willbe described elsewhere (J. S. Liu et al., in prep-aration).

25. We have developed and initially tested a stochas-tic multiple alignment procedure which permitscomplete flexibility in gaps. It uses the sameMarkov characteristic that is used in dynamicprogramming to align two sequences. To date, wefound that the increase in gap flexibility permittedby this algorithm is not worth the cost of theincrease in noise from chance local optima.

26. R. G. Brennan and B. W. Matthews, J. Biol. Chem.264, 1903 (1989); C. 0. Pabo and R. T. Sauer,Annu. Rev. Biochem. 61, 1053 (1992); J. Treis-man, E. Harris, D. Wilson, C. Desplan, Bioessays14, 145 (1992).

27. D. Kostrewa etal., J. Mol. Biol. 226, 209 (1992); R.M. Lamerichs et al., Proc. Nati. Acad. Sci. U.S.A.86, 6863 (1989); M. A. Schell, P. H. Brown, S.Raju, J. Biol. Chem. 265, 3844 (1990); A. Contre-ras and M. Drummond, Nucleic Acids Res. 16,4025 (1988); M. H. Drummond, A. Contreras, L. A.Mitchenall, Mol. Microbiol. 4, 29 (1990); L. M.Shewchuk et al., Biochemistry28, 2340 (1989); H.S. Yuan et al., Proc. NatI. Acad. Sci. U.S.A. 88,9558 (1991); A. Brunelle and R. Schleif, J. Mol.Biol. 209, 607 (1989); S. Spiro et al., Mol. Micro-biol. 4,1831 (1990); C. S. Barbier and S. A. Short,J. Bacteriol. 174, 2881 (1992); M. D. Yudkin, J. H.Millonig, L. Appleby, Mol. Microbiol. 3, 257(1989); C. Berens, L. Altschmied, W. Hillen, J.Biol. Chem. 267,1945 (1992); R. J. Rolfes and H.Zalkin, ibid. 263, 19653 (1988).

28. M. Drummond, P. Whitty, J. C. Wootton, EMBO J.5, 441 (1986); I. B. Dodd and J. B. Egan, NucleicAcids Res. 18, 5019 (1990); D. Akrigg et al.,Comput. Appl. Biosci. 8, 295 (1992).

29. M. D. Yudkin, Protein Eng. 1, 371 (1987); I. B.Dodd and J. B. Egan, ibid. 2, 174 (1988).

30. A. Nagata et al., Proc. Nati. Acad. Sci. U.S.A. 88,4020 (1991); M. C. Peitsch and M. S. Boguski,New Biol. 2, 197 (1990).

31. S. W. Cowan, M. E. Newcomer, T. A. Jones,Proteins 8, 44 (1990).

32. L. Sawyer, Nature 327, 659 (1987); A. C. T. North,Int. J. Biol. Macromol. 11, 56 (1989); Z. Bocskei etal., Nature 360, 186 (1992); H. M. Holden, W. R.Rypniewski, J. H. Law, I. Rayment, EMBO J. 6,1565 (1987); R. Huber etal., J. Mol. Biol. 198, 499(1987); H. L. Monaco and G. Zanotti, Biopolymers

32, 457 (1992); M. Z. Papiz et al., Nature 34, 383(1986); D. R. Flower, A. C. T. North, T. K. Attwood,Protein Sci. 2, 753 (1993).

33. M. S. Boguski, J. Ostell, D. J. States, in ProteinEngineering:A PracticalApproach, A. R. Rees, M.J. E. Steinberg, R. Wetzel, Eds. (IRL Press, Ox-ford, 1992), pp. 57-88.

34. D. J. States and M. S. Boguski, in SequenceAnalysis Primer, M. Gribskov and J. Devereux,Eds. (Freeman, New York, 1991), pp. 141-148.

35. M. S. Boguski, A. W. Murray, S. Powers, New Biol.4, 408 (1992).

36. S. Clarke, Annu. Rev. Biochem. 61, 355 (1992); W.R. Schafer and J. Rine, Annu. Rev. Genet. 30, 209(1992).

37. S. F. Altschul, R. J. Carroll, D. J. Lipman, J. Mol.Biol. 207, 647 (1989); M. Vingron and P. R.Sibbald, Proc. Nati. Acad. Sci. U.S.A. 90, 8777(1993).

38. When sufficient data are available, it is possible toderive an asymptotic test for the statistical signif-icance of any pattern found. In practice, sufficientdata are not usually available for this asymptotictest, especially for protein sequences. Given thespeed of the algorithm, Monte Carlo tests basedon shuffled sequences [S. F. Altschul and B. W.Erickson, Mol. Biol. Evol. 2, 526 (1985)] provide aviable alternative. In any case, Monte Carlo testsare superior under many circumstances toasymptotic tests [P. Hall and D. M. Titterington, J.R. Stat. Soc. B 51, 459 (1989)].

39. M. 0. Dayhoff, R. M. Schwartz, B. C. Orcutt, inAtlas of Protein Sequence and Structure, M. 0.Dayhoff, Ed. (National Biomedical ResearchFoundation, Washington, DC, 1978), vol. 5, suppl.3, pp. 345-352; R. M. Schwartz and M. 0. Day-hoff, in ibid., pp. 353-358; S. F. Altschul, J. Mol.Biol. 219, 555 (1991); S. Henikoff and J. G. Heni-koff, Proc. Nati. Acad. Sci. U.S.A. 89, 10915(1992); D. F. Feng, M. S. Johnson, R. F. Doolittle,J. Mol. Evol. 21, 112 (1985).

40. M. Gribskov, A. D. McLachlan, D. Eisenberg,Proc. Natl. Acad. Sci. U.S.A. 84, 4355 (1987); R.F. Smith and T. Smith, ibid. 87, 118 (1990).

41. M. Brown et al., in Proceedings of the First Inter-national Conference on Intelligent Systems forMolecular Biology, L. Hunter, D. Searls, J. Shavlik,Eds. (AAAI Press, Menlo Park, CA, 1993), pp.47-55.

42. A. P. Dempster, N. M. Laird, D. B. Rubin, J. R.Stat. Soc. B39, 1 (1977).

43. D. Haussler, A. Krogh, S. Mian, K. Sjolander, CISTech. Rep. No. UCSC-CRL-92-23 (University ofCalifornia, Santa Cruz, 1992).

44. For a representative data set, such as that fromthe HTH problem studied above, T- 50, N 30,L 250, and W 20, yielding -8,000,000iterations of the innermost loop. The number ofpossible solutions for such a problem is >1065.Executed on an Indigo R4000 workstation (SiliconGraphics Corp.), the HTH example (Figs. 1 to 3)required 2 CPU seconds on average, while theexamples of Figs. 4 and 5 required 3 and 31 CPUseconds, respectively.

45. Abbreviations for the amino acid residues are: A,Ala; C, Cys; D, Asp; E, Glu; F, Phe; G, Gly; H,His; I, lie; K, Lys; L, Leu; M, Met; N, Asn; P, Pro;0, Gin; R, Arg; S, Ser; T, Thr; V, Val; W, Trp; andY, Tyr.

46. J. C. Wootton and S. Federhen, Comput. Chem.17, 149 (1993).

47. J.S.L. is supported in part by NIMH grant MH-31-154.

14 May 1993; accepted 3 September 1993

SCIENCE * VOL. 262 * 8 OCTOBER 1993214

on

Apr

il 13

, 200

9 w

ww

.sci

ence

mag

.org

Dow

nloa

ded

from