SnakeSort: A Family of Simple Optimal Randomized Sorting Algorithms

David T. Blackston * Abhiram Ranade

Department of EEC'S University of California, at Berkeley

Berkeley. CA 94720 Email : davidb@cs.berkeley.edu, ranade@cs.berkeley.edu

Abstract We present a family of exceedingly sim­ple sorting algorithms called Snakesort. Snakesort is a natural generalisation of the simple parallel bubble sort algorithm, also known as odd-even transposition sort.. Instances of Snakesort can be implemented on any in­terconnection network so long as it has a Hamiltonian Path. We present two Snake sort algorithms for the hy­per-cube and one for the two dimensional mesh. These algorithms are asymptotically optimal for the respec­tive networks and have very small constant factors. The hypercube algorithms also give linear speedup over the best sequential algorithm. The main attraction of these algorithms is their extreme simplicity. Indeed, it seems unlikely that one could even conceive of simpler sorting algorithms for the hypercube and mesh.

1 Introduction We present a family of exceedingly simple parallel sort­ing algorithms called Snakesort. Snakesort. can be im­plemented on any interconnection network so long as it has a Hamiltonian path. Snakesorl. is a natural gen­eralization of the simple bubble-sort algorithm (also called odd-even transposition sort[3]). Snakesort starts with a single preprocessing step (A' denotes number of processors):

S t e p 0: Fix a Hamiltonian path, and number proces­sors 0 A'— 1 according to their order along the path (Henceforth called the "Snake").

On all networks, the algorithm has the following struc­ture:

S t e p 1: Locally sort the keys held by each processor.

'Research supported by an ONR Fellowship.

S t e p 2: Repeat the following Merge-Splil step suffi­ciently many times:

2.1 Pick a matching from the edges in the net­work.

2 .1 Let (H, c) be an edge in the matching, with it < v according to the numbering along the Snake. Let each processor have k keys. For all (u,v), in parallel, first merge the 2k keys held by the two processors into a single sorted order, and then move the smallest k elements into processor u. and the largest k into processor v.

The odd-even transposition sort is obtained when the network is a linear array (and k — 1). A7 Merge-Split steps suffice, with the matching selected for the odd step consisting of processors (2/, 2 / + 1), and for the even step consisting of processors (2i — 1,2?) [3]. In the case of a richer graph, the hope is that by pick­ing matchings that, involve processors far apart on the Snake, we may be able to reduce the number of Merge-Split steps required considerably. This idea is similar to that of Shellsort.

We present two instances of Snakesort for hyper-cubes called Normal Sort (section 3) and Gray Sort (section 4), and an instance called Fast Shearsort which runs on meshes (section 5). All these algo­ri thms sort, correctly very quickly with high proba­bility assuming that the input is a random permu­tation of the Nk keys, with k sufficiently large. In particular. Normal Sort will sort correctly in t ime 0(k\ogk) + k(\ogN + 1) for k = Ω(A' 2 log"v) . Gray Sort is a modification of Normal Sort that will sort correctly in time 0(k log/:) + k('2\ogN) but improves the bound on k to Ω(A' log.Y). It is easily seen from these bounds that both algorithms give linear speedup

in-201

1993 International Conference on Parallel Processing

over Ihc: I>cst. S T Y ~ ~ I P I I ( . ~ ~ ~ ~ i l lp,ori t . t~~~is. I:A>I, Slitmrsort, w i l l sort ill t.itnr> O(A: logb.) + k ( 2 6 + o ( 6 ) wit11 I3igll pro1ml)llit.y for A. = ~;(f i log ;V), x t ~ d t . l~ is is also o p t - i ~ ~ ~ a l to w i t , I l i l ~ i-III?: COIIS~ ant, farl-or5 for l . l ~ t % I ~ ~ I I . For a l l t.llree algori l l111-r~ it. is possi l j l r 1.0 stloir t.l1;-1t. i i k i s s ~ ~ h s t . a ~ ~ t . i a l I y s t t~ i l , I I ~ r ~,II;III 1 . 1 1 ~ l)o1111(11; g i v w t . l l v ~ ~ ~IIP algorit.llrns w i l l fa i l t .0 sort- cluirk\y wit.l l colrs\.a~lt. yrohal>ilit,;. I t . ran also 1~ ~IIOWII t l ~ i j t . ;I' t I l r ' i ~ i i t . i i l l distril>11 t,i011 of k ~ y s is )lot, t ~ ~ l i l ' o r ~ l ~ . t II<\II ii si1111>l<> r a w domizat.ion schc~ne call t~ 11wd 1.1131. wi l l :111ow t . t ~

algori t.h~ns t.o st~c.crbc.tl !vil.l~ 6111, S~LIIIV l ) r o l> i~ l> i l i t.y H S if' t.111, 111i1,iaI ( l is t - r i l j t~t io11 ~ ( ~ r ( ~ I I I ~ ~ ~ O C I I I , I ~ I I I ~~ I I J , , \+,I, l w t v bh: i t . Normal Sort, is i n fac~. i l 1tor1ua1 itll?;orit.l~nt, a11rl r an I)P irnplemctr~t.etl on st i t~ l l l t~ -c~xct ln~~gi~s~ (.'C'(,!s t?l.c.

using nl-.ll ~ I ~ O W I I t.rausforrna t.io~~s[:i].

1.1 Previous Work

h l t1~11 work hi^^ also h i 3 i 3 1 l r l o ~ ~ e or1 sort . i l~g OII mesl~cs. for c ~ ? ; ; t ~ p l ~ ~ t . 1 1 ~ ~ v o r l i of Sclll iorr and Slla11lir[5], and I,t\igllr.o~~, R.lihkvclo~~ ;I t ~ c l 'I'ollis[.l], who give ul) t imal al- p , ~ r i t . l ~ l ~ ~ s . 01li. t ~ ~ ( * s l j i11gorit.lt111 ill Fact, call I>(. t.liorlght of its :I ( o p t . i ~ n i ~ l . ~bxr rd:ilrt!ly i~l)I~l. t~~i:~t.c.cI) vcrsio~t of t IIP ( I IOI~-O~>~,~I~I~I I ) Sli(*;~rsort i i lgorit 11111 of [?I, O u r al- gori l l \ t r \ i s ro~\>i t l r~r i+ l+ s i ~ > ~ l ) l r ~ ~ t.han t . 1 ~ ol)t . i lna\ al- gorir IIII~S ill Id, 51.

2 Preliminaries

t~ylwrc:~~l , r~ is I i ~ l ~ r ~ l l t ~ c l 11si11g HII n-hit. si.rillg, a110 nodcs tlill'i-ri11g i ~ i :I si11gI11 \ ) i t . art* r o t ~ ~ t ( * r ~ i > l I I>>- a11 edge. For 1,1143 S11:ikt:. \ r ~ 115 t~ t , l ~ i b 1<111ar>- 1b~lIt:cteil ( ira! Code 0 1 1 I , i ) i~,s. ' l ' t l ih ih <l<>li~lt:d r(:(*tjrsiv(:ly 21s ~ o l l o ~ v s , Let { ; , , ( J : ) c( t . r )o l . r> t C ;ray rot\(\ II\III~I>I'F oI' a prDCesSOr la- I j t ~ l l ( ~ l .r i t 1 a 2" ~loclv j~v j~cxr r~~t ) t= . \\:P ]lave Gl (O) = 0 i11)<1 (-;l(l) = 1. F t~r t , l~~ : r , Cl,(O.r) = G , l - l ( . r ) , aiad G,,(i .t .) = 2'' - I - C : , , - , ( J . ) ~ wtirre .I. i s all 7 1 - 1 \>it st , r i~ lg. 11' II is a11 1 1 hit, s t , r i ~ ~ g . (1~i i11e .YIICC,~(U) t,o be I I I I~ 1 1 ljit S I , ~ ~ I I ~ S ~ I V ~ I t11:il (.;,,( 1 1 ) = ( ~ l h ( . ~ t ~ c ~ , h ( * ~ ) ) - 1.

OJI r a11;ilysi.s I I S ~ : ~ ;I ~~II( IOI 11 ixvd versi011 o f [.lie 0- 1 Sort.itlg Lt :~~ln~a[: l ] . ?'hi, s~ - i i ~~ t I : i r ( l vers io~i of the 0-1 Sort ing I , c n ~ ~ ~ l n st.ilt.cls t.llat. if a11 o l~ l i v i ous algor i thm sorts 311 xero-o~ic, i111)tlt. i l rr i lxs correctly, t11e11 it will sorl, ally ~II[)II~. arr:~y, i v l ~ ~ : r ~ . i k l l a l g o r i t l i ~ i ~ is said t.0 be ol)lirt;o~rs i l ' t.llit o ~ ~ l v o l> t= ra t i o~~ i t 1>erTor111 is t,o cottl- luli-t, I ~ v o ~ ~ \ I I I ( ~ I I ~ , S 01' t-II<-b I I I~I I I ;III~ swit ,c l~ ~,IIYIII if t,lley ill.? o ~ ~ t . of o t . t l~~r , a ~ l t l , i ~ t i~ ( l t l i i . i r l~ t , al l 1.1w compacisous art5 sl~+:viii(~cl i t ) ~<IV;III~:~:.

L(!llllllil 1 If. Sol- C 1.!/ 0 5 1 ) 5 1 . (1 11 oblilrio N S 0.lg0- 1 - t t 1 ~ 1 1 t s ~ l , / a N 1:cgs ulitlt y?~oI)nb~l i ty rrt l e o s ~ 1 - 6 1st

llrt cnsr tlrrri er1c11 bcy t s r i l ~ d r p r ~ ~ d r ~ r i l y sct to 1 with ~ ) r o b n b i / t / y 1) ( I I I ~ 0 n t I t e i - ~ ~ : ~ s ~ , i h f i ) ~ P Y t l ) pl .obnbil i fy crt Ir(8sf I -(:V + 1)6 t b c nl~jur i i l r~r) sol-1s ctlrg mud0111 per- 1ll 11 /(/, 1 1 0 )/ -

SOY has :I I a r p I I I ~ ~ ~ I > ~ T o l ' k~%y+ (wl i ic l l ih w l~a t , w3 WII- 'rtw ~>roo l ' call hc I b u ~ ~ r l ill 1 1 1 ~ f't111 vcrsion of th is

si~lt'l. l~rrc\). I,~:I~IIIOH'F ~'~IIIIIIII>OI.~ iilgori ~.IIIII[~] call 1jv 1"11""'.

i ~ s t ~ ( l t,o <lerivrb work ( r l r * t ~ ~ r t ~ ~ i ~ ~ i s t ic) opt,i111a1 S O ~ I , I I I ~ al- gor i t , t i~~i .s for l~ ,vpvrct~l ) i r ~~(bt-worl;s. 0 1 1 r ( ~ ~ I I I ~ I ~ I I I ~ V , < ~ ~ I ) '1'111. p r imary t t~a t . Iw~nat . ic i~ l loo1 11srt1 i n t.he analyses aIgorit.111112 fir,, co~~sitlvr;il>lh- sill~l~It.t., ant1 I~avi , sl igl~t. ly o f 0111. i l l g o r i t . l i ~ ~ ~ s is t .11~ F o l l ~ w i ~ ~ ~ I r r n ~ n a , which can s111aI1t>r ~ O I I ~ I , ~ I I I f i ~ r t ~ t + ~ . ljv I ~ ~ O \ . I ~ I ) I I S I I I ~ (.'II~:~IIO~~' I > o ~ I I ~ ~ t v c l l ~ ~ i q t ~ e s .

1993 International Conference ort Parallel Processing

Leinlnn 2 Suppose 11. - . - , S,, n re

mdependent BernorIlz random ~larrables wrth parom- e t e r p. Lt I S = I t S,. rt.r/b €[XI = r r p . Tlrra for any J betweel~ O and I

3 Normal Sort

The Snake for Yorrnal Sort is the Binary Reflected Gray Code. The algorithm has three pl~ases:

1. Each processor does a local sort of i1.s ke!.s.

2. ( I I I P ~ ~ P Phase) For i ru lming f rom H to 1 . procrs- sors t.ha~ are conl~ec tpd a l o l l ~ ; d in le t ls~ou i ~~erfor l l l a hIerge-Split.

:I (.\lop-l'p Step) For a l l r lv i t l l GI,(x) orld. 00 a

l lprgp-Spli t Iwt WPPII J and ~ ~ r r , , ($1.

Sote t b a ~ the iiiprgc: p l~ase 1s a uor lnal alc;orill1lii[:3]. and the 1iio11-11p st el) can a l m 1 , ~ t'ie\v\'rrl its a t l ~ ~ t t ~ i q i aI- go r i t , l ~m by espcut ir ig i t OIIP i l i i n p ~ ~ s i o n at a r i t ~ l f . . Il'i! li this change. Sormal Sort call b e po r t rd l a 5l111IRe- exchanges. But ter flips, CC'Cs PI c.

I\'? a i~a l ) ZI? S o r ~ ~ i a l Sorr 11>11ig t !I* ~:\~I~JQIIIIX[~~I n a r - siou of the U- 1 Sort iuq L ~ ~ n r n a , rrhicl~ c~llows 11s to l i ~ i i ~ t our analysis to inputs ill w l ~ i c l l kpys a r r pt!lt~i- 0s OF IS. !Ire I E P ~ ~ a few r l e l i ~ i i t,ioi\s. \\'p say ;I s u b c i ~ l ~ c at' the hypercube i s c l ~ n t l II i t s processors c o n ~ a i a all otws or a l l zerops, and d ~ r t 11 i f t III~IY, ;~r t \ hot 11 ZP~QI*S ~ T I ~ I OIII%~

prespnr. IYP also (IPFII~P il C ~ I ~ C I I ~ > P to I l r ~arr t r - r l tn l ) i f tl lerc are S~rr\\+pr t II~II k 011~s or A- Zl?TOeS l l r ( w l i t over I 1112

c ~ l ~ t i r e st l l>cl~I)r. \Ye S C ~ ~ , C t 11,: Ibl low.~ 11g I t * l l ~ l i i a rir llo111 proof.

1k.p iws t ro~lsid~r ~IIP case i t1 1vliicI1 t l ie i l ~ l ~ l l t to f IIP >or1 iizg algorir ill11 is bninj#rrd. I .V. I lir i l ipur is wro-onr. ,71t(1 t Iwre exists r i t ~ ~ ~ l l l r r s 0 5 f~ 5 .1 5 i h i ~ c l ~ 111:tt IIIP r i u r ~ ~ l ~ ~ r oi' ZVI.OVS 111 t.;tcll ~ruc.~>%sor i:, Eli,! \v~*.II r t A. i l11(1

1 IL.. :111t! + - 1) 5 7

[Sketch) \fTe prove the lemma by induction on IV. If .v = 1 r l iw l abet the locat sorc the input is sorted, and ive are clone. Suppose N > 1. and consider the effect nF I IIC (Erst J l c r y - S p l ~ t atep. 'Ttlete are two cases to consider.

Suppose a 5 .I < 4 or 4 < a 5 0. In this case, it can be sliown tlrat ~ P P t.1; first Merge-Split step one of t l ~ e a u l ~ c u h ~ s 1, clea~l . and the other 1s balanced. T l i e induction hypotl~mis now holds.

If a 5 5 3 then it can l ~ e shown that afr.er I he Rrst .\ltsge-Sdit step. hot 11 stihcuher, are near-clean. The N~ar-C'1~a1i Lenin~a now implie that after t l ~ e Jlerge Pllase Imth subcubes w i l l be sorted. The Mop-up Step l l oa ~ielfices to C O ~ I I I > I P ~ P ~IIC w r t .

Tlienrc~t~ 1 ,i;trpp.rf 1r.r run .i'nrrnnl .i;ort o n rin iY ~rorlc b y p r r c t l b r rcllh k kryp per node. The nigo- r rfhrrr tr .111 \or! n raadatt~ pcrrnrrlnlroti c o r ~ c t l y uslrig lo$.\' + 1 .Ilrryr-,5plrt ctcps rrath probnbrirtpr ni least 1 - 2.\'(.l'k .t 1 ) ~ - I...IL. l[X. = R[.\.'log1\1). ilten the ~ I ~ o r t r h t t r f n t t aifb ~ r o l y n o t t l t n l l y stnoll pmbnbrltty.

i S k ~ t r h ) Fis a prol,al~i l i ry p, a i d s u p p w t llak each ~ I I J ~ I J T 1s 1 wi111 prolx+l)il ity 1) HIHI 0 r3 t l l e r~ . l~e . \Ye ap- lrly Lrtnma i! to sllolr. t h ~ t 111 t his rase the probability

I l ~ t r l ~ c i ~ l p u ! is balaucetl is at. Irast I - 2 . i ' ~ - * . Tltta ra~~ t l am ize t l 0- 1 Irlnm;i I ~ O W i ~ i ~ p l i e s t ha t t.he algo- r i l I I : ~ w i l l sort a rai i( lom p r r n ~ ~ l r a t i o n rvlt l i probabi l i ty

k

:!I Iraqt 1 - 2.2'(.\'A. + t )r-I-. .- , and he l>o~ lnd on C is rasily ohs~rvetj.

4 Gray Sort

2 5 III t l ~ e case of 3or1nal Sort. t lie S~take lor Gray Sort 1s also the Bil lary R e H ~ c 1 ~ r l <;ray Code. Let If I)e a

2" notle I ~ y p ~ r c u h e . a i ld Hn and f f l he the subcubes wir11 Ir*asr sigl~ificaut bit. U a11d 1 respect~vely. Gray 5ori 1s rlr:fi~ied rpcursively for a 2" node l i ypercu l~e a5

101 10\\.9.

* 11 I t = 1 E do a local sort..

I f 1 1 > U r he11 rlo r Ire follu\r.~rlg

1 . 111 paral lr l . r lr l i ( ; r ay Sort 011 H d l a l ~ d I.!

2 . Tor a l l .I+ ~ v i t l l (;,,(r) eveil, do a llerge-Split h1'1 \YPPll .l' i l l 1 4 1 .+t!Cf',3 ( J ' ) .

1993 International Conference on Parallel Processing

rrl l is a l g o r i t . i ~ ~ ~ i is w r y ai111i1;ir 1 0 Y<)t-t~i, l I Sor l : 111 -I'II(? S(-VOII(I ~ I I ( I 1,111rd i > l ~ . w s car11 r t q i ~ i r ~ fi fact it. is va.<il>. s e t ~ t,I~at, t,Ilv ~ I ~ O ~ I ~ ~ I I I I ~ , < I I I ~ ~ A > or i lit, hlt-rga~-Sl~l i t . SI,~:I\S? ~ I I ~ I it, C ~ I I I)(> ~IIO~VII t,t1;1! il' k is Merge-Split st ~1)s of Norn,~ i l l So1.1. ;ill t , r ~ b i ~ ~ . i ~ ~ g \\-1t.1\ I I ( ~ \ V 1i11 $,(' (-IIOII~II t 1 1 4 ' 1 ) I Iltb fi11aI I'OIIIHI of ~ ~ i t ~ ~ ~ l l ~ l lbt~lible hlerge-Split, st ,~lis that. acl. a:. h! 01,- ( I 1, s\ tll)c;. 'Kl~c~sib wrt ;IIOII~ i IIP r ~ \ v > will oa~ ly s t ~ c ~ ~ ~ i s ~ ~ o( fl) klcrge- I ) P W kl t>rg~-Sl i l i t . slq=ps allorl- 11s 60 i ~ ~ ~ l > ~ . o v c % 1.11rb I)OIIII(I ~ , , l , ~ kt,,l)5, a~~o,,.,llK tkcjI(,it-,: t,ll,, ,.,> <,,, il-,>cj ~ ~ ~ 1 1 1 1 ~ ,

on A. w 1it.r t .11~ rollorvillg t , l ~ c - ( ~ . c \ ~ ~ i u b l b i c l ~ \~.c~st.iirc* ivit.11- l ' l ~ t * l o l l o w i ~ ~ g I,~I(~O~~~III. s i , ~ t . ~ ( l wi t I~o i l t , 11rooS. s1111111ia- o\l t. prool'. ri>.i\> 011r r(*s11It-5 for 1111s idgorit,11111.

T ~ I C O ~ C * I ~ I 2 11 (;ruy ,\'(>I,/ I . I I I I ( + I I (111 .Y ~ t o d ( 11,y- T l u : o r c ~ ~ ~ 3 If IJ{I \I , \ ' / j ~ ( t r , w r I i ,s r t t j l O I I f i x 0 I)<)-r 1 t h f ?1#1t1> A+ ktys p c ) . 11odf. / / I ( 11 1 ) t - 0 1 1 t i o ) t t prr- ~rrcrb 1r.1111 A. = ;(n lop, ,Y) kry.5 per rrmrlc. lbc)r i b e 1)l l l i t t t l # ~ t I,% ,%O?- / ( t l corr~rI1.q l l . ~ I ? l , o 2 log .Y .\ti 1 ~ ~ ~ - . ~ ~ 1 I l l

~ ~ l g ~ , r l / l , ~ , t tl*,/l r ( ( , , , , l - ( 2 f i + o ( f i ) , l l tr!,~-~~/~li/ s t c j 1 . 5 s 1 ~ p . 5 t1j1i11 pr-o~)ohil~/,t, ( 1 1 l < t ~ s / I - rlN(:VL* + I) [ -& .

t11ltl , s o t I 4 l , l , l t It!/ ~ / ! ( l l l ] ~ ~ - ~ , ~ l ~ t l ~ t / t / ~ ~ /It ( 1 1 g0i.v j l i > l . q l t # l I ~ ~ - li A- = i2( .v log ,\' ) l i t { I 1 l l i l % ~ l l - ~ ~ ~ t ~ 1 i ~ l l l ~ ~ ~ ca4/ ill l # l f 1 ( / 4 111lq lo O / t ( p o l q 1 1 o 1 1 1 1 ( / 1 1 ~ j cl(*.v( t o 0 1 1 < .

5 Fast Shearsort References