hannu peltola jorma tarhio aalto university finland variations of forward-sbndm

Hannu Peltola Jorma Tarhio

Aalto University

Finland

Variations of Forward-SBNDM

Aug. 29, 2011

Aims

Tuning algorithms for exact string matching.

Studying the effect of simultaneous 2-byte read.

Aug. 29, 2011

SBNDMSimple Backward Nondeterministic DAWG Matching

SBNDM [18] is a simplification of BNDM [17]. Both are bit-parallel algorithms.

Text T = t1...tn, pattern P = p1...pm.

At each alignment window of P in T, scan T from right to left until the suffix of the window is not a factor of P or an occurrence of P is found.

Aug. 29, 2011

Shift of SBNDM

No factor: m

P found: 1

Else: next alignment starts at the last factor

Aug. 29, 2011

SBNDM, example

P = banana, T = antanabadbanana...

alignment: antanabadbanana a na ana

Aug. 29, 2011

SBNDM, example

P = banana, T = antanabadbanana...

alignment: antanabadbanana a na ana

not a factor: tananext alignment: antanabadbanana

Aug. 29, 2011

SBNDM, example

P = banana, T = antanabadbanana...

alignment: antanabadbanana a na ana

not a factor: tananext alignment: antanabadbanana not a factor: dnext alignment: antanabadbanana

Aug. 29, 2011

SBNDMq

SBNDMq [6] is a tuned version of SBNDM.

Processing of an alignment starts with checking a q-gram.

Let q = 4. Consider an alignment at antana. Instead of testing four suffixes a, na, ana, tana,only tana is tested.

Testing is done in a fast loop.

Aug. 29, 2011

Forward-SBNDM

Forward-SBNDM (FSB for short) by Faro & Lecroq [7] is a lookahead version of SBNDM2.

Both FSB and SBNDM2 read a 2-gram x1x2 before a factor test.

x1x2 is matched with the end of P in SBNDM2.

Only x1 is matched with the end of P in FSB, and x2 is a lookahead character following the current alignment.

FSB is faster than SBNDM2 for large alphabets.

Aug. 29, 2011

Generalization of FSB: FSB(q,f)

FSB(q,f) (= Forward-SBNDM(q,f)) is SBNDMq with f lookahead characters, f = 0, 1, ..., q-1.

FSB(2,1) = FSB and FSB(q,0) = SBNDMq.

Motivation: SBNDMq works well on modern processors also for q>2.

Aug. 29, 2011

FSB(q,f)

Let UV be a q-gram, where |V| = f.

After reading UV there are 3 alternatives:i. If U is a suffix of P, reading continues leftwards.

ii. Else if UV is a factor of P, reading continues leftwards.

iii. Else the state vector is zero and P is shifted m-q+f+1 positions

(f positions more than in SBNDMq).

Aug. 29, 2011

Occurrence vectors in FSB(q,2)

Example: P = banana

bananaSBNDMq: B[n] = 00001010

FSB(q,2): B[n] = 00101011 B[a] = 01010111 B[x] = 00000011

extra bits

Aug. 29, 2011

State vectors in FSB(q,2) for q=4

4-gram nanx: x 00000011 n 00101011 a 01010111 n 00101011

00001000

4-gram State vector Conclusionnanx 00001000 na is a suffix of Pxana 00000000 not a factoranan 01000000 factor of P

nanx is not a factor

Aug. 29, 2011

Benefits / drawbacks of lookahead characters and extra bits

Benefits

• Longer shifts more speed

• Combined suffix/factor test

Drawback

• More q-grams accepted less speed

Aug. 29, 2011

Greedy skip loop for SBNDM2 (GSB2 = Greedy-SBNDM2)

Factor tests of two 2-grams are done in one round.

Let B2[x,y] denote the combined occurrence vector of characters x and y. B2[x,y] = B[x] & (B[y]<<1)

next:D B2[ti,ti+1]if D = 0 then if B2[ti+m-1,ti+m] = 0 then i i+2*m-2

goto next

Aug. 29, 2011

2-byte read

Read two characters (= 2 bytes = 16 bits) in one instruction (in a skip loop).

Suits well q-gram algorithms with even q.

For experiments we made two versions of the algorithms:• Standard (1-byte read)

• b-version using 2-byte read

Aug. 29, 2011

2-byte read (cont.)

Advantage: a part of computation can moved to preprocessing phase

• Example: B2[x,y] = B[x] & (B[y]<<1)

Speed-up factor even more than 2

Drawback: extra 0.1 ms for preprocessing.

Aug. 29, 2011

4-byte read?

Many border crosses happen => slow down

232 tables too big for practice

Aug. 29, 2011

Experimental results/KJV Bible

In the recent comparison S. Faro, T. Lecroq: The Exact String Matching Problem: a Comprehensive Experimental Evaluation

(2010), the algorithms EBOM and Hash3 were the fastest

in the bible text for m = 4,...,20.4 8 16

Hash3 14.6 5.42 2.79

EBOM 6.53 3.87 2.91

Aug. 29, 2011

KJV: EBOM & Hash3 (on ThinkPad X61s)

0

0,5

1

1,5

2

2,5

3

3,5

4

4 8 12 16 20

m

GB

/s

EBOM

Hash3

Aug. 29, 2011

KJV: EBOMb & Hash3b (with 2-byte read) added

0

0,5

1

1,5

2

2,5

3

3,5

4

4 8 12 16 20

m

GB

/s

EBOM

EBOMb

Hash3

Hash3b

Aug. 29, 2011

KJV: SBNDM2b = FSB(2,0)b added

0

0,5

1

1,5

2

2,5

3

3,5

4

4 8 12 16 20

m

GB

/s

EBOM

EBOMb

Hash3

Hash3b

FSB(2,0)b

Aug. 29, 2011

KJV: GSB2b added

0

0,5

1

1,5

2

2,5

3

3,5

4

4 8 12 16 20

m

GB

/s

EBOM

EBOMb

Hash3

Hash3b

FSB(2,0)b

GSB2b

Aug. 29, 2011

KJV: FSB(4,i)b added, i = 0,1,2

0

0,5

1

1,5

2

2,5

3

3,5

4

4 8 12 16 20

m

GB

/s

EBOM

EBOMb

Hash3

Hash3b

FSB(2,0)b

GSB2b

FSB(4,0)b

FSB(4,1)b

FSB(4,2)b

Aug. 29, 2011

KJV: Speed-up factors of 2-byte read

GSB2 1.32FSB(2,0) 1.34FSB(2,1) 1.24FSB(4,0) 1.72FSB(4,1) 2.15FSB(4,2) 2.03Hash3 1.05EBOM 1.17

Aug. 29, 2011

Other experiments

DNA and binary data was also tested.• Gain of lookahead characters or the greedy loop was smaller

than with the bible data.

Gain of 2-byte read was smaller with 64-bit code than with 32-bit code.

Aug. 29, 2011

Conclusions

Two new algorithms were presented: • FSB(q,f)

• GSB2

The new algorithms are faster than earlier algorithms on English data:• GSB2 for m = 4, …, 8

• FSB(q,f) for m = 8, …, 20

2-byte read makes most string algorithms faster.

Aug. 29, 2011

Web site for practical speed comparison

cse.aalto.fi/stringmatching