scaling saved searches at ebay kleinanzeigen

Scaling Saved SearchesServing real time push-notifications for millions

saved searches

466382733

André Charton, Christiane Lemke

Who we are?

ebay

kleinanzeigen

≠

ebay

local platforms

some numbers

19M ads

12M searches/day

0.6M new ads/day

6M saved searches

36.000.000.000

theoretical

matches

p r o c e s s

i t !

make it run,

make it fast,

make it stable,

make it better.Special thx fly to Ramkumar Aiyenger – how see some project in the same way.

* * 0/6 * * ?

r e a l t i m e ?

s c a l a b l e ?

C a n

w e

d o

b e t t e r ?

2 0 1 5

src=https://www.esciencecenter.nl/img/main/logo-elastic.png

Percolator

Traditionally you design documents based on your data, store

them into an index, and then define queries via the search

API in order to retrieve these documents. The percolator

works in the opposite direction. First you store queries into an

index and then, via the percolate API, you define documents

in order to retrieve these queries.

src=https://www.elastic.co/guide/en/elasticsearch/reference/current/search-percolate.html

H o w

m a n y

p u s h s

p e r d a y ?

C r e a t e

d e l a y s

f r o m

s i z e

0 - 100: RT

101 - 1000: 1h

1001 - 10000: 2h

> 10000: 6h

a s k

s o l r

s k i p

o n

o v e r l o a d

S e t u p

cloud

2 data centers

cloud

2 data centers

10 + 3 machines

replication x1

shards x80

l i f e t i m e

o f

a

s e a r c h

s l e e p ...

Z Z Z Z Z Z

Z Z Z Z Z Z

e l a s t i c

f a s t

o n

i n d e x i n g

r e m o v e

s l e e p i n g

s e a r c h e s

o n l y 3 0 %

s e a r c h e s

a r e

o n l i n e

d e s k t o p

a v o i d

d b - r e a d

p e r

s e a r c h

b l o o m

f i l t e r

i n

c o o k i e

src=https://upload.wikimedia.org/wikipedia/commons/thumb/a/ac/Bloom_filter.svg/2000px-Bloom_filter.svg.png

a p p s

d e e p l i n k

o n

r e s u l t

s i z e

s t o r e

s e a r c h e s

l o c a l

b a c k e n d

s y n c

o n

a c t i o n s

S a v e d

S e a r c h

S t a b l e ?

S t a b i l i z e

e l a s t i c

Boost your percolator!Tips & Tricks

“This indeed seems like a

large application of

percolate.”Elastic support, June 2015

Performance linear with number of queries

1.Consider using other systems.


“It is worth noting that simple exist matches on a field are probably not a

great application for percolator. This doesn’t utilize any text matching

capability or complex boolean.”

Anything,

anywhere!Everything

for free!

2. Optimise your data structure.

3. Filter, filter, filter!


“The filter only works on the metadata fields.

The query field isn’t indexed by default.”

{

"query":

{...},

"category": CARS

}

{"doc" : {...},

"filters": [

{"or": {"filters":[

{"term":{

"category":CARS

}},

{"term":{

"category":ALL

}}

]}}

]

}

{

"query":

{...},

"category": ALL

}


… what else can we filter?


4. Consider prioritising.

Josh:

Flysurfer kite in Hamburg for less

than 300 Euro

Christiane:

Bicycle in Berlin



Josh:


than 300 Euro

Christiane:

Bicycle in Berlin

Josh:


than 300 Euro

Christiane:

Bicycle in Berlin


HIGH PRIORITYLOW PRIORITY

5. Use multipercolate.

6. Use parallel multipercolate.


● Scenario: 2 shards, replication factor 1 (Shards a1, a2, b1, b2)

-> Queries sent to (a1 OR a2) AND (b1 OR b2)

● Open issue: https://github.com/elastic/elasticsearch/issues/13177

“Currently, to utilise all of your shards, you would need to consider

sending multiple mpercolate requests in parallel.”

https://github.com/elastic/elasticsearch/issues/13177

Outcome

Reduced percolation time:

Outcome

Doubled the number of push notifications:

S t a b i l i z e

e l a s t i c

S t a b l e ?

S t a b i l i z e

p l a t f o r m

6 0 0 0 0 0 0

s e a r c h e s

6 0 0 0 0 0

a d s / d a y

u s i n g

k a f k a

T h a n k

y o u

scaling saved searches at ebay kleinanzeigen

Technology