ux: internal search for e-commerce

SEARCH ENGINE RANKINGS

Internal SearchFOR E-COMMERCE

2016GET STARTED

Apache SOLR: full text search capabilities and rich document handling

Elastic Search: schema free, REST and JSON based document store.

Internal Search Engines in this presentation

2

Search experience and performance are heavily influenced by non-visible factors, such as search logic and product data integration.

Intro

3

Out of the 50 top grossing US ecommerce websites, few provide a great internal search experience.

The state of e-commerce

4

82% have auto-complete…36% of these are detrimental to UX

70% requirejargon

Only 40% offerFaceted search

60% fail withabbreviations

Or symbols

16% fail with model or product number searches

18% fail with misspellings

1.Gauging the competition’s search experience requires extensive testing and evaluation.

2.That means that your internal search efforts can’t be easily copied by competitors.

3.Poorly performing search experience can be pretty.

A few things to keep in mind

5

Half prefer to use on-page navigation while 47% prefer to filter down on the product page.

Onsite search vs Onpage navigation

6

65% of test subjects required 2 or more attempts to complete their search

Reality check

7

•Navigational: reach a specific page

•Informational: acquire information

•Transactional: perform a web-

mediated activity.

3 Categories of « intent »

8

These are the same ”intents” as for

generic web searches

Navigational

Transactional

Informational

• Location – top right hand corner• Simple search + link to advanced

options• Case sensitivity• Search labels – just call it search!

• Put text in the search box • Search bar in a different color

Best practices 101 – Search Engine

9

LOCATION

SEARCH LABELS CASE SENSITIVITY

DIFFERENT COLOR

TEXT IN SEARCH BOX

SIMPLE SEARCHADVANCED OPTIONS

Websites with semantic search engines have lower rates of cart abandonment.

Why?

« red sneakers size 9 » shows a search intent that’s further along the conversion path than « sneakers »

Best practices 101 – Search Engine

10

LONG TAILSEMANTICSEARCHES

HIGHLIGHTRESULTS

AUTOCOMPLETE

COMMONMISPELLINGS

• Keep the search query • Give filterable options• Go beyond generic filters • Have clear titles and descriptions

Best practices 101 – Search Result Pages

11

Takeaway: don't force users into a tunnel of limited search results.

Let them check, uncheck, clear, refine their way to a better search.

That often leads to higher conversions.

Best practices 101 – Search Result Pages

12

• Fine tune the number and presentation of search results

• Fine tune how and what gets returned

• Search Analytics: get data on the way users

search on your site

• Let search behavior guide site structure for

better information architecture & UX

• Let search behavior guide site content

• Using constrained search to reflect a strict

information architecture in search

Search is good for the soul…and the site

13

DIG INTO SEARCH ANALYTICS USE CONSTRAINEDSEARCH

LET SEARCH BEHAVIORGUIDE SITE STRUCTURE

LET SEARCH BEHAVIORGUIDE SITE CONTENT

• Unique URLs for each search result for long tail SEO traffic.

• Use search result pages for PPC • Google Search Console for keywords• Google's “sitelinks search box” • Highjack search queries

Search Engine Optimization and Marketing Tips

14

Unique URLs

Use search keywords

PPC Landing Pages

Highjack Search Queries

Sitelinks Searchbox

Google Search Box

15

Here are 7 points to get started:•Avoid returning low relevance results •Map synonyms & misspellings•Map symbols, abbreviations •Audit auto-suggestions •Allow users to iterate •Implement faceted search •Provide hierarchical breadcrumbs & history-based breadcrumbs

Search: a competitive advantage

16

BROADEN THE SCOPE

SMARTAUTOSUGGESTIONS

SYSNONYMSAND MISSPELLINGS

FACETED SEARCH

PREFILL SEARCHFIELD

ABBREVIATIONSAND SYMBOLS

BREADCRUMBS

There are 3 highlighters:

•Standard Highlighter: The swiss-army knife of the highlighters. •FastVector Highlighter: it works better for more languages than the standard highlighter. •Postings Highlighter: This highlighter a good choice for classic full-text keyword search.

https://cwiki.apache.org/confluence/display/solr/Highlighting

Highlighting Results in Solr

17

Tip: for better results and performance go for a fuzziness parameter of 1 (string of 3, 4 or 5 characters).

Typos and Misspellings in Elastic Search

18

Fuzzy matching allows for query-time matching of misspelled words. It functions by building a Levenshtein automaton (big graph with all the strings) of the original string.

https://www.elastic.co/guide/en/elasticsearch/guide/current/fuzziness.html

Solr has a Phonetic Filter that has the double methaphone algorithm.

The SpellCheck component helps provides inline query suggestions based on other, similar, terms.

You can do this with terms in a field in Solr, externally created text files, or fields in other Lucene indexes.

SOLR and misspellings

19

https://cwiki.apache.org/confluence/display/solr/Spell+Checking

• Audit auto-suggestion from the website’s search logs

• Machine learning should be based on the success rate of a query

• Filter out duplicate suggestions• Allow users to iterate on auto-

suggestions

Auto-suggestions

20

Autocomplete affects how and what a user searches for.

Aim for 6 out of these 8 things:•Style Auxiliary Data Differently•Avoid Scrollbars – show 10 items max•Highlight the differences •Support Keyboard Navigation•Treat Hover Expectations as a non-committal actions•Show Search History CSS :visited selector•Reduce Visual Noise•Consider Labels & Instructions

Autocomplete design patterns

21

The Completion Suggest feature is built for extreme speed (at query time).

You can find out how to do all this here: https://qbox.io/blog/quick-and-dirty-autocomplete-with-elasticsearch-completion-suggest

Quick autocomplete with ElasticSearch

22

Persistent search makes the iteration process less frustrating.

Persistent search

23

Faceted Search

24

Users can’t always Specify their queries

Think of design detailsAnd filtering logic

Faceted Search offers filters

Faceted Search

25

DynamicLabelling system

Map filtering types to the users' purchasing

parameters

Provide productspecific filters

Damned if you do, damned if you don’t!

Putting everything in no-index or letting everything be crawlable are not good SEO options.

Faceted search and SEO

26

Let’s go for aggregations to sculpt precise multi-level calculations that occur at query time within a single request.

Multi select within active bracket significantly improves and simplifies navigation experience for end customers.

Elastic Search Facets and Aggregations

27

Hierarchical breadcrumbs are great for non-linear navigation History-based breadcrumbs give a way to go back to search results

Breadcrumbs – Hierarchy & History

28

SEARCH ENGINE RANKINGS

Users combine 12 query types mapped in 3 groups:• Spectrum• Qualifiers• Structure

The Search Query

29

• Query spectrum: base of the search query.

• Query qualifiers: refine the boundaries of the query spectrum.

• Query structure: how it should be interpreted.

Spectrum, qualifiers and structure help design search logic that aligns with user behavior and expectations. 

Anatomy of a search query: spectrum, qualifiers & structure

30

QERY QUALIFIERS QUERY STRUCTURE

QUERY SPECTRUM

1. Exact search

2. Product type search

3. Symptom search

4. Non product search

Query Spectrum – Setting the range

31

The query spectrum is used to indicate the range of what should be searched

• Product title or number search• Handle phonetic mistakes, products

having alternative titles.

The logic should search the entire data set to broaden the query’s scope.

Exact Search

32

Exact search is the simplest query type

Product type query: the user knows the type of product he/she wants but not the particular product.

This requires: •Detailed categorization & product labels •Proper handling of synonyms & alternate spellings of those groupings

Product type searches

33

Synonym mapping can be added two ways:

•Two comma-separated lists of words with the symbol “=>” between them. •A comma-separated list of words.

Modify the synonyms.txt file located under the folder \server\solr\jcg\conf:

Mapping synonyms in Solr

34

Users look to solve specific problems and want products to help them solve it.

Symptom Searches

35

Search engines should also handle auxiliary content search like that as often users will have a hard time finding these in the navigational links.

Non-Product Searches

36

Conditions for what should and/or shouldn’t be included.

Thematic, compatibility and subjective searches are a little more challenging from a technical perspective, but they are often used by users.

Query Qualifiers – Delineating the search boundaries

37

• Feature search• Thematic search• Relational search• Compatibility search• Subjective search

Feature is the most common qualifier.

Feature definition: any type of product aspect or attribute.

• Color

• Material

• Performance specs

• Format

• Price

• Brand

Feature Searches

38

TinyCross

PatternCushion

CrossMedical

Tiny

This a common browsing pattern and product arrangement in physical retail.

Good product categorization and labelling will get you half the way there.

Thematic searches

39

• Seasons• Intended usage (outdoors, office, etc.)• Occasions (birthday, wedding)• Events (NBA, Olympics)

Relational searches are searches where users enter the name of entities involved with or related to the product.

Relational searches

40

Users often don’t know the name of the accessory or spare part they need – instead they know the details of the product they already own.

Compatibility Searches

41

Common structure of a compatibility search: Name of brand + type of accessory or spare required

Subjective qualifiers like “high-quality” or “cheap” are often vital to the user’s purchase decision.

Tips1.Approximate user intent by using one or more attributes as a proxy2.Identify an attribute that could serve as a useful proxy. Deconstruct what the query is about.

Subjective Searches

42

1. Slang, abbreviations and symbol search

2. Implicit search

3. Natural language search

Query Structure – Constructing the query

43

It deals with how the query is constructed by the user. It cares about the context, the syntax and the search engine interpretation.

Users rely on a wide range of linguistic shortcuts when they search.

Slang and abbreviations are easier to support.

Symbols change meaning depending on the arrangement of the query.

Slang, abbreviation and symbol searches

44

• Slang: “RayBan shades”• Abbreviations: “13in laptop sleeve”• Symbols: “sleeping bag -5 degrees”

Detected implied components can alter the search experience.

•Bias when a search is done from a category. •Suggest relevant search scopes or query clarifications - “did you mean…”•Auto-refine: auto-correct to include implied components.

Implicit Searches

45

Certain aspects of search queries are left out because of the user’s context.

It’s about understanding semantics, context and relationships of the query instead of parsing the query as a set of keywords.

Natural Language Searches

46

Category:Women

Product type:shoes

Variation:red

Variation:Size 7.5 RESULTS

QUERYRed women’s Red women’s

shoes 7.5shoes 7.5

Start with these 5 query types: •#1 Exact•#2 Product Type•#5 Feature •#6 Thematic•#7 Relational Searches

Not investing in good search usability can cost sales in the short, mid and long term.

Improving support for the 12 queries

47

FEATURE SEARCH

THEMATIC SEARCH

RELATIONAL SEARCHES

EXACT SEARCH

PRODUCT SEARCH

Query spectrums (what should be searched)

48

Query Type User behavior How you can support it#1. Exact Search“Keurig K45”

Searching for specific products by title

Basic keyword matching, along with support for multiple title variations and intelligent handling of misspellings

#2. Product Type Search“Sandals”

Searching for groups or whole categories of products

Support for synonyms as well as categories that aren’t part of the site’s navigation / hierarchy

#3. Symptom Search“Stained rug”

Searching for products by querying for the problem they must solve

Symptom database mapping “symptoms” to “cures” (i.e. problems to solutions)

#4. Non-Product Search“Return policy”

Searching for help pages, company information, and other non-product pages

Search engine must index the entire website, not just products

Query Qualifiers (specify condition)

49

Query Type User behavior How you can support it#5. Feature Search“Waterproof cameras”

Searching for products with specific attributes or features

Intelligent parsing of product specifications (i.e. structured product data)

#6. Thematic Search“Living room rug”

Searching for categories or concepts that are vague in nature

Interpretive labelling of products and categories

#7. Relational Search“Movies starring Tom Hanks”

Searching for products by their affiliation with another object

Association data linking products and objects, ideally specifying the nature of the relationship too

#8. Compatibility Search“Lenses for Nikon D7000”

Searching for products by their compatibility with another item

Compatibility database mapping compatible products to one another

#9. Subjective Search“High-quality kettles”

Searching for products using non-objective qualifiers

Handling of quantifiable single-attribute degrees (e.g. “cheap”), quantifiable but multi-attribute mix (“value for money”), and taste-based (“delicious”) qualifiers

Query structure (how the query is constructed)

50

Query Type User behavior How you can support it#10. Slang, Abbreviation, and Symbol Search“Sleeping bag -10 deg.”

Searching for products using various linguistic shortcuts

Synonym mapping of slangs, abbreviations, and symbols, as well as interpretation of symbol intent (ranges, modifiers, etc)

#11. Implicit Search“[Women’s] Pants”

Forgetting to include certain qualifiers in the search query due to one’s current frame of mind

All available environmental variables must be used to infer any implicit aspects of the user’s query

#12. Natural Language Search“Women’s shoes that are red and available in size 7.5”

Searching in full sentences rather than bundles of keywords

Intelligent parsing and deconstruction of the user’s query

Other e-commerce internal search solutions

51

Always think of query types when setting up internal search.

Don’t rely on static layouts shared for category products and search results.

Adapt filtering and sorting to adapt to the user’s query and context.

Faceted search is the foundation of a contextual filtering experience and less resource intensive than query support.

Conclusion

52