full text search using azure search. shankar subramanyam senior consultant...
TRANSCRIPT
Full Text Search using Azure Search
Shankar SubramanyamSenior Consultant
[email protected] | @Shankarblr
Enthusiast:• Web/Cloud Technologies• Financials• Economics
About me
Agenda
• Overview of Full text search using Azure Search• Demo – Build Movies Catalog using Azure Search• Q & A
What is full text search?
• In text retrieval, full text search refers to techniques for searching a single computer-stored document or a collection in a full text database. Full-text search is distinguished from searches based on metadata or on parts of the original texts represented in databases (such as titles, abstracts, selected sections, or bibliographical references).
• Reference - http://en.wikipedia.org/wiki/Full_text_search
Full text search platform
• Lucene • Solr• ElasticSearch• Azure Search• FSIS• Database Full text search • etc.
Key words in search
• Indexes• Documents• Fields
• Types of searchability• Retrievable • Non-retrievable
• Tokenization - Analyzer • Facets• Scoring
What is Azure Search ?
• Azure Search is a PaaS service • ElasticSearch as a managed service• High performance• Horizontally scalable• Administration and querying
• Via REST API• Via C#, using Azure Search Client Library (NuGet: AzureSearchClient)
• Secured using API keys• Query keys (multiple)• Management keys (two
Demo – Create Search Service in Azure portal
1. Login to Azure Portal2. Click New 3. Select Data + Storage4. Select Search as shown
1. Fill all the information 2. For Pricing Tire - Make
sure to select Free3. Click Create to create
Azure Search Service
Service Name and API Key
Azure Search Architecture
Index(schema type W)
Field
Service (a.k.a. resource)
Index(schema type X)
Index(schema type Y)
Index(schema type Z)
Document(type X)
Document(type X)
Document(type X)
Document(type X)
FieldField
Field
{json}
… …
… …
Inverted index
aardvark
hood
red
little
riding
robin
women
zoo
Little Red Riding Hood
Robin Hood
Little Women
0 1
0 2
0
0
2
1
0
1
2
{(2, 1)}
0 1
{(2, 0)}
{(0, 0)}
0
Terms/ Words Documents
A z u re S e a rc h is s tru c tu re d
• A search index has a predefined structure• It is not dynamic• Each document can have below operations
• Search• Suggestion • Lookup • Count
• Each field in the index has characteristics defined when created
• Filterable?• Searchable?• Faceted?• Retrievable?• Sortable?
Field Characteristics: Key
• Required!• Can only be on one field for the document• Can be used to look up a document directly
• Update• Delete
Field Characteristics: Searchable
• Makes the field full-text-search-able• Allowed data types are string and collection • Breaks the words of the field for indexing purposes
• “Big Red Jeep” will become separate components
• A search for “big”, “red”, “jeep”, or “big jeep” will hit this record
• Searchable fields cause bloat!• Only make it searchable if it needs to be
Field Characteristics: Filterable
• Doesn’t under go word breaking• Exact matches only• Only searches for “big red jeep” will hit a “big red jeep”
record
Field Characteristics: Sortable
• By default, results are sorted by score
Field Characteristics: Facetable
• Data the data type except Geography points are facetable
• Used to rank records by other notions• Jeeps that sold by this {dealer}• Jeeps that are this {color}
Field Characteristics: Suggestions
• Used for auto-complete• Only for string or collection of string• False by default• Causes bloat in the index!
Field Characteristics: Retrievable
• Allows the field to be returned in the search results• Key fields must be retrievable
Data Type / Properties Matrix
Demo : Movies Catalog Create Index
Query parameters Request - GET /indexes/[index name]/docs?[query parameters]
Parameter – • search =[string] $orderby=[string]• searchMode=any|all facet=[string] • searchFields=[string] api-version=[string] • $skip=# scoringProfile=[string]• $top=# highlight=[string] • $count=true|false scoringParameter=[string] • $select=[string] highlightPreTag=[string]• $filter=[string] highlightPostTag=[string]
Example : https://rocks.search.windows.net/indexes/movies/docs?search=*&$count=true&$orderby=Year desc&api-version=2015-02-28&facet=Year
Demo : Movies Catalog Consume Index
Q & A
References
• Azure Search REST API• http://msdn.microsoft.com/en-us/library/azure/dn798935.aspx• http://azure.microsoft.com/en-in/documentation/articles/searc
h-api-2014-10-20-preview/
• Azure Search Client Library Getting Started• https://code.msdn.microsoft.com/Getting-Started-with-Azure-5
0b624b7
• Inverted indexhttp://en.wikipedia.org/wiki/Inverted_index