amazon cloudsearch meetup august 15, 2012
Post on 25-Feb-2016
40 Views
Preview:
DESCRIPTION
TRANSCRIPT
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Amazon CloudSearchMeetup
August 15, 2012
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Welcome
Housekeeping
Slides will be posted
Drawing
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
AgendaIntroduction to CloudSearch• Jon Handler, CloudSearch Solutions Architect
Relevance and Ranking• Jack Conradson, Software Engineer
Case Study: Reddit• Keith Mitchell, Programmer
Q&A
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Introduction to CloudSearch
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Introduction to Search
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Inverted Index
US President
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Search On The WebRelevance/RankingFacetingRange SearchingFielded SearchingBoolean QueriesComplex Relevance
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Search On The WebRelevance/RankingFacetingRange SearchingFielded SearchingBoolean QueriesComplex relevance
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Search On The WebRelevance/RankingFacetingRange SearchingFielded SearchingBoolean QueriesComplex relevance
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Search On The WebRelevance/RankingFacetingRange SearchingFielded SearchingBoolean QueriesComplex relevance
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Search On The WebRelevance/RankingFacetingRange SearchingFielded SearchingBoolean QueriesComplex relevance
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Search On The WebRelevance/RankingFacetingRange SearchingFielded SearchingBoolean QueriesComplex relevance
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Search On The WebRelevance/RankingFacetingRange-SearchingFielded SearchingBoolean QueriesComplex Relevance
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Amazon CloudSearch
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Amazon CloudSearch
Fully-managed, full-featured search service Automatically scales for data & trafficHandles both structured and unstructured dataNear real-time indexingUp and running in less than 1 hour
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
SEARCH CLIENTwww.example.com
SEARCH DEVELOPER
Search API Console
SEARCH ENDPOINT DOCUMENT SERVICE ENDPOINT CONFIGURATION SERVICE ENDPOINT
ConfigurationAPI
CommandLine Tools
ConsoleDocumentService API
CommandLine Tools
Console
SEARCH SERVICESearch Documents
DOCUMENT SERVICEAdd Documents
Update Documents
Delete Documents
Create Domains
Configure Domains
Delete Domains
CONFIGURATION SERVICE
SendSearchRequests
SearchResults
Use the Search Tester
SendDocuments
Create andManage Domains
ACCESS CONTROL ACCESS CONTROL ACCESS CONTROL
Amazon CloudSearch Architecture
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Automatic Scaling: Data & Traffic
SEARCH INSTANCEIndex Partition n
Copy 1
SEARCH INSTANCEIndex Partition 2
Copy 2
SEARCH INSTANCEIndex Partition n
Copy 2
SEARCH INSTANCEIndex Partition 2
Copy n
SEARCH INSTANCE
DATADocument Quantity and Size
TRAFFICSearch Request Volume and Complexity
Index Partition nCopy n
SEARCH INSTANCE
SEARCH INSTANCEIndex Partition 1
Copy 1
SEARCH INSTANCEIndex Partition 2
Copy 1
SEARCH INSTANCEIndex Partition 1
Copy 2
SEARCH INSTANCEIndex Partition 1
Copy n
Index Partition 1Copy 1
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Example: Build Your Playlist
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Use Case
Million song dataset http://labrosa.ee.columbia.edu/millionsong/
Search documents are songs• Attributes: title, artist names, years, genre, artist familiarity
We’ll use this to create a “Build Your Playlist” web application.
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Demo
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
SDF Documents[
{"type":"add", "id": "sombzze12a8c134960",
"version":5, "lang":"en", "fields":
{"title":"Cajun Twisters", "artist_name":"Adam Ant", "year":"1993", "song_id":"sombzze12a8c134960", "artist_familiarity":449425, "genre":["alternative", "electronic", "instrumental", "rock"] }
}, … ]
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Configuration
cs-configure-from-sdf• Analyzes source files for fields and types. Heuristic
Individually
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Upload Documents
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
PHP Integration$results =
file_get_contents(http://search-mn-songs-5bbplyghbb5tk257rsb7iamlsy." ."us-east-1.cloudsearch.amazonaws.com" . "/2011-02-01/search?q=" . $keyword . $bqParam . "&return-fields=title,artist_name,year,genre_result,artist_familiarity&"."facet=year_facet,genre&" . "facet-year_facet-sort=alpha&" ."facet-genre-sort=alpha&" ."facet-genre-top-n=100000&" . "facet-year_facet-top-n=100000&" ."t-year=1985..&" ."t-title=a..&" ."rank=-" . $rank);$resultsObj = json_decode($results);
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Common Feature Requests
Field Weighted RelevanceAdditional Regions and LanguagesHigh AvailabilityTighter integration with other AWS services (Dynamo/S3)Support For Very Large Use CasesGeo Sorting
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Field-Weighted Values
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Field Weights Use Case
Music Search• Dataset composed of the following fields:
• Title• Album• Artist• Lyrics• Popularity
Results without field weights• May end up with results based heavily on lyrics when searching for an artist’s
name (Guns & Roses vs. roses, guns)Results with field weights• Possibly apply a greater weight to artist than lyrics
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
FWV in Rank Expressions
Rank expressions can be used within CloudSearch to customize relevance computations for better returned search results.• song_relevance = text_relevance + popularity
Natural to extend rank expressions to allow field-weighted values using JSON objects.• song_relevance = cs.text_relevance({weights: {artist=3.0,
song=4.0}, default_weight=0.5} + 0.5*popularity
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Query-Time Rank Expressions
Each set of defined rank expressions may take a while to be deployed to your search domain.Query-time rank expressions would allow rank expressions to be defined during a query without having to wait• q=‘guns roses’&rank-qtre=cs.text_relevance({weights:
{artist=3.0, song=4.0}, default_weight=0.5}&return-fields=qtre&rank=-qtre
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
ResourcesAmazon CloudSearch Overview Pagehttp://aws.amazon.com/cloudsearch/• FAQs• Community Forum• Documentation & Getting Started Tutorial (IMDb)
Demos and Tutorials• What Is Amazon CloudSearch • Introducing Amazon CloudSearch (Features)• Building a Search Application Using Amazon CloudSearch• Getting Started Tutorial
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Upcoming Events
Enterprise Search Summit/KMworld, DC, Oct. 17-19Bay Area Amazon CloudSearch Group: Oct. 24
Las Vegas, November 27-29
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Q&A
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Thank You
top related