webinar: building conversational search with fusion
TRANSCRIPT
Based in San Francisco
Offices in Bangalore, Bangkok, New York City, Raleigh, Munich
Over 300 customers across the Fortune 1000
Fusion, a Solr-powered platform for search-driven apps
Consulting and support for organizations using Solr
Produces the world’s largest open source user conference dedicated to Lucene/Solr
Lucidworks is the primary sponsor of the Apache Solr project
Employs over 40% of the active committers on the Solr project
Contributes over 70% of Solr's open source codebase
40%
70%
• Over 50 connectors to integrate all your data
• Robust parsing framework to seamlessly ingest all your document types
• Point and click Indexing configuration and iterative simulation of results for full control over your ETL process
• Your security model enforced end-to-end from ingest to search across your different datasources
SECURITY BUILT-IN
Shards Shards
Apache Solr
Apache Zookeeper
ZK 1
Leader Election
Load Balancing
Shared Config Management
Worker Worker
Apache SparkCluster
Manager
RE
ST A
PI
Admin UI
Lucidworks View
LOGS FILE WEB DATABASE CLOUD
HD
FS
(Op
tio
nal
)
Core Services
• • •
ETL and Query Pipelines
Recommenders/Signals
NLP
Machine Learning
Alerting and Messaging
Security
Connectors
Scheduling
Fusion Simplifies the Deployment
Core Services
• • •
NLP
Recommenders / Signals
Blob Storage
Pipelines
Scheduling
Alerting / Messaging
Connectors
RE
ST A
PI
Admin UI
Lucidworks View
LOGS FILE WEB DATABASE CLOUD
• Seamless integration of your entire search & analytics platform
• All capabilities exposed through secured API's, so you can use our UI or build your own.
• End-to-end security policies can be applied out of the box to every aspect of your search ecosystem.
• Distributed, fault-tolerant scaling and supervision of your entire search application
Conversational Search: “The principle behind conversational search
is that a user can speak a sentence into a device, and that device can respond with a
full sentence.” -Technopedia
A Pathway to Better Search
•Better Query Parsers
•Similarity
•Collaborative Recommendations
•Personalization
Similarity / More Like This
• Fusion Pipeline Stage takes a document as an argument, returns like documents
• BM25 Similarity
• https://en.wikipedia.org/wiki/Okapi_BM25
Collaborative Recommendations
• Boost results users commonly click on
• Based on “Signals” - built into Fusion
• https://www.youtube.com/watch?v=0AwLSdcrJSc&sns=tw - Demonstrated in last Webinar
Personalization
•Signals on a more per-user basis
•Boost things that I click on often
•Boost results like the ones I’ve clicked on
•More on this next time!
• Easier and more accurate than Natural Language
• Domain phrases PE Ratio, Stock Price, SKU
• Entities: GE, IBM, etc
Domain Specific Language
• https://twitter.com/deepdrumpf
• http://karpathy.github.io/2015/05/21/rnn-effectiveness/
• Speech to Text enabled Search
• Plugged into Fusion Query Pipeline
• https://lucidworks.com/2016/10/13/lucidworks-integrates-ibm-watson-into-fusion-enterprise-discovery-platform/
Watson
RTFMBot
•Based on Chatterbot
•NLTK
•Train on Slack
•Answer Questions generally with Fusion search results