behind the scenes of multi-cloud solr-as-a-service
TRANSCRIPT
O C T O B E R 1 1 - 1 4 , 2 0 1 6 • B O S T O N , M A
Orchestration, Management & Analytics platform for Apache Solr in the Cloud
Sameer Maggon Founder, Measured Search, Inc.
3
01About Me
• Sameer Maggon
• Founder of Measured Search, Inc.
• Been working with Lucene since 2001
• Work and Live in Los Angeles
• Engineering Alumni, USC
Helped numerous Fortune 500 companies and startups build search based teams, platforms and applications.
4
01About Measured Search
• Provides Search Infrastructure Management and Expert Solutions within Public and Private Clouds.
• Headquartered in California (Manhattan Beach).
• Global team that provides follow the sun SLA-backed support and services.
• Customers include public and private companies in Financial Services, Life sciences, Knowledge Management, Publishing, Supply Chain, eCommerce and Technology SaaS across 3 continents.
Solr-as-a-Service | Managed Services | Professional Services
5
01About this talk
• Context & Challenge
• Overview of SearchStax
• Under the hood - Orchestration & Management
• Under the hood - Monitoring & Alerting
• Under the hood - Search Analytics
• Questions
6
01Move to cloud is happening and inevitable
Data Centers / Physical Machines
Cloud Infrastructure
7
01Data Growth and Virtualization is a reality
Few servers tens and hundreds of servers
8
01
Configuration, Management and Scaling of Solr infrastructure becomes a nontrivial
and time consuming job.
9
01
SearchStax® (AS A SERVICE)
SearchStax® Solr Cloud Manager
SearchStax® Pulse
SearchStax® Analytics
Comprehensive Solr Monitoring & Alerting with service level reporting to proactively manage your clusters.
Realtime feedback & user insights to help optimize your Search Experience
Easiest way to run & manage Solr in the cloud - saves time, money and reduces risk.
10
01Conceptual Architecture - OrchestrationCustomer A
Customer C Customer C
Measured Search Account
Measured Search Account
Measured Search Account
Customer A
Customer C Customer C
Customer A
Customer C Customer C
Orchestration Engine
Monitoring Engine Analytics Engine
• Customer’s choice of deploying on any cloud provider
• Customer’s choice of deploying on any available region across the world.
• Customer’s choice of whether to use Measured Search cloud relationship or Customer’s existing cloud vendor relationship.
11
01Individual tenants and deployments
Load Balancers
• All customer’s deployments within one VPCs
• Each deployment within it’s own subnet for isolation
• Each customer get their own dedicated clusters (no sharing of virtual nodes)
• Load Balancers provide High Availability and Failover
• Point and Click ability to add/delete/change nodes within deployments.
• Pre-baked SSLs or bring your own certs
• Native Cloud based support for Encryption
Solr/Zk
VPCSubnet A
Subnet B
12
01Deployment Cloning
ProductionStaging
Bug TriageProduction
Region YRegion X
Within a region or across regions
13
01Monitoring Architecture (SearchStax Pulse)
Trac
king
API
SearchStax Agent
• Agents on each node sending data periodically to API
• Zabbix and SearchStax Pulse is used internally by the Operations team
• Data retention and intervals are configurable per customer and deployments
MongoDB Clusters
Alerting Service
Zabbix
Disk Space Notifications Resource Utilization Notifications Heartbeat Threshold based Notifications Dashboard allows configurability of alerts
Dashboard with Graphs and
Notification Mgmt
SearchStax Agent SearchStax
Agent
14
01Analytics Architecture (SearchStax Analytics)
Colle
ctor
(API
)
MongoDB Clusters
Aggregation Services / EMR Pipeline
Analytics Dashboard
Web App
Web App
Mobile (Native)
S3
15
01
Enable our Customers to:
Focus on their application as opposed to infrastructure
Peace of mind by SLA backed guarantees
Be in control and have visibility into their setup
Standardize and Automate the entire Solr usage within organization
