what is the problem? - wordpress.com · twitter @derek_stadnicki { } what is the problem? both the...
TRANSCRIPT
DEVOPS IN THE
ENTERPRISE
Introducing DevOps practices in the
enterprise with ElasticSearch Logstash
and Kibana
SHANE RAINVILLEEmail [email protected]
Web http://www.serverlab.ca
Twitter @ShaneRainville
DEREK STADNICKIEmail [email protected]
Web http://storageblocks.wordpress.com
Twitter @derek_stadnicki
{ }
What is the Problem?
Both the Development team and Operations team gets
called for the same problems
Developers have insight into the application but not
overall server performance
Operations can see server performance but does not
have insight into the application
No way to correlate application events to performance
events
Development and Operations don’t share the same
information
Development vs Operations Tool Sets
Development Tools Operations Tools
• Application Logs • Vmware Performace Tools
• OS Logs • Network Perfromance Tools – Solarwinds, MRTG
• Linux Performance tools like ps, top, etc • OS Logs
• Linux Performance Tools like ps, top, etc
Requirements and Goals
The same information made available to both teams
Information made available in management friendly
form
The ability to correlate application events with
performance events
Ability to scale the solution to the rest of the enterprise
as needed
Ease of deployment and management
LogstashLog aggregation
and parsing
ElasticsearchSearch and Analytics
KibanaVisualization of
logged data
ELK Stack
Okay. Now What?
The Architect’s Dilemma
Why Redis
Logstash Forwarder
CentOS 5
Scalability
Gathering the Logs
• Pick target servers
• Install log shipper
• Decide on what logs to capture
Log Ingestion
• All logs get sent via the log shipper to the Logstash server
• Logs get ingested into ElasticSearch
• Filters are using Grok to parse unstructured logs
Apache Error Log Dashboard
Root Logon Dashboard
Discovery
Day one found issues that none of the teams were
aware of
Development and operations teams were able to sit
down and discuss the application using the same info
Both teams are now collaborating on a new architecture
for the application
Logstash is now being used by both teams together to
test assumptions about new applications during the
development process
DEVOPS IN THE
ENTERPRISE Q&A
SHANE RAINVILLEEmail [email protected]
Web http://www.serverlab.ca
Twitter @ShaneRainville
DEREK STADNICKIEmail [email protected]
Web storageblocks.wordpress.com
Twitter @derek_stadnicki
{ }