jae kim, logdna - sales engineerevangelist. he comes from zerto and dellemc where he consulted on...
Post on 09-Jul-2020
0 Views
Preview:
TRANSCRIPT
1
DeveloperWeek 2020
Jae Kim, LogDNA - Sales Engineer
2
DeveloperWeek 2020
Loglines: Create Collect Send Parse Store View More!
Logging - start to finish!
3
DeveloperWeek 2020
- Application type - Monolithic
- Logs written to disk, inline - Limited primary insight into application state
- Distributed / microservice - Event-based
- Plain language vs Structured logs - Levels (debug > info > error > FATAL) - Types (system, application, audit, security, …)
Create your logline
4
DeveloperWeek 2020
- syslog - Compare to SNMP
- OS level (Linux, Win) - /var/log - %windir%/system32/...
- Platform level (Docker, K8S, etc) - Logspout - STDOUT and STDERR
- Environment level (AWS Cloudwatch, Azure Event Hub, etc) - Cloudwatch capture from: EC2, DynamoDB, S3, ECS, EKS…
Collect your logline
5
DeveloperWeek 2020
- Local logs - printf, et al
- Stream to a repository! - Local or remote - Depends on data
type / source - Efficiencies on
sending side
Aggregate / stream your logline
6
DeveloperWeek 2020
- Messaging - Message-queue vs Pub-Sub - Synchronous vs async
- Use both!
- Parsing / searching - grok / REGEX - Parse to create field:value objects
- Alerting - Archiving - ...
Parse / route / process your logline
7
DeveloperWeek 2020
Parse / route / process
8
DeveloperWeek 2020
- Log volume? - MB of data > write to file(s) - GB of data > write to DB - TB of data > write to NoSQL
- <-----> scalability - very fast search
- Other considerations - Distribution > sharding / replicas
Store your logline
9
DeveloperWeek 2020
- Search - grok / REGEX - Full-text search
- Visualize - Graphing - Dashboards
View your logline
10
DeveloperWeek 2020
- Data analytics - Advanced visualization
- Tableau, et al
- Next chapter of DB technology
More!
11
DeveloperWeek 2020
- https://charity.wtf/2019/02/05/logs-vs-structured-events/ - https://logdna.com/bring-structure-to-your-logs-with-custom-parsing-on-logdna/ - https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html
Resources
12
DeveloperWeek 2020
Q&A
13
DeveloperWeek 2020
Find us at Booth 208!
14
DeveloperWeek 2020
thank you!
15
DeveloperWeek 2020
Systems Engineer, LogDNA Jae wears several hats at LogDNA, including Sales/Solutions Engineer and Product Evangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions.
Log aggregation is no longer cutting-edge, rather a needed part of the infrastructure toolkit, as well as a prudent business decision. How were logs leading to actionable conclusions in the past and how do we do it today? From log collector to aggregator to ingester, from parsing to indexing and storage, what are the top pain points and how can we do it [better]? And what's next...?
Jae Kim On the Nature of Logging
Sponsored Session
16
DeveloperWeek 2020
Booth MockUp
Booth #208
top related