jae kim, logdna - sales engineerevangelist. he comes from zerto and dellemc where he consulted on...
TRANSCRIPT
![Page 1: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/1.jpg)
1
DeveloperWeek 2020
Jae Kim, LogDNA - Sales Engineer
![Page 2: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/2.jpg)
2
DeveloperWeek 2020
Loglines: Create Collect Send Parse Store View More!
Logging - start to finish!
![Page 3: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/3.jpg)
3
DeveloperWeek 2020
- Application type - Monolithic
- Logs written to disk, inline - Limited primary insight into application state
- Distributed / microservice - Event-based
- Plain language vs Structured logs - Levels (debug > info > error > FATAL) - Types (system, application, audit, security, …)
Create your logline
![Page 4: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/4.jpg)
4
DeveloperWeek 2020
- syslog - Compare to SNMP
- OS level (Linux, Win) - /var/log - %windir%/system32/...
- Platform level (Docker, K8S, etc) - Logspout - STDOUT and STDERR
- Environment level (AWS Cloudwatch, Azure Event Hub, etc) - Cloudwatch capture from: EC2, DynamoDB, S3, ECS, EKS…
Collect your logline
![Page 5: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/5.jpg)
5
DeveloperWeek 2020
- Local logs - printf, et al
- Stream to a repository! - Local or remote - Depends on data
type / source - Efficiencies on
sending side
Aggregate / stream your logline
![Page 6: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/6.jpg)
6
DeveloperWeek 2020
- Messaging - Message-queue vs Pub-Sub - Synchronous vs async
- Use both!
- Parsing / searching - grok / REGEX - Parse to create field:value objects
- Alerting - Archiving - ...
Parse / route / process your logline
![Page 7: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/7.jpg)
7
DeveloperWeek 2020
Parse / route / process
![Page 8: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/8.jpg)
8
DeveloperWeek 2020
- Log volume? - MB of data > write to file(s) - GB of data > write to DB - TB of data > write to NoSQL
- <-----> scalability - very fast search
- Other considerations - Distribution > sharding / replicas
Store your logline
![Page 9: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/9.jpg)
9
DeveloperWeek 2020
- Search - grok / REGEX - Full-text search
- Visualize - Graphing - Dashboards
View your logline
![Page 10: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/10.jpg)
10
DeveloperWeek 2020
- Data analytics - Advanced visualization
- Tableau, et al
- Next chapter of DB technology
More!
![Page 11: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/11.jpg)
11
DeveloperWeek 2020
- https://charity.wtf/2019/02/05/logs-vs-structured-events/ - https://logdna.com/bring-structure-to-your-logs-with-custom-parsing-on-logdna/ - https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html
Resources
![Page 12: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/12.jpg)
12
DeveloperWeek 2020
Q&A
![Page 13: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/13.jpg)
13
DeveloperWeek 2020
Find us at Booth 208!
![Page 14: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/14.jpg)
14
DeveloperWeek 2020
thank you!
![Page 15: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/15.jpg)
15
DeveloperWeek 2020
Systems Engineer, LogDNA Jae wears several hats at LogDNA, including Sales/Solutions Engineer and Product Evangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions.
Log aggregation is no longer cutting-edge, rather a needed part of the infrastructure toolkit, as well as a prudent business decision. How were logs leading to actionable conclusions in the past and how do we do it today? From log collector to aggregator to ingester, from parsing to indexing and storage, what are the top pain points and how can we do it [better]? And what's next...?
Jae Kim On the Nature of Logging
Sponsored Session
![Page 16: Jae Kim, LogDNA - Sales EngineerEvangelist. He comes from Zerto and DellEMC where he consulted on virtualization data storage/protection solutions. Log aggregation is no longer cutting-edge,](https://reader033.vdocument.in/reader033/viewer/2022042417/5f32ebd4372fc15b7e475d86/html5/thumbnails/16.jpg)
16
DeveloperWeek 2020
Booth MockUp
Booth #208