building a data platform tnt
TRANSCRIPT
![Page 1: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/1.jpg)
8 learnings from building a Data Platform
DIGITAL INTERNATIONAL
![Page 2: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/2.jpg)
Goal
Of this talk
![Page 3: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/3.jpg)
About
The org - me
![Page 4: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/4.jpg)
Why
Did we embark on this anyway?
![Page 5: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/5.jpg)
First, take a step back
Vision
Mission
Objectives
Key Results
![Page 6: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/6.jpg)
We have a dream: Vision
It's our vision to be the engine of customer obsessed
intelligence, continuously driving agile fact based decision making
![Page 7: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/7.jpg)
Tangible: Objectives
We have a flat file data lake updated in daily batch with at least 1 data source
![Page 8: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/8.jpg)
Tangible: Key results
Tables ABC are from source X offloaded in landing zone
Staging, transformation and harmonisation are automated
1 2
![Page 9: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/9.jpg)
Learning #1
Laser focus
![Page 10: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/10.jpg)
Learn by trying
![Page 11: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/11.jpg)
Learn by testing
Set criteria BEFORE
Maintainability Scalability Reliability Security Cost(ability?)
etc...
![Page 12: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/12.jpg)
Learning #2
Do POCs
![Page 13: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/13.jpg)
Are you ready to roll?
Right skills Budget Data access Infra freedom Sponsor
![Page 14: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/14.jpg)
Learning #3
Remove dependencies
![Page 15: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/15.jpg)
Hindsight engineering = Pain
GDPR
![Page 16: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/16.jpg)
Learning #4
Actively involve privsec
![Page 17: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/17.jpg)
Design Principles
● PaaS● Automate all things● Separate compute from storage● Event driven● Loosely coupled
![Page 18: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/18.jpg)
Learning #5
Acceptance criteria
![Page 19: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/19.jpg)
With the business for the business
![Page 20: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/20.jpg)
Learning #6
Include your customer
![Page 21: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/21.jpg)
Don’t make it a big project
![Page 22: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/22.jpg)
Don’t make it perfect
Your goal
![Page 23: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/23.jpg)
Learning #7
Be Agile - adopt
![Page 24: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/24.jpg)
Data is an intangible asset - make it real
![Page 25: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/25.jpg)
![Page 26: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/26.jpg)
Learning #8
Share!
![Page 27: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/27.jpg)
![Page 28: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/28.jpg)
![Page 29: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/29.jpg)
Learnings
- Learning 1: laser focus
- Learning 2: do POCs
- Learning 3: be independent
- Learning 4: involve privsec
- Learning 5: design principles
- Learning 6: include your customer
- Learning 7: be Agile
- Learning 8: share!
![Page 30: Building a data platform tnt](https://reader035.vdocument.in/reader035/viewer/2022062503/5a6493477f8b9a27568b6bf5/html5/thumbnails/30.jpg)
Thank you