ubiservices devops journey - adaptavist marketing...devops journey june 2019 –adaptavist 1 2 •...
TRANSCRIPT
UbiServicesDevOps Journey
June 2019 – Adaptavist
1
2
• Since University (2002), always worked on Online problems
• Manage technical people since 2005• Always been a technical manager• Joined Ubisoft in 2008 as Team Lead• Progressed as Project Lead in 2013• Now Product Director
Alain Cormier
2
3
• Founded in 1986• >15 000 employees• World Wide presence• > 40 studios around the world
• 3 studios in Quebec• Publisher of popular brands
3
4
Game Productions
• Very large teams (1000+)• Various expertise (Historian, Artists, Comedians,
Sounds specialists, Game designers, UX specialists, Programmers, etc)
• 3+ years of development for AAA games• Big bang release!
4
5
Harbour
• Provide Online Solutions to Game Productions• Multiplayers• Global Online Services• Development pipelines• Club (loyalty platform)
• ~300 employees• Global presence
• Montreal (200+)• Dusseldorf (20+)• Bucharest & Craiova (30+)• Shanghai (20+)
5
UbiServices storyPART | 2
Today and futurePART | 3
Take awaysPART | 4
QuestionsPART | 5
Devops – Key conceptsPART | 1
6
Devops – Key conceptsPART | 1
7
Common problem
8
1. Low-stress
2. Feedback Loop
3. Decouple Deploy
From Release
4. Small Batch Size
5. Autonomous Teams
6. Reduce Lead
Time
9
UbiServices StoryPART | 1
10
11
(2010) - Once upon a time…• Use custom techno (Rendez-Vous) to build online
services
• Each projects had its “branch”
• Services were forked and customized for
each productions
• Lot of time spent to retests, do load tests and to put in
place infrastructure for development and launch…
Repeated for all projects.
• Still in use today for Productions Custom Services
• Priority was isolation between projects
11
12
(2012) - Small group want to work differently
• Use standard technos
• Do it once for everyone (global services)• Path base loadbalancing
• Microservices architecture• Let’s start with Critical Service: authentication!
12
13
(2013) – Some challenges before first launch
• API – Referential & Versioning
• Autonomy• Collaboration with IT team• Monitoring
13
(2013) - API management: turn on the light!
14
15
(2014) – Grow
• Double project size• More Teams
• autonomous… but interdependent• From Load Balancer to API Gateway• Capacity planning
15
16
(2015) - Lets innovate
• Until now, C#/MSSQL shop• Dev teams want to explore new technos
• Start with one Critical Service: Players Notifications
• Stretch IT current knowledge• Do Kanban
• Start with one Critical Service: API Gateway
16
17
(2016) – The cloud!• IT prefer to keep us on premise.
• We insist. Need scalability and easier provisioning.
• New dedicated “squad team” from IT to support us
• Start with one Critical Service: Authentication!
• But…• Not as automated as we would like. Pretty much
done the same way as on premise.
• Heavy on operation.
• People get called regularly for dumb reasons (ex: log rotation).
• Which resulted
• Dev team take full ownership of ops. 17
18
(2017) – Autonomy
• All in to the cloud and automate!
• Improve deployment process (spawn new stack)
• Tools run scripts. Now everyone can spawn a
new stack of a version of the service with a few clicks.
• Closer to immutability. Less in-place changes.
• Development team take ownership of spawning
their servers stack. They write less features,
but feel more autonomous.
• Sys admin team now focus on transversal infrastructure (log stack, jump servers, etc).
18
19
(2018) – Do more with less
• Hiring is a challenge.• Time to market is top priority.
• Prefer managed solutions.
• Let dev team focus on business value. • But keep them owner and responsible.
• Improve dev team agility: containers• Start with one Critical Service: account!
19
Today and FuturePART | 2
20
21
(Today) – The Ubisoft platform
• Working on a “Platform” for all Ubisoft
• New CiCd pipeline for microservices
• Enforce referential• Enforce immutability (no access to machines)• Helps with automated tests• Give API management, logs, metrics and alerts
for free to all projects.
• Containers• Help isolation• Contract is clear between devteam and infra
• Infrastructure as code (for real!)
21
22
(Future) – Challenges
• Onboard Productions into devops and
microservices.
• Review custom service development pipeline.
• Automatic deployment up to Live
environment.
• Move to Inner Sourcing mindset and accept
contribution from any internal teams.
22
Take awaysPART | 4
23
24
Recipe for changes
• Start with small team (strike team) of
highly motivated guys.• Make sure you get support from management.
• Choose critical project/feature.
• Demonstrate the improvement.• Propagate to all teams.• Rince and repeat.
24
25
Sys admin / DBA
• Still need them.• Although, no more authoritative on tech choice.
• More in an expert role and consulted when
needed by devteam.
• Have their own “services” (ex: log stack)
25
26
Teams
• Autonomous but interdependent
• Small: 5-10 people.
• Ramp-up new managers• Split teams (organic growth)
26
27
Devops & Microservices
• Tight together• Microservices mindset:
• Loosely couple components
• Highly cohesive components
• Independently Develop and Deploy• Each component has a contract and must
respect if (need tests!)
27
Thank you!
28