hashicorp at holaluz
TRANSCRIPT
7 YEARS AGO…• Vagrant first commit was in 17 Jan 2010
• Holaluz was founded in 12 Nov 2010
• DevOps starts becoming a concept
• A backend developer in a big e-commerce
WHO AM I?• Currently working as CTO at Holaluz
• Ex Wonga, Hailo, SocialPoint, Ulabox, Privalia…
• Developer for many years, been automating things for a while, DevOps before it was trendy!
• https://devops.barcelona conference co-organiser
HOLALUZ WEBAPPS INFRA
Web servers under ELBS
running Apache+ PHP7.1
Dedicatedcron job boxesusing PHP7.1
RDS PostgreSQLinstances
EC2 Elasticsearch
Some S3buckets tostore files
ElasticacheRedis
for messagededuping
SQS queuesbackground
jobs
RDS MySQLinstances
EFS to storeshared files(moving to S3 & ELK)
HOLALUZ BIGDATA INFRA
Python apps & cron tasks using Django and Celery
R runninginstances
RDSPostgreSQL
instances
EC2 Elasticsearch
EC2 MongoDB
Some S3buckets tostore files
ElasticacheRedis
for Celery
HOLALUZ ERP INFRA
RDSSQLServer
Per env S3 buckets and SQS communicating
with Web Infra
License ServersCluster
Windows 2012R2 Servers Cluster running Java8
PROD
INTEGRATION
DEVELOPMENT
OTHER THINGS AROUND
Shared ELK Stack for logs aggregation and inspection
Middleman static sitesgenerator
Grafana for Time Series inspection and alerting
InfluxDB for Time Series metrics storage
E-Invoice Document Management + approval
flows
Cloudwatch Time Series metrics storage and alerting
HASHICORP TOOLS• Confession: I am a massive fan
• Work everywhere, automate anything
• Excellent open-source community
• Well documented, specially BC changes
VAGRANT AT HOLALUZ• Static website generator middleman (Ruby based)
• Local environments for PHP projects
• Local environments for Python Big Data projects
• Coming soon, Windows boxes for people who need to connect to sites needing old IE
CONCEPTS• Builders: Platforms you build images in. It is all
about what you start from!
• Provisioners: Installs and configures
• Post-processors: Optional final steps
WHAT I LIKE• Builds for multiple platforms from a single
source configuration
• VERY Easy to understand
• Works (and can provision) in Win, Mac, Linux
• Easy to share provisioning scripts or use Puppet / Ansible recipes
CAVEATS• Need to be very prescriptive or you end up
with multiple very similar templates
• A bit hard to go with a DRY approach
• Some things are hard to destroy / replace with new images
PACKER AT HOLALUZ• All PHP infrastructure with ASGs using
AMIs we recycle periodically
• Kibana, Logstash, Grafana, InfluxDB
• Python BigData infrastructure as well (R soon)
• ERP first steps on top of official Windows AMIs
CONCEPTS• Provider: Platform we are automating
• Resources: Automatable things in the Provider
• Modules: Reusable set of resources
• State: Used to diff desired state to existing. Can be stored remotely and supports distributed locking
WHAT I LIKE• Can integrate with anything that has an API
• Easy to extend, contribute and really quick to add new features. Excellent Github community
• Existing resources can be imported (PAIN)
• Have used it for 18 months, multiple providers, rarely hit a bug and was always quickly fixed
CAVEATS• Once you go Terraform, STOP using Console
• Some providers don´t have nice update support
• Terraform modules feel a bit hacky
• Sometimes state needs manual edition (getting much better but beware new providers)
TERRAFORM AT HOLALUZ• Most infrastructure rebuilt using Terraform
• VPC / Network for test environments
• Some S3 buckets and SQS configuration
• Most IAM policies
INTERESTING TRICKS (III)
Environments Modules
State in S3 per envLock via DynamoDB
Use env output as variables in another env
BACK IN JANUARY• Deployments were painful
• Unreliable test and local dev environments
• Hard to integrate new things
• Poor logging and monitoring metrics
• Painful error recovery, no real DR strategy
BAD HABITS AND PROBLEMS• Development teams did not talk much
• Little understanding of network and infra
• Lack of technical guidelines
• No time to rethink processes
• Lots of legacy code with no tests at all
HOLALUZ IN SEPTEMBER• One-click deployments almost everywhere
• Development teams talk and collaborate
• Releases speed MASSIVELY improved
• Decent logging and monitoring
• Better error recovery, decent DR strategy
LONG WAY TO GO• Lots of technical debt still to pay
• Fragility in crucial parts of our business
• Still too many bugs, still too little testing
• Incoming changes in Energy (electric cars, smart batteries, prosumers, blockchain…)
NEAR FUTURE• Complete missing Vagrant environments
• Packer generating Docker containers for Dev envs
• Import all existing resources into Terraform and rebuild the production network (PAIN)
• Increase AWS usage (Redshift, DynamoDB…)
• Full automated EVERYTHING
DEVOPS IS…• Devs and Ops working together to deliver value
• Empower teams, reduce hard dependencies
• Communicaton, Integration, Collaboration
• Boosting productivity, make life easier!
• Automation, CI/CD, Infrastructure as code…
BOOKS AND REPOS• The Phoenix Project - Gene Kim, Kevin Behr, George Spafford
• The DevOps Handbook - Gene Kim, Patrick Debois
• Terraform: Up and Running - James Turnbull
• https://github.com/ricardclau/geekshubsbcn (Packer + Terraform)
• https://github.com/kaorimatz/packer-templates (Linux) & https://github.com/joefitzgerald/packer-windows (Windows)
• https://github.com/terraform-community-modules (MANY examples)
QUESTIONS? CONTACT?• Email: [email protected]
• Twitter : @ricardclau
• Github: https://github.com/ricardclau
• If you think these techniques help your company, let´s talk!