reliably deploying web apps with database migrations
DESCRIPTION
TRANSCRIPT
![Page 1: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/1.jpg)
Reliably Deploying Web Apps with Database
MigrationsPratik Vyas
![Page 2: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/2.jpg)
To deploy new code
with a database migration
that takes 10 seconds to run per site
The Problem
![Page 3: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/3.jpg)
in batches of five sites at a time
Then the 600th site would be running new code with
unpatched database for 20 minutes!
If we have to migrate 600 sites
run migrationfor 600th site
code deployed
danger zone
![Page 4: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/4.jpg)
Deployments need to be handled more carefully
![Page 5: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/5.jpg)
Create a new environment with the new code
Migrate sites one at a time by moving them to the new environment
An Intuitive Solution
![Page 6: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/6.jpg)
build x
build x+1
site c
site d
site e
site b
site a
site f
site g
site h
site i
site j
● move to build x+1● run migrations
maintenance area(No HTTP requests)
site c
The Two Environments
![Page 7: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/7.jpg)
The Solution - Part 1
![Page 8: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/8.jpg)
Building an Environment
Use virtualenv and pip?
pip install would also need to compile C extensions
eg. lxml takes a lot of cpu time and memory
Makes creating new environmentsEXPENSIVE
![Page 9: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/9.jpg)
Wheels are the new standard of python distribution (PEP 427)
Unzip the wheel in site-packages and the package is installed … like magic!
Build wheels on a different machine and watch a new environment get deployed
in a few seconds
Python Wheels
![Page 10: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/10.jpg)
Cost of creating virtualenv is mostly unzipping
No compiler required on production server
PyPI availability is not a factor during deployments
Benefits of Build Step
![Page 11: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/11.jpg)
buildfile.tar.gz├── scripts│ ├── virtualenv.py│ └── virtualenv_support│ ├── pip-1.5.2-py2.py3-none-any.whl│ └── setuptools-2.1-py2.py3-none-any.whl├── sites│ └── statics│ ├── css│ └── *.css│ ├── img│ └── *.png│ └── js│ └── *.js└── wheels └── *.whl
![Page 12: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/12.jpg)
The Solution - Part 2
![Page 13: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/13.jpg)
Running Processes
The new code will require us to start
a new wsgi process + workers
use Supervisor
![Page 14: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/14.jpg)
Supervisor
Use Jinja and generate
process list per virtualenv
Protip: For adding/removing a group of processes, use reread and update instead of reload. It prevents non-related processes
from restarting.
![Page 15: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/15.jpg)
The Solution - Part 3
![Page 16: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/16.jpg)
Serving sites, both old and new
Serve unmigrated sites using old code,
migrated sites using new code.
nginx to the rescue
![Page 17: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/17.jpg)
Configuring nginx awesomeness
Use Jinja to generate configuration and reload nginx
Fallback to old environment based on error condition
For performance, 404s should be cheap
![Page 18: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/18.jpg)
upstream upstream-1 { server 127.0.0.1:8000 fail_timeout=0; }
upstream upstream-2 { server 127.0.0.1:8001 fail_timeout=0; }
server {
. . . .
location / {
error_page 404 = @upstream2handler;
proxy_redirect off;
proxy_intercept_errors on;
proxy_pass http://upstream-1;
}
location @upstream2handler {
proxy_redirect off;
proxy_intercept_errors on;
proxy_pass http://upstream-2;
}
}
![Page 19: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/19.jpg)
Background Workers?
Stop executing scheduled jobs when in handover state
Preferably, use a fresh broker for every build
Track pending jobs per site(we prepend sitename to queue name for this)
![Page 20: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/20.jpg)
The Final Picture
build x
build x+1
site c
site d
site e
site b
site a
site f
site g
site h
site i
site j
● Delay enqueuing scheduled tasks.● Continue if no pending jobs, else
retry after 60 seconds.
● Run migrations.● Move to build x+1.
site c
maintenance area(No HTTP requests)
![Page 21: Reliably Deploying Web Apps with Database Migrations](https://reader031.vdocument.in/reader031/viewer/2022020217/54b6c0584a7959fa048b469c/html5/thumbnails/21.jpg)
ध यवाद@pdvyas, @frappe_io