stupid buildout tricks

Click here to load reader

Post on 17-Dec-2014

2.729 views

Category:

Technology

0 download

Embed Size (px)

DESCRIPTION

 

TRANSCRIPT

  • 1. Stupid Buildout Tricks Ricardo NewberyWednesday, November 9, 2011
  • 2. First, a basic buildout [buildout] parts = pyramid myapp [pyramid] recipe = zc.recipe.egg dependent-scripts = true eggs = pyramid interpreter = py [myapp] recipe = collective.recipe.template input = templates/myapp.ini.in output = ${buildout:directory}/etc/myapp.ini key_a = value_a key_b = value_bWednesday, November 9, 2011
  • 3. The problem Multiple environments (dev, qa, staging, production) requiring different buildout congurations. $ bin/buildout -c dev.cfg $ bin/buildout -c qa1.cfg $ bin/buildout -c qa2.cfg ad nauseum ... The non-dev builds are performed by release engineers who have better things to do than to read complicated release notes to gure out the right buildout incantation for the current environment.Wednesday, November 9, 2011
  • 4. The solution Add environment variables into each non-dev environment and have buildout gure out which cong is the right one.Wednesday, November 9, 2011
  • 5. First try... Used a buildout extension to munge the buildout cong dynamically. Similar to the way collective.recipe.python works. Two issues: 1) The variant congs are hard to read. 2) Doesnt play well with mr.developer since "bin/develop" turns off extensions when reading the cong. This was painful so I redid this w/o the buildout extensions...Wednesday, November 9, 2011
  • 6. Nothing fancy... $ bin/buildout -c config/variant-staging.cfg buildout:directory= The "buildout:directory" value xes the problem where buildout assumes the build context is the directory where the main cong resides. Now, I just need a script to generate this call to buildout... $ bin/environ Lets make this script...Wednesday, November 9, 2011
  • 7. import os import sys import logging import subprocess logger = logging.getLogger(buildout.variants) logger.setLevel(logging.INFO) handler = logging.StreamHandler() handler.setFormatter(logging.Formatter(%(message)s)) logger.propagate = False logger.addHandler(handler) log = logger.info # Is config file specified on command line? config_file = False args_in = sys.argv[1:] while args_in: op = args_in.pop(0) if op[:2] == -c: config_file = True log("buildout.variants: Disabled.") # If config file not specified, check for one matching the environment args = [bin/buildout] if not config_file: environ = os.environ.get(BUILDOUT_ENVIRON) if environ: log("buildout.variants: Detected environment %s" %environ) config_file = config/variant-%s.cfg %environ if not os.path.exists(config_file): log("buildout.variants: Not found %s" %config_file) config_file = False if config_file: args.append(-c%s %config_file) args.append(buildout:directory=%s %os.getcwd()) log("buildout.variants: Applying variant %s" %config_file) else: log("buildout.variants: No variants found.") # Run buildout args = args + sys.argv[1:] subprocess.call(args)Wednesday, November 9, 2011
  • 8. # Is config file specified on command line? config_file = False args_in = sys.argv[1:] while args_in: op = args_in.pop(0) if op[:2] == -c: config_file = True log("buildout.variants: Disabled.")Wednesday, November 9, 2011
  • 9. # Check the environment args = [bin/buildout] if not config_file: environ = os.environ.get(BUILDOUT_ENVIRON) if environ: log("buildout.variants: Detected %s" %environ) config_file = config/variant-%s.cfg %environ if not os.path.exists(config_file): log("buildout.variants: Not found %s" %config_file) config_file = False if config_file: args.append(-c%s %config_file) args.append(buildout:directory=%s %os.getcwd()) log("buildout.variants: Applying variant %s" %config_file) else: log("buildout.variants: No variants found.")Wednesday, November 9, 2011
  • 10. # Run buildout script args = args + sys.argv[1:] subprocess.call(args)Wednesday, November 9, 2011
  • 11. release engineers = :-) Only need to know one command to update a buildout $ bin/environ Environment buildouts can always be overridden. $ bin/environ -c local.cfg or $ BUILDOUT_ENVIRON=staging bin/environWednesday, November 9, 2011
  • 12. Extra credit All the documentation says to run buildout with... $ bin/buildout Lets not confuse the release engineers with yet another way to run a buildout so I just created a custom buildout script (generated by the bootstrap.py) that folds this environment hook into bin/buildout itself. [customizing buildout script left as an exercise for the reader] So no more bin/environ. Just bin/buildout again. Release engineers even happier :-)Wednesday, November 9, 2011
  • 13. An alternative Another way to get environment-aware buildout congurations is with mr.scripty, http://pypi.python.org/pypi/mr.scripty Allows dening values in a buildout part with regular python expressions. This is possibly a more general solution although the congs may be a little harder to read. You also cant adjust a buildout cong "extends" value with this method.Wednesday, November 9, 2011
  • 14. Another problem Staging environment sucks. Occasionally, stuff in python site packages in staging borks the