how i learned to stop worrying & love python packaging
TRANSCRIPT
![Page 2: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/2.jpg)
6How I learned to stop
worrying & love Python Packaging
![Page 3: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/3.jpg)
1) The past of Python packaging2) Common pitfalls and gotchas3) Everyday software development4) The future of Python packaging
6
![Page 4: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/4.jpg)
- Python packageA directory with Python files (module)and a __init__.py file
- ReleaseA version of a specific software, resultof a development cycle, e.g. “Django 1.3”
- DistributionSource or binary form of a release,e.g. zipped tarball or Windows installer
Ambiguous terms
![Page 5: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/5.jpg)
1The past of Python packaging, a brief historical overview
![Page 6: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/6.jpg)
“[..] a standard mechanism for building, distributing, and installing Python modules – or, more realistically, multi-module distributions.”
Distutils history
Greg Ward
- creates distributions
- installs to library directory
- builds C extensions
- processes documentation
![Page 7: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/7.jpg)
- PEP 241 (2001)Metadata in PKG-INFO files added to with package distributions
- PEP 301 (2002)Package Index and Trove classifiers added and register command
- PEP 314 (2003)Metadata 1.1 with license, platform, download URL, dependencies fields
Distutils enhancements
![Page 8: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/8.jpg)
- PEP 345 (2005-today)Metadata 1.2, better dependencies
- PEP 376 (2009-today)Database of Installed Python packages
- PEP 386 (2009-today)Sane version comparison module and better version handling on PyPI
Distutils enhancements
![Page 9: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/9.jpg)
2Common pitfalls and gotchas of packaging
Python software
![Page 10: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/10.jpg)
from distutils.core import setup
setup( name='MyApp', version='0.1.0', description='My app that does things', author='John Doe', author_email='[email protected]', license='BSD', packages=['myapp'], package_data = { 'myapp': [ 'templates/*.html', ], },)
The standard setup.py
![Page 11: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/11.jpg)
Long description
Rendered on PyPI with docutilsreStructuredText renderer
import codecsdef readme(filename): file = codecs.open(filename, 'utf-‐8') return unicode(file.read())
setup( # ... long_description = readme('README.rst'),)
![Page 12: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/12.jpg)
Lists all additional, non-Python files of a release to be installed from a distribution
setup( # ... package_data = { 'myapp': [ 'static/myapp/*/*', 'templates/myapp/*', 'locale/*/LC_MESSAGES/*' ], },)
package_data
![Page 13: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/13.jpg)
Template for the “MANIFEST” file thatlists all files to be put in a distribution
include README.rstrecursive-‐include docs *.txtrecursive-‐include myapp/locale *.mo *.porecursive-‐include myapp/static *.css *.pngrecursive-‐include myapp/templates *.html
MANIFEST.in
![Page 14: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/14.jpg)
- 0.18 “Catty And The Major”
- .000001
- “unreleased.uno!cialdev”
Don’t do that.
- Use PEP 386 for formal versions, e.g.“1.0.1” or “0.5a4” etc
- Use release names in any prose documentation and changelogs,e.g. super hero names
Use sane versions, dammit!
![Page 15: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/15.jpg)
3 Using packaging in everyday’s software
development
![Page 16: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/16.jpg)
- Install binary packages globally using your operatings system’s package management, e.g. database adapters, PIL, lxml
- Vendorize packages that require modifications or short term patches
- Install often updated and your project specific packages locally, e.g. Django, pytz, your own app
System package management or not?
![Page 17: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/17.jpg)
- Isolated Python environments
- Create by topic, e.g. by “branch”, “client”, “milestone”, “staging/prod” etc.
- Use virtualenv-wrapper for even more shell helpers
- pip included in every virtual environment
virtualenv
![Page 18: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/18.jpg)
- Sane installation and uninstallation
- freezing and unfreezing of requirements
$ server1:~ pip freeze > dev-‐deps.txt
$ server2:~ pip install -‐r dev-‐deps.txt
- VCS integration for quick and dirty use
- PyPI mirror support (PEP 381), e.g. d.pypi.python.org
pip
![Page 19: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/19.jpg)
- HTTP server with directory index feature, e.g. Apache’s DirectoryIndex
- PyPI clone Chishop
pip install -‐f http://localhost/dists/
pip install -‐i http://pypi.corp.local/
- Use ~/.pip/pip.conf
[install]
index-‐url = http://pypi.corp.local/
Simple own package index
![Page 20: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/20.jpg)
- Use ad-hoc HTTP server
$ python -‐m SimpleHTTPServer 8000
- .pydistutils.cfg sdist dir
[sdist]
dist-‐dir = /var/www/dists
- Enables pip download-cache
[global]
download-‐cache = ~/.pip-‐downloads
Local index
![Page 21: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/21.jpg)
4The future
of packagingwith Distutils
![Page 22: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/22.jpg)
- toolbox with reference implementation for PEP 345, PEP 376, PEP 386
- the standalone pysetup tool thatinstalls and uninstalls distributions(e.g. “pysetup install Django”)
- metadata available outside of Python file in setup.cfg
- already in Python 3 trunk as “packaging”
What is distutils2?
![Page 23: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/23.jpg)
- defined requirements depending on environment
Requires-‐Dist: bar; python_version == '2.4' or python_version == '2.5'
Requires-‐Python: >=2.5,<3
- Obsoleting other releases
Obsoletes-‐Dist: OtherProject (<3.0)
Other distutils2 features
![Page 24: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/24.jpg)
- “pysetup create” for conversion setup.py to setup.cfg for forward compatibility
- “pysetup generate-setup” for conversion setup.cfg to setup.py for backwards compatibility
Moving from distutils to distutils2
![Page 25: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/25.jpg)
[metadata]name = myappversion = 0.1author = John Doeauthor-‐email = [email protected] = My awesome appdescription-‐file = README.rsthome-‐page = http://johndoe.com/p/myapp/project-‐url: Repository, http://code.johndoe.com/myapp/classifier = Development Status :: 3 -‐ AlphaLicense :: OSI Approved :: MIT License
[files]packages = myappextra_files = setup.py READMEresources = etc/settings.py.dist {confdir}/myapp
Example distutils2 setup.cfg
![Page 26: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/26.jpg)
- METADATA
- RECORDa list of installed files (CSV formatting)
- RESOURCESlist of non-python files
- INSTALLERname of the installer
- REQUESTEDexists if distribution wasn’t installed as dependency
Other files that are shipped with distutils2
![Page 27: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/27.jpg)
1) Keep your setup.py files simple2) Follow the standards3) Use virtualenv/pip4) Look out for distutils2/packaging5) In doubt, read the
“Hitchhiker’s Guide to Packaging”
6
![Page 28: How I learned to stop worrying & love Python Packaging](https://reader033.vdocument.in/reader033/viewer/2022052523/5554afbfb4c905fd608b54d3/html5/thumbnails/28.jpg)
d @enn_io
oW Jannis Leidel
Thanks!
Questions?
Don‘t forget to
while we’re in Amsterdam!