pyconit6 - making sessions and caching roommates
Embed Size (px)
BEAKER - MAKING SESSIONS AND CACHING ROOMMATES
Who am I
● CTO @ AXANT.it, mostly Python company
(with some iOS and Android)
● TurboGears2 dev team member
● Took over Beaker maintenace in 2015
● Mostly contributed to web python libraries:
Formencode, MING MongoDB ODM, ToscaWidgets2
● Framework for handling Caching and
Sessions in web applications
● Used by many different frameworks:
TurboGears, Bottle, Pyramid, etc...
● Created by Pylons Author to solve the
● Handles the DogPile effect
● Provides many backends: Memcache, File,
MongoDB, Redis, SQLAlchemy
● A single tool to handle both caching and
sessions with common backends.
● File based synchronization, what about
● Session as a big BLOB of data doesn’t
perform well in case of big BLOBs of data
● Updated for too long without breaking
backward compatibility: Too many APIs
Getting on project: PY3 Support
● Beaker supported Python3 using 2to3
● This lead to some bugs (not everything got
● Made really hard to run the test suite and
maintain the project.
So my first action as new maintainer...
Rewriting for Py3 a project that involves serializing/deserializing
Porting to PY3: #1 Snakes speak ?
Beaker uses BASE64 to encode pickled data
>>> import base64, pickle
>>> data = dict(key='value', otherkey='othervalue')
Here are a few requirements that
determine which alphabet should be
o Handled by humans.
>>> import base64, pickle
Porting to PY3: #2 Session Cookies?
● Session ID is stored in cookies
● CookieSession stores even the whole
session in a cookie
○ Great idea, makes really simple to scale
○ HTTP Headers are plain text
○ We are pickling data and base64 so already ASCII
○ Just need to encrypt it to avoid people from
messing with them.
HTTP CookiesHTTP has allowed field content with text
in the ISO-8859-1 charset [ISO-8859-1],
supporting other charsets only through
use of [RFC2047] encoding. In practice,
most HTTP header field values use only a
subset of the US-ASCII charset [USASCII].
Newly defined header fields SHOULD limit
their field values to US-ASCII octets
Encoding/Encripting data on Py3
● Beaker expected to be able to perform
text operations on the result of base64
● Relied on AES to encrypt cookies, pbkdf2
to generates keys and base64/binascii
to make it text.
● Reads the encrypt_key from config file
4 lines of code == 10 doubts
● Even Python has no clear idea of what is
text and what are bytes:
○ Both BASE64 and BINASCII work with ASCII
○ BINASCII accepts unicode (text)
○ BASE64 accepts bytes
○ Between Python3.2 and Python3.3 the binascii
changed behaviour of accepting unicode vs bytes
○ ConfigParser returns text on py3, bytes on py2
Now crying in a corner...
Let’s go for something simpler...
● Review patch to caching decorator
● To cache a function, just apply a decorator
that calls function and caches the result
● Caching decorator generates cache key
○ Cache Key from the function name
○ convert parameters to strings
○ add parameters to cache key
Getting the Cache Key def cache(self, *args, **kwargs):
"""Decorate a function to cache itself with supplied parameters
:param args: Used to make the key unique for this function, as in region()
:param kwargs: Parameters to be passed to get_cache(), will override defaults
# Assuming a cache object is available like:
cache = CacheManager(dict_of_config_options)
def load(search_term, limit, offset):
return load_the_data(search_term, limit, offset)
return load('rabbits', 20, 0)
return _cache_decorate(args, self, kwargs, None)
● Decorator gets *args and **kwargs
● Makes sense... doesn’t know the real
● Not so much in fact… func(1) and func
(a=1) end up with two different cache
Avoid the doubt
● Beaker solved this by accepting only
positional arguments on cached functions .. note::
The function being decorated must only be called with
● Users not so happy, breaks people code
when they introduce caching:
What to do?
● Leave it as is… As been like that for years
● Add **kwargs and just document the issue
● Add **kwargs and try to be smart using
inspect.getargspec / inspect.
getcallargs but that would be slow
Leave it as is… at least isn’t broken
Has been a great trip!
● Making sessions and caching in a single
solution seemed simple, but has actually a
lot of corner case
● Beaker Co-Author thrown in the towel and
created dopgile.cache which is based on
beaker code but is much simpler and
doesn’t provide sessions
Still is incredibily convenient
● There are more shared features than
differences (especially in cookie based
● It’s really convenient, write backends once
and have support for both sessions and
caching on them.