BEAKER - MAKING SESSIONS AND CACHING ROOMMATES

Alessandro Molina@__amol__

amol@turbogears.org

Who am I

● CTO @ AXANT.it, mostly Python company

(with some iOS and Android)

● TurboGears2 dev team member

● Took over Beaker maintenace in 2015

● Mostly contributed to web python libraries:

Formencode, MING MongoDB ODM, ToscaWidgets2

What’s Beaker

● Framework for handling Caching and

Sessions in web applications

● Used by many different frameworks:

TurboGears, Bottle, Pyramid, etc...

● Created by Pylons Author to solve the

dogpile effect.

DogPile!

The Good

● Handles the DogPile effect

● Provides many backends: Memcache, File,

MongoDB, Redis, SQLAlchemy

● A single tool to handle both caching and

sessions with common backends.

The Bad

● File based synchronization, what about

distributed solutions?

● Session as a big BLOB of data doesn’t

perform well in case of big BLOBs of data

● Updated for too long without breaking

backward compatibility: Too many APIs

Getting on project: PY3 Support

● Beaker supported Python3 using 2to3

● This lead to some bugs (not everything got

properly converted)

● Made really hard to run the test suite and

maintain the project.

So my first action as new maintainer...

Rewriting for Py3 a project that involves serializing/deserializing

Porting to PY3: #1 Snakes speak ?

Beaker uses BASE64 to encode pickled data

in sessions.

>>> import base64, pickle

>>> data = dict(key='value', otherkey='othervalue')

>>> base64.b64encode(pickle.dumps(data))

'KGRwMApTJ290aGVya2V5JwpwMQpTJ290aGVydmFsdWUnCnAyCnN

TJ2tleScKcDMKUyd2YWx1ZScKcDQKcy4='

BASE64: rfc4648

Here are a few requirements that

determine which alphabet should be

used:

o Handled by humans.

Human handle?

TEXT!

Human handle?

TEXT!

Human handle?

BYTES!

>>> import base64, pickle

>>> base64.b64encode(pickle.dumps(‘HELLO’))

b'gANYBQAAAEhFTExPcQAu'

Porting to PY3: #2 Session Cookies?

● Session ID is stored in cookies

● CookieSession stores even the whole

session in a cookie

○ Great idea, makes really simple to scale

○ HTTP Headers are plain text

○ We are pickling data and base64 so already ASCII

○ Just need to encrypt it to avoid people from

messing with them.

HTTP CookiesHTTP has allowed field content with text

in the ISO-8859-1 charset [ISO-8859-1],

supporting other charsets only through

use of [RFC2047] encoding. In practice,

most HTTP header field values use only a

subset of the US-ASCII charset [USASCII].

Newly defined header fields SHOULD limit

their field values to US-ASCII octets

Encoding/Encripting data on Py3

● Beaker expected to be able to perform

text operations on the result of base64

● Relied on AES to encrypt cookies, pbkdf2

to generates keys and base64/binascii

to make it text.

● Reads the encrypt_key from config file

4 lines of code == 10 doubts

● Even Python has no clear idea of what is

text and what are bytes:

○ Both BASE64 and BINASCII work with ASCII

○ BINASCII accepts unicode (text)

○ BASE64 accepts bytes

○ Between Python3.2 and Python3.3 the binascii

changed behaviour of accepting unicode vs bytes

○ ConfigParser returns text on py3, bytes on py2

Now crying in a corner...

Let’s go for something simpler...

● Review patch to caching decorator

● To cache a function, just apply a decorator

that calls function and caches the result

● Caching decorator generates cache key

○ Cache Key from the function name

○ convert parameters to strings

○ add parameters to cache key

Getting the Cache Key def cache(self, *args, **kwargs):

"""Decorate a function to cache itself with supplied parameters

:param args: Used to make the key unique for this function, as in region()

above.

:param kwargs: Parameters to be passed to get_cache(), will override defaults

Example::

# Assuming a cache object is available like:

cache = CacheManager(dict_of_config_options)

def populate_things():

@cache.cache('mycache', expire=15)

def load(search_term, limit, offset):

return load_the_data(search_term, limit, offset)

return load('rabbits', 20, 0)

"""

return _cache_decorate(args, self, kwargs, None)

Well, maybe...

Unknown parameters

● Decorator gets *args and **kwargs

● Makes sense... doesn’t know the real

function arguments

● Not so much in fact… func(1) and func

(a=1) end up with two different cache

keys

Avoid the doubt

● Beaker solved this by accepting only

positional arguments on cached functions .. note::

The function being decorated must only be called with

positional arguments.

● Users not so happy, breaks people code

when they introduce caching:

○ https://github.com/bbangert/beaker/issues/62

What to do?

● Leave it as is… As been like that for years

● Add **kwargs and just document the issue

● Add **kwargs and try to be smart using

inspect.getargspec / inspect.

getcallargs but that would be slow

Leave it as is… at least isn’t broken

Has been a great trip!

● Making sessions and caching in a single

solution seemed simple, but has actually a

lot of corner case

● Beaker Co-Author thrown in the towel and

created dopgile.cache which is based on

beaker code but is much simpler and

doesn’t provide sessions

Still is incredibily convenient

● There are more shared features than

differences (especially in cookie based

marshalled sessions).

● It’s really convenient, write backends once

and have support for both sessions and

caching on them.

Questions?