![Page 1: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/1.jpg)
Antonio Verardi@porosVII
poros.github.io
write more decorators(and fewer classes)
![Page 2: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/2.jpg)
Yelp’s MissionConnecting people with great
local businesses.
![Page 3: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/3.jpg)
Stop Writing Classes by Jack Diederich
The controller pattern is awful (and other OO heresy) by Lexy Munroe aka Eevee
this talk has been inspired by
![Page 5: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/5.jpg)
let users utilize all python featuresinstead of
just inheritance
![Page 6: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/6.jpg)
go for decoratorswhen your classes
have only one methodand
are instantiated only once
![Page 7: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/7.jpg)
what’s this all about?
![Page 8: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/8.jpg)
from celery import Celery
app = Celery('tasks', broker='pyamqp://guest@localhost//')
@app.taskdef add(x, y): return x + y
![Page 9: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/9.jpg)
from flask import Flask
app = Flask(__name__)
@app.route("/")def hello(): return "Hello World!"
if __name__ == "__main__": app.run()
![Page 10: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/10.jpg)
from wsgiref.simple_server import make_serverfrom pyramid.config import Configuratorfrom pyramid.view import view_config
@view_config(route_name='hello', renderer='string')def hello_world(request): return 'Hello World'
if __name__ == '__main__': config = Configurator() config.add_route('hello', '/') config.scan() app = config.make_wsgi_app() server = make_server('0.0.0.0', 8080, app) server.serve_forever()
![Page 11: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/11.jpg)
but what about classes?
![Page 12: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/12.jpg)
import unittest
class WidgetTestCase(unittest.TestCase): def setUp(self): self.widget = Widget('The widget')
def test_default_widget_size(self): self.assertEqual(self.widget.size(), (50, 50))
def test_widget_resize(self): self.widget.resize(100, 150) self.assertEqual(self.widget.size(), (100, 150))
![Page 13: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/13.jpg)
import pytest
@pytest.fixturedef widget(): return Widget('The widget')
def test_default_widget_size(widget): assert widget.size() == (50, 50)
def test_widget_resize(widget): widget.resize(100, 150) assert widget.size() == (100, 150)
![Page 14: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/14.jpg)
that’s cool, what’s the trick?
![Page 15: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/15.jpg)
def fixture(scope="function", params=None, autouse=False, ids=None, name=None): if callable(scope) and params is None and autouse == False: # direct decoration return FixtureFunctionMarker( "function", params, autouse, name=name)(scope) if params is not None and not isinstance(params, (list, tuple)): params = list(params) return FixtureFunctionMarker(scope, params, autouse, ids=ids, name=name)
class FixtureFunctionMarker: def __init__(self, scope, params, autouse=False, ids=None, name=None): self.scope = scope self.params = params self.autouse = autouse self.ids = ids self.name = name
def __call__(self, function): if isclass(function): raise ValueError( "class fixtures not supported (may be in the future)") function._pytestfixturefunction = self return function
![Page 16: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/16.jpg)
class view_config(object): venusian = venusian def __init__(self, **settings): if 'for_' in settings: if settings.get('context') is None: settings['context'] = settings['for_'] self.__dict__.update(settings)
def __call__(self, wrapped): settings = self.__dict__.copy() depth = settings.pop('_depth', 0)
def callback(context, name, ob): config = context.config.with_package(info.module) config.add_view(view=ob, **settings)
info = self.venusian.attach(wrapped, callback, category='pyramid', depth=depth + 1)
if info.scope == 'class': if settings.get('attr') is None: settings['attr'] = wrapped.__name__
settings['_info'] = info.codeinfo return wrapped
![Page 17: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/17.jpg)
class Celery(object): def task(self, *args, **opts): if USING_EXECV and opts.get('lazy', True): from . import shared_task return shared_task(*args, lazy=False, **opts)
def inner_create_task_cls(shared=True, filter=None, lazy=True, **opts): _filt = filter # stupid 2to3
def _create_task_cls(fun): if shared: def cons(app): return app._task_from_fun(fun, **opts) cons.__name__ = fun.__name__ connect_on_app_finalize(cons) if not lazy or self.finalized: ret = self._task_from_fun(fun, **opts) else: ret = PromiseProxy(self._task_from_fun, (fun,), opts, __doc__=fun.__doc__) self._pending.append(ret) if _filt: return _filt(ret) return ret
return _create_task_cls
if len(args) == 1: if callable(args[0]): return inner_create_task_cls(**opts)(*args) raise TypeError('argument 1 to @task() must be a callable') if args: raise TypeError( '@task() takes exactly 1 argument ({0} given)'.format( sum([len(args), len(opts)]))) return inner_create_task_cls(**opts)
![Page 18: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/18.jpg)
![Page 19: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/19.jpg)
decorators are hard
![Page 20: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/20.jpg)
@decoratordef func(): pass
# is equivalent to
def func(): pass
func = decorator(func)
![Page 21: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/21.jpg)
@decorator(arg)def func(): pass
# is equivalent to
def func(): pass
func = decorator(arg)(func)
![Page 22: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/22.jpg)
@dec1(arg)@dec2def func(): pass
# is equivalent to
def func(): pass
func = dec1(arg)(dec2(func))
![Page 23: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/23.jpg)
decorators are hard
![Page 24: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/24.jpg)
decorators are hardto write
![Page 25: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/25.jpg)
def decorator(fn): print("Decorating function") return fn
@decoratordef func(arg1, arg2): print("Running")
![Page 26: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/26.jpg)
def decorator(fn): print("Decorating function") return fn
@decoratordef func(arg1, arg2): print("Running")
>>> from foo import funcDecorating function>>> foo("A","B")Running
![Page 27: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/27.jpg)
decorators are hardto write
![Page 28: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/28.jpg)
decorators are hardto write
easy
![Page 29: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/29.jpg)
@decorator(arg)def func(): pass
# is equivalent to
def func(): pass
func = decorator(arg)(func)
![Page 30: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/30.jpg)
def decorator(arg1, arg2): def actual_decorator(fn): return fn
print(f"Decorating function with {arg1} and {arg2}") return actual_decorator
@decorator("bar", "baz")def func(arg3, arg4): print("Running")
![Page 31: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/31.jpg)
def decorator(arg1, arg2): def actual_decorator(fn): return fn
print(f"Decorating function with {arg1} and {arg2}") return actual_decorator
@decorator("bar", "baz")def func(arg3, arg4): print("Running")
>>> from foo import funcDecorating function with bar and baz>>> foo("A","B")Running
![Page 32: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/32.jpg)
def decorator(fn): @functools.wraps(fn) # preserve function's metadata def wrapper(*args, **kwargs): print("Before the function runs") return fn(*args, **kwargs)
print("Decorating function") return wrapper
@decoratordef func(arg1, arg2): print("Running")
![Page 33: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/33.jpg)
def decorator(fn): @functools.wraps(fn) # preserve function's metadata def wrapper(*args, **kwargs): print("Before the function runs") return fn(*args, **kwargs)
print("Decorating function") return wrapper
@decoratordef func(arg1, arg2): print("Running")
>>> from foo import funcDecorating function>>> foo("A","B")Before the function runsRunning
![Page 34: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/34.jpg)
decorators are hardto write
easy
![Page 35: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/35.jpg)
decorators are hardto write
easya bit tedious
![Page 36: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/36.jpg)
class Flask(_PackageBoundObject): def route(self, rule, **options): def decorator(f): endpoint = options.pop('endpoint', None) self.add_url_rule(rule, endpoint, f, **options) return f return decorator
![Page 37: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/37.jpg)
real life example
![Page 38: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/38.jpg)
statmonster
![Page 39: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/39.jpg)
statmonster is a framework to extract real-time metrics out of logs
![Page 40: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/40.jpg)
![Page 41: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/41.jpg)
logs → statmonster → metrics
![Page 42: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/42.jpg)
{"start": 1000, "duration": 42, "method": "GET", "code": 200}
Timer("request", 1042, 42, {"method": "GET"})
→→
![Page 43: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/43.jpg)
logs → statmonster → metrics
![Page 44: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/44.jpg)
from enum import Enumfrom collections import namedtuplefrom functools import partial
MetricType = Enum("MetricType", ("COUNTER", "TIMER"))
Metric = namedtuple("Metric",("name", "ts", "value", "dims", "type")
)
Counter = partial(Metric, type=MetricType.COUNTER)Timer = partial(Metric, type=MetricType.TIMER)
![Page 45: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/45.jpg)
from enum import Enumfrom collections import namedtuplefrom functools import partial
MetricType = Enum("MetricType", ("COUNTER", "TIMER"))
Metric = namedtuple("Metric",("name", "ts", "value", "dims", "type")
)
Counter = partial(Metric, type=MetricType.COUNTER)Timer = partial(Metric, type=MetricType.TIMER)
![Page 46: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/46.jpg)
from enum import Enumfrom collections import namedtuplefrom functools import partial
MetricType = Enum("MetricType", ("COUNTER", "TIMER"))
Metric = namedtuple("Metric",("name", "ts", "value", "dims", "type")
)
Counter = partial(Metric, type=MetricType.COUNTER)Timer = partial(Metric, type=MetricType.TIMER)
![Page 47: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/47.jpg)
from enum import Enumfrom collections import namedtuplefrom functools import partial
MetricType = Enum("MetricType", ("COUNTER", "TIMER"))
Metric = namedtuple("Metric",("name", "ts", "value", "dims", "type")
)
Counter = partial(Metric, type=MetricType.COUNTER)Timer = partial(Metric, type=MetricType.TIMER)
![Page 48: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/48.jpg)
logs → statmonster → metrics
![Page 49: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/49.jpg)
import json
def decode_json(line): return json.loads(line)
class Log: name = None decoder = decode_json
@classmethod def decode(cls, line): return cls.decoder(line)
def __init__(self): assert self.name, "log name must be specified"
![Page 50: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/50.jpg)
import json
def decode_json(line): return json.loads(line)
class Log: name = None decoder = decode_json
@classmethod def decode(cls, line): return cls.decoder(line)
def __init__(self): assert self.name, "log name must be specified"
![Page 51: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/51.jpg)
import json
def decode_json(line): return json.loads(line)
class Log: name = None decoder = decode_json
@classmethod def decode(cls, line): return cls.decoder(line)
def __init__(self): assert self.name, "log name must be specified"
![Page 52: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/52.jpg)
from statmonster import Log
class EventsLog(Log): name = "events"
![Page 53: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/53.jpg)
from statmonster import Log
def decode_text(line): time, request = line.split() return {'time': time, 'request': request}
class RequestsLog(Log): name = "tmp_requests" decoder = decode_text
![Page 54: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/54.jpg)
logs → statmonster → metrics
![Page 55: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/55.jpg)
a trigger is a class which encapsulate the logic to extract metrics from a log
![Page 56: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/56.jpg)
class Trigger: owners = None def __init__(self): assert self.owners
def digest(self, entry): raise NotImplementedError
![Page 57: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/57.jpg)
class Trigger: owners = None def __init__(self): assert self.owners
def digest(self, entry): raise NotImplementedError
![Page 58: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/58.jpg)
class Trigger: owners = None def __init__(self): assert self.owners
def digest(self, entry): raise NotImplementedError
![Page 59: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/59.jpg)
def process(log, triggers, line): entry = log.decode(line) for trigger in triggers: try: yield from trigger.digest(entry) except Exception as e: send_email(trigger.owners, e)
![Page 60: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/60.jpg)
from statmonster import Log, Trigger, Counter
class EventsLog(Log): name = "events"
class CountEventsTrigger(Trigger): owners = ["[email protected]"]
def digest(self, entry): yield Counter(
"events",entry["time"], 1, {"type": entry["type"]}
)
![Page 61: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/61.jpg)
from statmonster import Log, Trigger, Counter
class EventsLog(Log): name = "events"
class CountEventsTrigger(Trigger): owners = ["[email protected]"]
def digest(self, entry): yield Counter(
"events",entry["time"], 1, {"type": entry["type"]}
)
![Page 62: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/62.jpg)
from statmonster import Log, Trigger, Counter
class EventsLog(Log): name = "events"
class CountEventsTrigger(Trigger): owners = ["[email protected]"]
def digest(self, entry): yield Counter(
"events",entry["time"], 1, {"type": entry["type"]}
)
![Page 63: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/63.jpg)
from statmonster import Log, Trigger, Counter
class EventsLog(Log): name = "events"
class CountEventsTrigger(Trigger): owners = ["[email protected]"]
def digest(self, entry): yield Counter(
"events",entry["time"], 1, {"type": entry["type"]}
)
![Page 64: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/64.jpg)
from statmonster import Log, Trigger, Timer
class RusageLog(Log): name = "rusage"
class TimeCpuTrigger(Trigger): owners = ["[email protected]", "[email protected]"]
def digest(self, entry): for metric in ("stime", "utime"): yield Timer( f"cpu.{metric}", entry["start_time"], entry[metric], {} )
![Page 65: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/65.jpg)
from statmonster import Log, Trigger, Timer
class RusageLog(Log): name = "rusage"
class TimeCpuTrigger(Trigger): owners = ["[email protected]", "[email protected]"]
def digest(self, entry): for metric in ("stime", "utime"): yield Timer( f"cpu.{metric}", entry["start_time"], entry[metric], {} )
![Page 66: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/66.jpg)
from statmonster import Log, Trigger, Timer
class RusageLog(Log): name = "rusage"
class TimeCpuTrigger(Trigger): owners = ["[email protected]", "[email protected]"]
def digest(self, entry): for metric in ("stime", "utime"): yield Timer( f"cpu.{metric}", entry["start_time"], entry[metric], {} )
![Page 67: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/67.jpg)
from statmonster import Log, Trigger, Timer
class RusageLog(Log): name = "rusage"
class TimeCpuTrigger(Trigger): owners = ["[email protected]", "[email protected]"]
def digest(self, entry): for metric in ("stime", "utime"): yield Timer( f"cpu.{metric}", entry["start_time"], entry[metric], {} )
![Page 68: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/68.jpg)
![Page 69: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/69.jpg)
USERS
![Page 70: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/70.jpg)
ENGINEERS
![Page 71: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/71.jpg)
how do I inherit from a base log class?
![Page 72: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/72.jpg)
![Page 73: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/73.jpg)
from statmonster import Logfrom utils import decode_apache
class ApacheBaseLog(Log): decoder = decode_apache
_apache.py
![Page 74: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/74.jpg)
from statmonster import Logfrom utils import decode_apache
class ApacheBaseLog(Log): decoder = decode_apache
_apache.py
![Page 75: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/75.jpg)
![Page 76: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/76.jpg)
from ._apache import ApacheBaseLog
class AccessLog(ApacheBaseLog): name = "access"
access.py
![Page 77: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/77.jpg)
from ._apache import ApacheBaseLog
class AdminAccessLog(ApacheBaseLog): name = "admin_access"
admin_access.py
![Page 78: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/78.jpg)
how do I inherit from a base trigger class?
![Page 79: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/79.jpg)
![Page 80: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/80.jpg)
from statmonster import Trigger, Counter
class EndpointsTimingBaseTrigger(Trigger): metric_name = None endpoints = None
def __init__(self): super().__init__() assert self.metric_name assert self.endpoints
def get_additional_dimensions(self, entry): return {}
_endpoints.py
![Page 81: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/81.jpg)
from statmonster import Trigger, Counter
class EndpointsTimingBaseTrigger(Trigger): metric_name = None endpoints = None
def __init__(self): super().__init__() assert self.metric_name assert self.endpoints
def get_additional_dimensions(self, entry): return {}
_endpoints.py
![Page 82: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/82.jpg)
from statmonster import Trigger, Counter
class EndpointsTimingBaseTrigger(Trigger): metric_name = None endpoints = None
def __init__(self): super().__init__() assert self.metric_name assert self.endpoints
def get_additional_dimensions(self, entry): return {}
_endpoints.py
![Page 83: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/83.jpg)
def digest(self, entry): ts = entry['start_time'] add_dims = self.get_additional_dimensions(entry) for endpoint in self.endpoints: if endpoint == entry["endpoint"]: timing = entry["endpoint"]["time"] dims = {'endpoint': endpoint}.update(add_dims) yield Counter( self.metric_name, ts, timing, dims )
_endpoints.py
![Page 84: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/84.jpg)
def digest(self, entry): ts = entry['start_time'] add_dims = self.get_additional_dimensions(entry) for endpoint in self.endpoints: if endpoint == entry["endpoint"]: timing = entry["endpoint"]["time"] dims = {'endpoint': endpoint}.update(add_dims) yield Counter( self.metric_name, ts, timing, dims )
_endpoints.py
![Page 85: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/85.jpg)
def digest(self, entry): ts = entry['start_time'] add_dims = self.get_additional_dimensions(entry) for endpoint in self.endpoints: if endpoint == entry["endpoint"]: timing = entry["endpoint"]["time"] dims = {'endpoint': endpoint}.update(add_dims) yield Counter( self.metric_name, ts, timing, dims )
_endpoints.py
![Page 86: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/86.jpg)
![Page 87: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/87.jpg)
from statmonster import Logfrom ._endpoints import EndpointsTimingBaseTrigger
class HomePageLog(Log): name = "homepage"
class HomePageEndpointsTrigger(EndpointsTimingBaseTrigger): owners = ["[email protected]"] endpoints = ["best_of_yelp", "suggestions", "nearby"] metric_name = "home"
def get_additional_dimensions(self, entry): return { "mode": entry.get("mode", "sync"), "user": "loggedin" if entry["user"] else "anon", }
![Page 88: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/88.jpg)
from statmonster import Logfrom ._endpoints import EndpointsTimingBaseTrigger
class HomePageLog(Log): name = "homepage"
class HomePageEndpointsTrigger(EndpointsTimingBaseTrigger): owners = ["[email protected]"] endpoints = ["best_of_yelp", "suggestions", "nearby"] metric_name = "home"
def get_additional_dimensions(self, entry): return { "mode": entry.get("mode", "sync"), "user": "loggedin" if entry["user"] else "anon", }
![Page 89: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/89.jpg)
from statmonster import Logfrom ._endpoints import EndpointsTimingBaseTrigger
class HomePageLog(Log): name = "homepage"
class HomePageEndpointsTrigger(EndpointsTimingBaseTrigger): owners = ["[email protected]"] endpoints = ["best_of_yelp", "suggestions", "nearby"] metric_name = "home"
def get_additional_dimensions(self, entry): return { "mode": entry.get("mode", "sync"), "user": "loggedin" if entry["user"] else "anon", }
![Page 90: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/90.jpg)
how do I inherit from two trigger classes?
![Page 91: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/91.jpg)
![Page 92: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/92.jpg)
you…
![Page 93: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/93.jpg)
you…just…
![Page 94: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/94.jpg)
you…just…
don’t...
![Page 95: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/95.jpg)
from statmonster import Trigger, Timer, Counter
class ServiceBaseTrigger(Trigger): metric_name = None
def __init__(self): super().__init__() assert self.metric_name
def make_key(self, stat_name): return f"{self.metric_name}.{stat_name}"
_service.py
![Page 96: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/96.jpg)
class ServiceTimingBaseTrigger(ServiceBaseTrigger):
def digest(self, entry): key = self.make_key("request_latency") dimensions = {'method': entry['method_name']}
yield Timer( key, entry['time'], entry['time_elapsed'], dimensions )
_service.py
![Page 97: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/97.jpg)
class ServiceTimingBaseTrigger(ServiceBaseTrigger):
def digest(self, entry): key = self.make_key("request_latency") dimensions = {'method': entry['method_name']}
yield Timer( key, entry['time'], entry['time_elapsed'], dimensions )
_service.py
![Page 98: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/98.jpg)
class ServiceCountBaseTrigger(ServiceBaseTrigger):
def digest(self, entry): try: bytes_written = int(entry['headers']['Content-Length']) except KeyError: bytes_written = 0 dimensions = {'method': entry['method_name']} request_count_key = self.make_key("request_count") bytes_written_key = self.make_key("total_written_bytes") ts = entry['time']
yield Counter(request_count_key, ts, 1, dimensions) yield Counter( bytes_written_key, ts, bytes_written, dimensions )
_service.py
![Page 99: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/99.jpg)
class ServiceCountBaseTrigger(ServiceBaseTrigger):
def digest(self, entry): try: bytes_written = int(entry['headers']['Content-Length']) except KeyError: bytes_written = 0 dimensions = {'method': entry['method_name']} request_count_key = self.make_key("request_count") bytes_written_key = self.make_key("total_written_bytes") ts = entry['time']
yield Counter(request_count_key, ts, 1, dimensions) yield Counter( bytes_written_key, ts, bytes_written, dimensions )
_service.py
![Page 100: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/100.jpg)
from statmonster import Logfrom ._service import ServiceTimingBaseTriggerfrom ._service import ServiceCountBaseTrigger
class SuggestionsLog(Log): name = "suggest"
class SuggestionsTimings(ServiceTimingBaseTrigger): owners = ["[email protected]"] metric_name = 'suggestions_timings'
class SuggestionsCount(ServiceCountBaseTrigger): owners = ["[email protected]"] metric_name = 'suggestions_count'
suggestions_service.py
![Page 101: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/101.jpg)
from statmonster import Logfrom ._service import ServiceTimingBaseTriggerfrom ._service import ServiceCountBaseTrigger
class SuggestionsLog(Log): name = "suggest"
class SuggestionsTimings(ServiceTimingBaseTrigger): owners = ["[email protected]"] metric_name = 'suggestions_timings'
class SuggestionsCount(ServiceCountBaseTrigger): owners = ["[email protected]"] metric_name = 'suggestions_count'
suggestions_service.py
![Page 102: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/102.jpg)
from statmonster import Logfrom ._service import ServiceTimingBaseTriggerfrom ._service import ServiceCountBaseTrigger
class SuggestionsLog(Log): name = "suggest"
class SuggestionsTimings(ServiceTimingBaseTrigger): owners = ["[email protected]"] metric_name = 'suggestions_timings'
class SuggestionsCount(ServiceCountBaseTrigger): owners = ["[email protected]"] metric_name = 'suggestions_count'
suggestions_service.py
![Page 103: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/103.jpg)
how do I test my trigger class?
![Page 104: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/104.jpg)
![Page 105: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/105.jpg)
from metrics.rusage import TimeCpuTriggerfrom statmonster import Timerimport pytest
@pytest.fixturedef trigger(): return TimeCpuTrigger()
def test_time_cpu(trigger): entry = {"start_time": 1000, "stime": 42, "utime": 33} assert Timer( "cpu.stime", 1000, 42, {} ) in list(trigger.digest(entry))
![Page 106: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/106.jpg)
from metrics.rusage import TimeCpuTriggerfrom statmonster import Timerimport pytest
@pytest.fixturedef trigger(): return TimeCpuTrigger()
def test_time_cpu(trigger): entry = {"start_time": 1000, "stime": 42, "utime": 33} assert Timer( "cpu.stime", 1000, 42, {} ) in list(trigger.digest(entry))
![Page 107: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/107.jpg)
from metrics.rusage import TimeCpuTriggerfrom statmonster import Timerimport pytest
@pytest.fixturedef trigger(): return TimeCpuTrigger()
def test_time_cpu(trigger): entry = {"start_time": 1000, "stime": 42, "utime": 33} assert Timer( "cpu.stime", 1000, 42, {} ) in list(trigger.digest(entry))
![Page 108: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/108.jpg)
but how do I test my base trigger class?
![Page 109: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/109.jpg)
![Page 110: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/110.jpg)
class ServiceCountBaseTrigger(ServiceBaseTrigger):
def digest(self, entry): try: bytes_written = int(entry['headers']['Content-Length']) except KeyError: bytes_written = 0 dimensions = {'method': entry['method_name']} request_count_key = self.make_key("request_count") bytes_written_key = self.make_key("total_written_bytes") ts = entry['time']
yield Counter(request_count_key, ts, 1, dimensions) yield Counter( bytes_written_key, ts, bytes_written, dimensions )
_service.py
![Page 111: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/111.jpg)
class ServiceCountBaseTrigger(ServiceBaseTrigger):
def digest(self, entry): try: bytes_written = int(entry['headers']['Content-Length']) except KeyError: bytes_written = 0 dimensions = {'method': entry['method_name']} request_count_key = self.make_key("request_count") bytes_written_key = self.make_key("total_written_bytes") ts = entry['time']
yield Counter(request_count_key, ts, 1, dimensions) yield Counter( bytes_written_key, ts, bytes_written, dimensions )
_service.py
![Page 112: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/112.jpg)
from statmonster import Trigger, Timer, Counter
class ServiceBaseTrigger(Trigger): metric_name = None
def __init__(self): super().__init__() assert self.metric_name
def make_key(self, stat_name): return f"{self.metric_name}.{stat_name}"
_service.py
![Page 113: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/113.jpg)
from statmonster import Trigger, Timer, Counter
class ServiceBaseTrigger(Trigger): metric_name = None
def __init__(self): super().__init__() assert self.metric_name
def make_key(self, stat_name): return f"{self.metric_name}.{stat_name}"
_service.py
![Page 114: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/114.jpg)
from statmonster import Trigger, Timer, Counter
class ServiceBaseTrigger(Trigger): metric_name = None
def __init__(self): super().__init__() assert self.metric_name
def make_key(self, stat_name): return f"{self.metric_name}.{stat_name}"
_service.py
![Page 115: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/115.jpg)
class Trigger: owners = None
def __init__(self): assert self.owners
def digest(self, line): raise NotImplementedError
![Page 116: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/116.jpg)
class Trigger: owners = None
def __init__(self): assert self.owners
def digest(self, line): raise NotImplementedError
![Page 117: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/117.jpg)
![Page 118: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/118.jpg)
import pytestfrom metrics._service import ServiceCountBaseTrigger
@pytest.fixturedef trigger(self): return type( "TestTrigger", (ServiceCountBaseTrigger,), { "metric_name": "test", "owners": ["[email protected]"] }, )()
![Page 119: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/119.jpg)
let’s make things better
![Page 120: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/120.jpg)
before decorators
![Page 121: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/121.jpg)
after decorators
![Page 122: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/122.jpg)
logs → statmonster → metrics
![Page 123: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/123.jpg)
what is a trigger?
![Page 124: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/124.jpg)
what is a trigger?
it’s a class that...
![Page 125: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/125.jpg)
what is a trigger?
it’s a class that...
![Page 126: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/126.jpg)
let users utilize all python featuresinstead of
just inheritance
![Page 127: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/127.jpg)
logs → statmonster → metrics
![Page 128: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/128.jpg)
from enum import Enumfrom collections import namedtuplefrom functools import partial
MetricType = Enum("MetricType", ("COUNTER", "TIMER"))
Metric = namedtuple("Metric",("name", "ts", "value", "dims", "type")
)
Counter = partial(Metric, type=MetricType.COUNTER)Timer = partial(Metric, type=MetricType.TIMER)
![Page 129: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/129.jpg)
logs → statmonster → metrics
![Page 130: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/130.jpg)
class Log: def __init__(self, name, decoder=decode_json): self.name = name self.decoder = decoder self.fns = set()
def decode(self, line): return self.decoder(line)
![Page 131: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/131.jpg)
from statmonster import Log
events = Log("events")rusage = Log("rusage")suggestions_service = Log("suggest")home = Log("homepage")
![Page 132: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/132.jpg)
from statmonster import Log
def decode_text(line): time, request = line.split() return {'time': time, 'request': request}
requests = Log("tmp_requests", decoder=decode_text)
![Page 133: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/133.jpg)
logs → statmonster → metrics
![Page 134: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/134.jpg)
from emails import send_email
class Log: def process(self, line): entry = self.decode(line) for fn in self.fns: try: yield from fn(entry) except Exception as e: send_email(fn.owners, e)
![Page 135: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/135.jpg)
from .logs import eventsfrom statmonster import owners, register, Counter
@events.register@owners("[email protected]")def count_events(entry): yield Counter( "events", entry["time"], 1, {"type": entry["type"]})
![Page 136: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/136.jpg)
from emails import send_email
class Log: def register(self, fn): self.fns.add(fn) return fn
![Page 137: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/137.jpg)
class Flask(_PackageBoundObject): def route(self, rule, **options): def decorator(f): endpoint = options.pop('endpoint', None) self.add_url_rule(rule, endpoint, f, **options) return f return decorator
![Page 138: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/138.jpg)
def register(*logs): def decorator(fn): for log in logs: log.register(fn) return fn
return decorator
![Page 139: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/139.jpg)
def owners(*handlers): def decorator(fn): fn.owners = handlers return fn
return decorator
![Page 140: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/140.jpg)
from .logs import rusage, aux_rusagefrom statmonster import owners, Timer
@register(rusage, aux_rusage)@owners("[email protected]", "[email protected]")def time_cpu(entry): for metric in ("stime", "utime"): yield Timer( f"cpu.{metric}", entry["start_time"], entry[metric], {} )
![Page 141: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/141.jpg)
how do I inherit from a base log class?
![Page 142: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/142.jpg)
![Page 143: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/143.jpg)
from statmonster import Logfrom utils import decode_apache
ApacheLog = partial(Log, decoder=decode_apache)access = ApacheLog("access")admin_access = ApacheLog("admin_access")
![Page 144: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/144.jpg)
how do I inherit from a base trigger class?
![Page 145: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/145.jpg)
![Page 146: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/146.jpg)
from statmonster import Counter
def emit_endpoints_timings( metric_name, endpoints, additional_dims, entry): ts = entry['start_time'] for endpoint in endpoints: if endpoint == entry["endpoint"]: timing = entry["endpoint"]["time"] dims = {'endpoint': endpoint}.update(additional_dims) yield Counter(metric_name, ts, timing, dims)
![Page 147: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/147.jpg)
from .logs import homefrom .endpoints import emit_endpoints_timingsfrom statmonster import owners, register
ENDPOINTS = ["best_of_yelp", "suggestions", "nearby"]
@register(home)@owners("[email protected]")def time_home_endpoints(entry): dims = { "mode": entry.get("mode", "sync"), "user": "loggedin" if entry["username"] else "anon", } yield from emit_endpoints_timings( "home", ENDPOINTS, dims, entry )
![Page 148: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/148.jpg)
from .logs import homefrom .endpoints import emit_endpoints_timingsfrom statmonster import owners, register
ENDPOINTS = ["best_of_yelp", "suggestions", "nearby"]
@register(home)@owners("[email protected]")def time_home_endpoints(entry): dims = { "mode": entry.get("mode", "sync"), "user": "loggedin" if entry["username"] else "anon", } yield from emit_endpoints_timings( "home", ENDPOINTS, dims, entry )
![Page 149: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/149.jpg)
how do I inherit from two trigger classes?
![Page 150: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/150.jpg)
![Page 151: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/151.jpg)
def make_key(metric_name, stat_name): return f"{metric_name}.{stat_name}"
def emit_service_timings(prefix, entry): key = make_key(prefix, "requestLatency") dimensions = {"method": entry["method_name"]}
yield Timer( key, entry["time"], entry["time_elapsed"], dimensions )
![Page 152: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/152.jpg)
def emit_service_counters(prefix, entry): try: bytes_written = int(entry["headers"]["Content-Length"]) except KeyError: bytes_written = 0 dimensions = {"method": entry["method_name"]} request_count_key = make_key(prefix, "request_count") bytes_written_key = make_key(prefix, "total_written_bytes") ts = entry["time"]
yield Counter(request_count_key, ts, 1, dimensions) yield Counter( bytes_written_key, ts, bytes_written, dimensions )
![Page 153: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/153.jpg)
from statmonster import ownersfrom metrics.logs import suggestions_servicefrom metrics.service import emit_service_timingsfrom metrics.service import emit_service_counters
@suggestions_service.register@owners("[email protected]")def emit_suggestions_metrics(entry): yield from emit_service_timings("suggestion_timings", entry) yield from emit_service_counters("suggestion_count", entry)
![Page 154: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/154.jpg)
how do I test my trigger class?
![Page 155: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/155.jpg)
![Page 156: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/156.jpg)
from metrics.rusage import time_cpufrom statmonster import Timer
def test_time_cpu(): entry = {"start_time": 1000, "stime": 42, "utime": 33} assert Timer( "cpu.stime", 1000, 42, {} ) in list(time_cpu(entry))
![Page 157: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/157.jpg)
from metrics.rusage import time_cpufrom statmonster import Timer
def test_time_cpu(): entry = {"start_time": 1000, "stime": 42, "utime": 33} assert Timer( "cpu.stime", 1000, 42, {} ) in list(time_cpu(entry))
![Page 158: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/158.jpg)
but how do I test my base trigger class?
![Page 159: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/159.jpg)
![Page 160: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/160.jpg)
from metrics.service import emit_service_counters
def test_emit_service_counters(): entry = { "time": 123456, "status": 200, "method_name": "GET", "blob": "foo", } expected = Counter( "test.request_count", 123456, 1, {"method": "GET"} ) assert expected in list(emit_service_counters("test", entry))
![Page 161: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/161.jpg)
from metrics.service import emit_service_counters
def test_emit_service_counters(): entry = { "time": 123456, "status": 200, "method_name": "GET", "blob": "foo", } expected = Counter( "test.request_count", 123456, 1, {"method": "GET"} ) assert expected in list(emit_service_counters("test", entry))
![Page 162: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/162.jpg)
but how do I make sureall functions have owners?
![Page 163: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/163.jpg)
from collect import collect
def test_owners(): for name, log in collect(): for fn in log.fns: assert hasattr(fn, "owners")
![Page 164: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/164.jpg)
all of this only for users’ sake?
![Page 165: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/165.jpg)
import system
![Page 166: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/166.jpg)
from importlib import import_moduleimport globimport osimport inspectfrom statmonster import Log
def collect(): for file_path in glob.iglob(os.path.join("metrics", "*.py")): module_name = '.'.join(file_path[:-3].split('/')) import_module(module_name)
module = import_module("metrics.logs") for name, obj in inspect.getmembers(module): if isinstance(obj, Log): yield name, obj
if __name__ == "__main__": for name, log in collect(): print(f"{name}:{log}")
![Page 167: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/167.jpg)
from importlib import import_moduleimport globimport osimport inspectfrom statmonster import Log, Trigger
DEFAULT_DIR = "metrics"
def collect(): all_triggers = {} for module_name, module in get_triggers_modules(): log, triggers = get_log_and_triggers(module_name, module) if log: all_triggers[log] = triggers return all_triggers
def get_log_and_triggers(module_name, module): log = None triggers = [] for name, cls in inspect.getmembers(module, inspect.isclass): if is_valid_log(module_name, cls): assert not log, "Multiple logs in the same module: %s" % module log = cls() elif is_valid_trigger(module_name, cls): triggers.append(cls()) return log, set(triggers)
def is_valid_log(module_name, log): return (issubclass(log, Log) and log.__module__ == f"{DEFAULT_DIR}.{module_name}")
def is_valid_trigger(module_name, trigger): return (issubclass(trigger, Trigger) and trigger.__module__ == "%s.%s" % (DEFAULT_DIR, module_name))
def get_triggers_modules(triggers_folder=DEFAULT_DIR): for file_path in glob.iglob(os.path.join(triggers_folder, "*.py")): name = os.path.basename(file_path)[:-3] # Filter eg. _service and __init__ if not name.startswith('_'): yield name, dynamic_import_module(file_path)
def dynamic_import_module(module_path): module_name = '.'.join(module_path[:-3].split('/')) return import_module(module_name)
if __name__ == "__main__": for log, triggers in collect().items(): print(f"{log}:{triggers}")
![Page 168: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/168.jpg)
![Page 169: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/169.jpg)
let users utilize all python featuresinstead of
just inheritance
![Page 170: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/170.jpg)
go for decorators
![Page 171: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/171.jpg)
go for decoratorswhen your classes
![Page 172: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/172.jpg)
go for decoratorswhen your classes
have only one method
![Page 173: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/173.jpg)
go for decoratorswhen your classes
have only one methodand
are instantiated only once
![Page 174: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/174.jpg)
www.yelp.com/careers/
We're Hiring!
![Page 175: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/175.jpg)
@YelpEngineering
fb.com/YelpEngineers
engineeringblog.yelp.com
github.com/yelp
![Page 176: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/176.jpg)
QUESTIONS?
![Page 177: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/177.jpg)
backup slides
![Page 178: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/178.jpg)
some syntax
![Page 179: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/179.jpg)
yield from iterable
# shortened form of
for item in iterable: yield item
![Page 180: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/180.jpg)
from functools import partial
# convert base 2 string to an intbasetwo = partial(int, base=2)assert basetwo('10010') == 18
![Page 181: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/181.jpg)
from enum import Enum
Color = Enum("Color", ("RED", "GREEN", "BLUE"))
>>> print(repr(Color.RED))<Color.RED: 1>
![Page 182: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/182.jpg)
putting all together
![Page 183: poros.github.io @porosVII write more decorators Antonio ... · Antonio Verardi @porosVII poros.github.io write more decorators (and fewer classes)](https://reader036.vdocument.in/reader036/viewer/2022062403/5fe60c7971901a24cb29f5b9/html5/thumbnails/183.jpg)
def decorator(arg1, arg2): def actual_decorator(fn): @functools.wraps(fn) # preserve function's metadata def wrapper(*args, **kwargs): print("Before the function runs") return fn(*args, **kwargs) return wrapper
print(f"Decorating function with {arg1} and {arg2}") return actual_decorator
@decorator("bar", "baz")def func(arg3, arg4): pass