RESTful webservices with Python

For lazy developersor

developers with deadlines

source: http://s3-ec.buzzfed.com/static/enhanced/webdr01/2012/12/2/13/enhanced-buzz-wide-18383-1354473319-2.jpg

Who am I?

Justyna Żarna Woman in Django / Python World

JavaScript freak

@Ustinez

http://solution4future.com

1. REST software architecture.

Content

2. Benchmarks - problem analysis.

3. Flask + SQLAlchemy.

4. Django & class-based views. } case study

XML/RPC

API

SOAP REST

based on HTTP protocol

many restriction on data types

synchronous

based on HTTP protocol or other protocols

synchronous

based on XML and all xml defects

???

REpresentational State Transfer

What is so cool?

Resource

Representation Representation Representation

GET PUT POST DELETE

}

THE VERBS

http://a.com/resources/

http://a.com/resources/item1/ http://a.com/resources/item2/ http://a.com/resources/item3/

REpresentational State Transfer

What is so cool?

1. Scalability

2. Generalization of interface

3. Independence

4. Security

5. Simplicity

6. In HTTP context but not limited to this protocol.

7. Good designed interface.

make your API RESTful and go to rest...http://www.2010theyearinbooks.com/2012/12/december-beach-reads-for-australian.html

Benchmark JSON response for frameworks and languages.

Python frameworks:

* Django-stripped 13 269 per second

* Flask - 11 506 per sec

* Django - 7122 per sec

http://www.techempower.com/blog/2013/04/05/frameworks-round-2/

The source code and problems

https://github.com/TechEmpower/FrameworkBenchmarks/http://www.techempower.com/blog/2013/04/05/frameworks-round-2/

In this souce code for flask and django isn't used connection pool - cache of database connection.

database database

connectionconnectionconnection

connection

connection are maintained for future

connection for each request is open and closed.

1. Database optimalization for PostgreSQL:● pgBouncer● pgPool

Solutions?

http://www.askthepony.com/blog/2011/07/django-and-postgresql-improving-the-performance-with-no-effort-and-no-code/

2. JSON standard serialization in Python STD library is slow, so we can improve performance by using module ujson - ultra json.

Solutions?

code profiler can tell us more...

3. Rendering template can be faster with template engine jinja (inspired by Django templates)

Solutions?

● "micro" does not means micro possibilities,● core is light, simple and extensible,● support many extensions in each layer (for

example your database layer can by relational database or non-relational data persistence).

Flask

from sqlalchemy import Column, Integer, Stringfrom newsletter.database import Base

class Member(Base): __tablename__ = 'newsletter_members' id = Column(Integer, primary_key=True) last_name = Column(String(50)) first_name = Column(String(120)) email = Column(String(120), unique=True)

def __init__(self, last_name=None, first_name=None, email=None): self.last_name = last_name self.first_name = first_name self.email = email

def __repr__(self): return '<Member %r>' % (self.last_name)

Case study - micro framework Flask

class API(MethodView):

def get(self, member_id): if member_id is None: return Member.query.all() else: return Member.query.filter_by(id = member_id).first() def post(self, data ): member = Member(first_name = data['first_name'], email=data['email']) db.session.add(member) db.session.commit() return 'OK'

app.add_url_rule('/users/<int:user_id>', view_func=API.as_view('user_api'), methods=['GET', 'POST'])

#class flask.views.MethodView and recognizing each REST methods are based on generic dispatch_request()

Case study - micro framework Flask

It was simple and quick to code?

Case study - micro framework Flask

improve simplicity and spent time with...Flask RESTless

Flask RESTless - full scriptimport flaskimport flask.ext.sqlalchemyimport flask.ext.restlessapp = flask.Flask(__name__)app.config['DEBUG'] = Trueapp.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'db = flask.ext.sqlalchemy.SQLAlchemy(app)

class Member(db/Model): __tablename__ = 'newsletter_members' id = db.Column(Integer, primary_key=True) last_name = db.Column(String(50)) first_name = db.Column(String(120)) email = db.Column(String(120), unique=True)

db.create_all()

manager = flask.ext.restless.APIManager(app, flask_sqlalchemy_db=db)

manager.create_api(Member, methods=['GET', 'POST'])

app.run()

Too simple for real project.. ?

Flask RESTless - more sugar

● versioning:

GET /api/memberGET /api/member/(int: id)GET /api/member?q=<json>POST /api/member

apimanager.create_api(Member, url_prefix='/api/v2')GET /api/v2/member

● validation:

manager.create_api(Member, methods=['GET', 'POST'], validation_exceptions=[ValidationError])

{ "validation_errors": { "email": "Email is not valid..", }}

Flask RESTless - more sugar

● specify columns (include or exclude):

apimanager.create_api(Member, include_columns = ['last_name', 'email'])

● pagination:

manager.create_api(Member, methods=['GET', 'POST'], results_per_page=2)

{ "validation_errors": { "age": "Must be an integer", }{ "num_results": 6, "total_pages": 3, "page": 1, "objects": [ {"last_name": "Kovalsky", "email": "kovalsky@gmail.com", "id": 1}, {"last_name": "Novak", "email": "novak@gmail.com", "id": 2} ]}

Flask RESTless - more and more..

● pre/post-processors:def pre_get_single(instid): # do something pass

def pre_get_many(params): # do something pass

# Create an API for the Member model.manager.create_api(Person, methods=['GET', 'POST'], # A list of preprocessors for each method. preprocessors={'GET_SINGLE': [pre_get_single], 'GET_MANY': [pre_get_many],})

● Authentication:def auth_func(params): if not current_user.is_authenticated(): raise ProcessingException(message='Not authenticated!') return NO_CHANGEmanager.create_api(Person, preprocessors={'GET_SINGLE': [auth_func]})

Flask RESTless - more and more..● filtering:

import requestsimport json

url = 'http://127.0.0.1:5000/api/member'headers = {'Content-Type': 'application/json'}

filters = [dict(email='email', op='like', val='%y%')]params = dict(q=json.dumps(dict(filters=filters)))response = requests.get(url, params=params, headers=headers)

GET /api/member?q={"filters":[{"name":"email", "op":"like", "val": "kovalsky"}]}

OPERATORS examples:○ ==, eq, !=, neq,○ >, gte, <, lte○ in, not_in○ is_null○ like○ has○ any

FOR RELATIONS:● column__column example:

member__group

What about Django?

● Define your model:

from django.db import models

class Member(models.Model): last_name = models.CharField(max_length = 100, verbose_name = "Last name") first_name = models.CharField(max_length = 100, verbose_name = "First name") email = models.EmailField(max_length = 100, verbose_name = "Email")

def __unicode__(self): return self.email class Meta: verbose_name = "Newsletter member" verbose_name_plural = "Newsletter members"

What about Django?● Define your model serializer:

class MemberSerializer(serializers.ModelSerializer): class Meta: model = Member fields = ('last_name', 'first_name', 'email')

● Working with serializers:from newsletter.models import Memberfrom newsletter.serializers import MemberSerializerfrom rest_framework.renderers import JSONRendererfrom rest_framework.parsers import JSONParser

member = Member(last_name='Kovalsky', first_name= 'Johny', 'email' = 'kovalsky@gmail.com')member.save()

serializer = MemberSerializer(member)content = JSONRenderer().render(serializer.data)# content: {"pk": 2, "last_name": "Kovalsky", "first_name="Johny", email = "kovalsky@gmail.com"}, more object in format [{'foo': 'bar'}, {'foo': 'bar'}]

What about Django?● Define your class-based views:

class MemberDetail(APIView): """ Retrieve, update or delete a member instance. """ def get_object(self, pk): try: return Member.objects.get(pk=pk) except Member.DoesNotExist: raise Http404

def get(self, request, pk, format=None): member = self.get_object(pk) serializer = MemberSerializer(member) return Response(member.data)

What about Django?● Define your class-based views:

def put(self, request, pk, format=None): member = self.get_object(pk) serializer = MemberSerializer(member, data=request.DATA) if serializer.is_valid(): serializer.save() return Response(serializer.data) return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

def delete(self, request, pk, format=None): member = self.get_object(pk) member.delete() return Response(status=status.HTTP_204_NO_CONTENT)

What about Django?

● Or generic views:from newsletter.models import Memberfrom newsletter.serializers import MemberSerializerfrom rest_framework import generics

class MemberList(generics.ListCreateAPIView): model = Member serializer_class = MemberSerializer

class MemberDetail(generics.RetrieveUpdateDestroyAPIView): model = Member serializer_class = MemberSerializer

● URLs patterns:

urlpatterns = patterns('', url(r'^members/$', views.MeberList.as_view()), url(r'^members/(?P<pk>[0-9]+)/$', views.MemberDetail.as_view()),)

RESTful API - done

http://sarahwmackey.files.wordpress.com/

Thank you for your attention

http://solution4future.com/