pydata - consumindo e publicando web apis com python
TRANSCRIPT
![Page 1: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/1.jpg)
http://brunorocha.org *@rochaCbruno
Disclaimer:
O conteúdo dessa palestra representa a opinião pessoal do palestrante e não está de nenhuma forma relacionado oficialmente com a Red Hat Inc.
Bruno RochaQuality Engineer
![Page 2: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/2.jpg)
Episódio toda segunda-feira às 11 horas!
http://castalio.info
![Page 3: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/3.jpg)
web APIsconsumindo e publicando
Com Python
![Page 4: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/4.jpg)
1. O que são APIs?2. Consumindo web APIs3. O que fazer com os dados?4. Publicando web APIs
![Page 5: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/5.jpg)
API? Application Programming Interface
GUI CLI API
MUI
App A
App B
![Page 6: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/6.jpg)
API é só web?API
App A
App Bdef function_a(argument1, argument2):
...
def function_b(argument1, argument2):
...
return function_a('value1', 'value2')
API
API
Tipos de APIs: (baseadas em seus protocolos)- Memory Pointers- Object orientation members- Remote Procedure Call - Telnet- EDI (Eletronic Data Interchange)- Serial Com- Parallel Com
- Universal Serial Bus- ftp- Ssh- Gopher, POP, SMTP, IMAP...- http- …. (muitas outras)
![Page 7: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/7.jpg)
Especificações, padrões e protocolos- RPC / CORBA (Remote Procedure Call - Xerox 1976)- EDI (Eletronic Data Exchange - University of Minnesota in 1991)- Windows COMs (Component Object Model 1993)- HTTP ( ietf 1990)- SOA (Service-Oriented Architecture - Gartner Group 1996)- XML + XML-RPC (Microsoft 1998)- SOAP (W3C - 2000)- REST (Roy Fielding 2000)- AMQP (JPMorgan 2003)- Web Services / WS protocol (W3C 2005)- JSON + JSON RPC (Douglas Crockford - 2006)- Thrift, MQ, ProtoBuf (Apache, Google, Pivotal etc…. - WIP)- Microservices - (Martin Fowler - 2014)- GraphQL - (Facebook 2015)
40 anos!
![Page 8: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/8.jpg)
O que mudou em 20 anos?- RPC / CORBA (Remote Procedure Call - Xerox 1976)- EDI (Eletronic Data Exchange - University of Minnesota in 1991)- Windows COMs (Component Object Model 1993)- HTTP ( ietf 1990)- SOA (Service-Oriented Architecture - Gartner Group 1996)- XML + XML-RPC (Microsoft 1998)- SOAP (W3C - 2000)- REST (Roy Fielding 2000)- AMQP (JPMorgan 2003)- Web Services / WS protocol (W3C 2005)- JSON + JSON RPC (Douglas Crockford - 2006)- Thrift, MQ, ProtoBuf (Apache, Google, Pivotal etc…. - WIP)- Microservices - (Martin Fowler - 2014)- GraphQL - (Facebook 2015)
20 anos!
The Internet Days
![Page 9: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/9.jpg)
O que é uma API web (http)?Interface de comunicação baseada nos verbos HTTP com retorno de dados padronizado em formatos largamente adotados como JSON e XML etc.
VERBO CRRUD Retorno esperado
POST Create 201 (created), 409, 50x
GET Read 200 (Ok), 204, 404, 500
PUT Replace 200, 201, 409, 404, 50x
PATCH Update 200, 201, 409, 404, 50x
DELETE Delete 200, 404, 50x
http://www.restapitutorial.com/lessons/httpmethods.html
![Page 10: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/10.jpg)
O que é REST?O que faz uma web API ser REST é a concordancia com o padrão HATEOAS, ou seja, é a maneira como o fluxo de navegação e estado entre os recursos é arquitetada, em uma API Restfull de verdade além dos dados em si também trafegam informações semanticas, metadados de estado e localização dos recursos.
HyperMediadefine oprotocoloe o estado!
![Page 11: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/11.jpg)
GraphQL - (REST vai morrer?) http://graphql.org - http://graphene-python.org/
![Page 12: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/12.jpg)
Consumindo WEB APIsAPI100% RESTNão contém XML
![Page 13: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/13.jpg)
$ curl -XPOST http://localhost:5000/product -d “{‘id’: 5, ‘name’: ‘foo’, ‘category’: ‘bar’}”….Response 201 Created{ “msg”: “Produto inserido com sucesso”, “product”: “/product/5”}
Cliente HTTP*Programa capaz de se comunicar através dos verbos HTTP (GET, POST, PUT….)
![Page 14: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/14.jpg)
https://httpie.org/
● Colorized and formatted terminal output
● Built-in JSON support
● Forms and file uploads
● HTTPS, proxies, and authentication
● Extensions
● Linux, Mac OSX, and Windows support
● And more…
![Page 15: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/15.jpg)
$ pip install requests
>>> r = requests.get('https://api.github.com/user', auth=('user', 'pass'))
>>> r.status_code
200
>>> r.headers['content-type']
'application/json; charset=utf8'
>>> r.encoding
'utf-8'
>>> r.text
u'{"type":"User"...'
>>> r.json()
{u'private_gists': 419, u'total_private_repos': 77, ...}
![Page 16: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/16.jpg)
![Page 17: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/17.jpg)
HTTPbin
![Page 18: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/18.jpg)
Não reinvente a roda!
![Page 19: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/19.jpg)
![Page 20: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/20.jpg)
Não encontrei meu wrapper e agora?
![Page 21: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/21.jpg)
![Page 22: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/22.jpg)
API Wrapper Frameworkfrom tapioca import TapiocaAdapter
from tapioca.serializers import SimpleSerializer
class MyAPISerializer(SimpleSerializer):
def serialize_datetime(self, data):
return data.isoformat()
class MyAPIAdapter(TapiocaAdapter):
serializer_class = MyAPISerializer
...
http://tapioca-wrapper.readthedocs.io
![Page 23: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/23.jpg)
O que fazer com os dados?
![Page 24: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/24.jpg)
Easy Data Retrieval++- Es Engine https://github.com/catholabs/esengine - Py Mongo https://github.com/mongodb/mongo-python-driver- Monary https://monary.readthedocs.io - pyArango https://github.com/tariqdaouda/pyArango
![Page 25: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/25.jpg)
Converter?
![Page 26: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/26.jpg)
Converter, serializar e transformar?
import requests
import rows
# Get data from Portuguese Wikipedia
city_list_url = 'https://pt.wikipedia.org/wiki/Lista_de_munic%C3%ADpios_do_Brasil'
response = requests.get(city_list_url)
html = response.content
# Extract desired data using XPath
cities = rows.import_from_xpath(
BytesIO(html),
rows_xpath='//table/tr/td/ul/li',
fields_xpath=OrderedDict([('name', './/text()'),
('link', './/a/@href')]))
# Rows supports other data sources
- csv, txt, json, html, parquet, xpath, xls, xlsx, SQlite, ODS +
github.com/turicas/rows
![Page 27: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/27.jpg)
Publicando WEB APIs
![Page 28: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/28.jpg)
Flask para desenvolvimento de APIsUberRedditPinterestTwilioNetflixMailgun...
![Page 29: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/29.jpg)
@app.route("/hello")
def hello():
return jsonify({"hello": "world"})
Flask é extremamente simples e intuitivo para criação de APIs simples.
… Mas há muito mais do que JSON em boas APIs.
![Page 30: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/30.jpg)
Caracteristicas de uma boa web API1. CAP
a. Consistencia(c) imediata é sacrificada em troca de Disponibilidade(a) e Particionamento(p)!
2. Escalávela. MicroServices (fila de processamento)b. Cachingc. Async Response
3. Statelessa. REST/HATEOAS/OAuth
4. Concisaa. Versionamentob. Formato de dadosc. Documentação
5. Seguraa. Throtling b. Autenticação
![Page 31: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/31.jpg)
Tem framework para fazer as
mágica pra nóis?
MethodViews
Flask-Classy
Flask RestFull
Flask RestPlus
Flask-Potion
Eve
Conexxion
Flasgger
![Page 32: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/32.jpg)
1 from flask import MethodView, jsonify2 3 class ProductAPI(MethodView):4 def get(self):5 return jsonify(....) 6 def post(self):7 return jsonify(....) 8 def delete(self):9 return jsonify(....) 1011 app.add_url_rule(‘/product’, ProductAPI.as_view(‘product’))
![Page 33: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/33.jpg)
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_potion import Api, ModelResource
app = Flask(__name__)
db = SQLAlchemy(app)
class Book(db.Model):
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(),
nullable=False)
year_published = db.Column(db.Integer)
db.create_all()
class BookResource(ModelResource):
class Meta:
model = Book
api = Api(app)
api.add_resource(BookResource)
![Page 34: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/34.jpg)
from eve import Eve
app = Eve()
app.config.MONGO_HOST = “localhost”
app.run()
![Page 35: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/35.jpg)
PREÇO:ALTO ACOPLAMENTO
![Page 36: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/36.jpg)
BAIXO ACOPLAMENTO
![Page 37: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/37.jpg)
Escalabilidade 1. Adotar arquitetura de Micro Serviços e Async Message Queuesa. Nameko. Kafka, Celery, *MQ
2. Cachinga. Redis, Memcached, Elasticsearch*
3. Distribuiçãoa. API Gateway (ver Nginx OpenResty ;)
4. Elasticidadea. AWS, OpenShift, Kubernetes
*Materialized path
![Page 38: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/38.jpg)
from flask import Flask
from flask import jsonify
app = Flask()
@app.route("/")
def test():
return jsonify({"hello": "world"})
app.run(host="0.0.0.0", port=8000)
![Page 39: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/39.jpg)
from sanic import Sanic
from sanic.response import json
app = Sanic()
@app.route("/")
async def test(request):
return json({"hello": "world"})
app.run(host="0.0.0.0", port=8000)
![Page 40: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/40.jpg)
from sanic import Sanic as Flask
from sanic.response import json as jsonify
app = Flask()
@app.route("/")
async def test(request):
return jsonify({"hello": "world"})
app.run(host="0.0.0.0", port=8000)
from flask import Flask
from flask import jsonify
app = Flask()
@app.route("/")
def test():
return jsonify({"hello": "world"})
app.run(host="0.0.0.0", port=8000)
Flask (sync, blocking) Sanic (asyncio)
Otimização prematura é a raiz de todo o mal!
Pode ser que seu projeto não precise de AsyncIOAntes de partir para o Sanic + AsyncIO tente:
- Arquitetura distribuida (+workers)- Gunicorn, twisted, gevent- Async microservices (e MQs)
![Page 41: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/41.jpg)
![Page 42: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/42.jpg)
import nameko
class MyService(nameko.Service):
@nameko.rpc
def my_resource():
data = # expensive computation
return data
import nameko
from flask import Flask
from flask import jsonify
app = Flask()
@app.route("/")
def test():
with nameko.rpc(‘my-rabbit’) as rpc:
result = rpc.my_resource()
return jsonify(result)
# or return only the “future” as task id
MicroservicesNameko + rabbit MQ (microserviço)
+ $ gunicorn -w 4 -b 127.0.0.1:4000 myproject:app
+ $ twistd web --wsgi myproject.app
+ $ gevent.wsgi.WSGIServer(('', 5000), app)
Flask API
+ $ nameko run my_service -c 10
+ $ docker run -d --hostname my-rabbit --name my-rabbit -p 15672:15672 -p 5672:5672 rabbitmq:3-management
![Page 43: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/43.jpg)
API é Front-End!
● Performance● Design● Usabilidade● User Experience● Java Script Compat● ...
![Page 44: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/44.jpg)
$ curl -XPOST http://localhost:5000/product -d “{‘id’: 5, ‘name’: ‘foo’, ‘category’: ‘bar’}”….Response 201 Created{ “msg”: “Produto inserido com sucesso”, “product”: “/product/5”}
![Page 45: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/45.jpg)
Swagger UI?- Framework para
live-documentation (API Playground)
- Baseado na especificação JSONSchema e OpenAPI.
- Utiliza YAML e JSON
![Page 46: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/46.jpg)
Slate- Framework para
DocumentaçãoComercial de API
- Baseado na especificação JSONSchemaE Jquery.
- Utiliza Markdown
![Page 47: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/47.jpg)
Flask + Swagger = + slate
![Page 48: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/48.jpg)
$ pip install flasgger
![Page 49: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/49.jpg)
@app.route('/product', methods=['POST'])def post_product(): data = request.json product.create(**data) result = {'msg': 'Produto inserido com sucesso!', ‘product’: url_for(‘get_product’, product.id)} …. return jsonify(**result), 201
$ curl -XPOST http://localhost:5000/product -d “{‘id’: 5, ‘name’: ‘foo’, ‘category’: ‘bar’}”….Response 201 Created{ “msg”: “Produto inserido com sucesso”, “product”: “/product/5”}
![Page 50: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/50.jpg)
![Page 51: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/51.jpg)
from flasgger import Swagger, validateSwagger(app)
@app.route('/product', methods=['POST'])def post_product(): “””Create a new product parameters: - in: body schema: required: [name, category, id] properties: name: type: string description: The name of the product “”” data = request.json validate(data) return jsonify(result={'msg': 'Deu Certo!'}), 201
![Page 52: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/52.jpg)
from flasgger import Swagger, validate, swag_fromSwagger(app)
@app.route('/product', methods=['POST'])@swag_from(‘post_product.yml’)def post_product(): data = request.json validate(data) return jsonify(result={'msg': 'Deu Certo!'}), 201
![Page 53: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/53.jpg)
from flasgger import Swagger, validateSwagger(app)
@app.route('/product', methods=['POST'])@swag_from({‘parameters’: …, ‘definitions’: ...})def post_product(): data = request.json validate(data) return jsonify(result={'msg': 'Deu Certo!'}), 201
![Page 54: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/54.jpg)
from flasgger import Swagger, SwaggerViewfrom flasgger import validate, Schema, fields
class Product(Schema): name = fields.Str(required=True) category = fields.Nested(Category) id = fields.Str(required=True)
class ProductAPIView(SwaggerView): parameters = Product validation = True def post(self): return jsonify(result={'msg': 'Deu Certo!'}), 201
![Page 55: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/55.jpg)
![Page 56: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/56.jpg)
Tutorial de Flaskhttp://bit.ly/whattheflask
DEMO do Flasggerhttp://flasgger-rochacbruno.rhcloud.com/
Tutorial de Flasggerhttp://brunorocha.org/python/flask/flasgger-api-playgrou
nd-with-flask-and-swagger-ui.html
API com Flasgger baseada em MicroServiceshttp://brunorocha.org/python/microservices-with-python-
rabbitmq-and-nameko.html
![Page 57: PyData - Consumindo e publicando web APIs com Python](https://reader034.vdocument.in/reader034/viewer/2022051101/58f9b2ff760da3da068bd172/html5/thumbnails/57.jpg)
http://brunorocha.org @rochacbruno (everywhere)
Disclaimer:
O conteúdo dessa palestra representa a opinião pessoal do palestrante e não está de nenhuma forma relacionado oficialmente com a Red Hat Inc.
Bruno RochaQuality Engineer Obrigado!!!
(perguntas?)