py conkr 20150829_docker-python

D O C K E R를 이용하여 P Y T H O N 개발 환경을 빠르게 구성하고 , 백앤드 서비스를 탐색하는 기술 , 실행 환경

P Y C O N K R - 2 0 1 5 B Y U N G W O O K A H N

1

Who am I

ByungWook Ahn G+ : https://plus.google.com/+EricAhns

device driver(windows, linux) experienced media streaming experienced CDN experienced docker experienced Platform Architecture Team, SKPlanet

D O C K E R C O N TA I N E R D O C K E R - M A C H I N E D O C K E R - C O M P O S E S A M P L E A P P A R C H I T E C T U R E T E S T E N V I R O N M E N T P E R F O R M A N C E

TODAY…

https://www.docker.com/ B U I L D , S H I P, R U N

A n o p e n p l a t f o r m f o r d i s t r i b u t e d a p p l i c a t i o n s f o r d e v e l o p e r s a n d s y s a d m i n s

DOCKER

C O N TA I N E R

F R O M u b u n t u R U N \ a p t - g e t u p d a t e & & \ a p t - g e t i n s t a l l - y p y t h o n p y t h o n - d e v p y t h o n -p i p p y t h o n - v i r t u a l e n v & & \ r m - r f / v a r / l i b / a p t / l i s t s / *

$ c a t D o c k e r f i l e

$ docker build -t=“mypython” . <- image 생성 ( 현재 디렉토리에 있는

Dockerfile을 참조)

$ docker images <- Docker Host의 image 확인 REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE mypython latest ecc926c495f8 7 seconds ago 373.6 MB ubuntu latest 91e54dfb1179 4 days ago 188.4 MB mysql latest c45e4ba02f47 12 days ago 283.8 MB python 2.7 e1857ee1f3b5 5 weeks ago 674.4 MB nginx latest 6886fb5a9b8d 5 weeks ago 132.9 MB

$ docker run -it mypython python <- container 실행 Python 2.7.6 (default, Jun 22 2015, 17:58:13) [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>

$ docker ps -a <= docker host의 containers print CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES b596713d6934 mypython "python" 2 seconds ago Exited (0) 2 seconds ago clever_hypatia

Docker Host example $ top

docker container : ubuntu 2ea container : ubuntu(nginx)

D O C K E R - M A C H I N E

VirtualBox vmware Microsoft Hyper-V SOFTLAYER openstack Microsoft Azure AWS rackspace DigitalOcean

create & removerun

➜ 0 2 - S R C d o c k e r- m a c h i n e l s N A M E A C T I V E D R I V E R S TAT E U R L S WA R M d e f a u l t v i r t u a l b o x S t o p p e d d e v v i r t u a l b o x S t o p p e d p y c o n k r v i r t u a l b o x S t o p p e d p y c o n k r 0 2 v i r t u a l b o x S t o p p e d p y c o n k r 0 3 v i r t u a l b o x S t o p p e d p y c o n k r 0 4 v i r t u a l b o x S t o p p e d p y c o n k r 0 5 v i r t u a l b o x S t o p p e d

DOCKER-COMPOSE

W E B

D B

$ docker-compose up -d

C O N F I G R AT I O N ? = > YA M L

$ cat docker-compose.yml db: image: mysql environment: - MYSQL_ROOT_PASSWORD=hellopython - MYSQL_DATABASE=shopping_db ports: - "3306:3306" web: build: ./web_api/ command: python manage.py runserver 0.0.0.0:8000 volumes: - .:/web_api ports: - "8000:8000" links: - db:db

Sample Application

A N G U L A R J S O N

P Y T H O N R E S T A P I


D BL O A D

B A L A N C E RL O A D

B A L A N C E R





D BL O A D B A L A N C E R

L O A D B A L A C E R


J AVA R E S T A P I

J AVA R E S T A P I

G O L A N G R E S T A P I

G O L A N G R E S T A P I

R O U T I N G

R E D I SRestful API…

T E S T E N V I R O N M E N T

1 2

Traffic Gen. Server Dell R620 2.0 GHz/2P/12C 32G

Docker Host. HP DL380G7 2.13GHz/1P/4C 24GB

load testing tool? nGrinder/Apache ab/locust.io

T E S T E N V I R O N M E N T$ docker inspect web1 [{ "AppArmorProfile": "", "Args": [ "-g", "daemon off;" ], "Config": { "AttachStderr": false, "AttachStdin": false, "AttachStdout": false, "Cmd": [ "nginx", "-g", "daemon off;" ], "CpuShares": 1024, => "Cpuset": "", "Domainname": “", … …

$ docker inspect web2 [{ "AppArmorProfile": "", "Args": [ "-g", "daemon off;" ], "Config": { "AttachStderr": false, "AttachStdin": false, "AttachStdout": false, "Cmd": [ "nginx", "-g", "daemon off;" ], "CpuShares": 512, => "Cpuset": "", "Domainname": “", … …

$ docker inspect web3 [{ "AppArmorProfile": "", "Args": [ "-g", "daemon off;" ], "Config": { "AttachStderr": false, "AttachStdin": false, "AttachStdout": false, "Cmd": [ "nginx", "-g", "daemon off;" ], "CpuShares": 512, => "Cpuset": "", "Domainname": “", … …

C P U :5 0 %

C P U :2 5 %

C P U :2 5 %

T E S T C A S E

web_api_case_01 Django(1.8.3) + djangodb + MySQL 5.6.23

web_api_case_02 Django(1.8.3) + No DB operation

web_api_case_03 Django(1.8.3) + djangodb + PostgreSQL 9.4.4

web_api_case_04 Django(1.8.3) + djangodb + (django-cache-machine) + MySQL 5.6.23

web_api_case_demo load balancer -> web_api_case_01

http://web_api:8000/shoppling/ APIs를 가진 예로 :

N G R I N D E R / A PA C H E A B / L O C U S T. I O

nGrinder : agent 컨테이너 5개 생성하여 테스트

// controller $ docker run -d -v ~/.ngrinder:/root/.ngrinder --name ngrinder -p 80:80 -p 16001:16001 -p 12000-12009:12000-12009 ngrinder/controller:3.3

// agent $ docker run -d -e 'CONTROLLER_ADDR=ngrinder:80' --link ngrinder:ngrinder ngrinder/agent:3.3

nginx : 1.9.2 index.html, 600 bytes

D E M O - C A S E 1 : S A M P L E A P Psettings.py INSTALLED_APPS = ( 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'shopping' ) DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'shopping_db', 'USER': 'root', 'PASSWORD': 'hellopython', 'HOST': 'db', 'PORT': 3306, } }

S A M P L E A P P

models.py from django.db import models

class Shopping(models.Model): product_code = models.TextField() product_title = models.CharField(max_length=100, verbose_name = "Product name") product_price = models.IntegerField(default='0', verbose_name = "Product price")

class Meta: verbose_name = "Product list" verbose_name_plural = "Favorite lists"

S A M P L E A P P

// views.py def shopping_list(request): """ List all code shoppinglists, or create a new shopping. """ if request.method == 'GET': shoppinglists = Shopping.objects.all() serializer = ShoppingSerializer(shoppinglists, many=True) return JSONResponse(serializer.data)

// serializers.py from rest_framework import serializers from shopping.models import Shopping

class ShoppingSerializer(serializers.ModelSerializer): class Meta: model = Shopping fields = ('product_code', 'product_title', 'product_price')

urls.py urlpatterns = [ url(r'^shopping/$', views.shopping_list) ]

S A M P L E A P P

db: image: mysql environment: - MYSQL_ROOT_PASSWORD=hellopython - MYSQL_DATABASE=shopping_db ports: - "3306:3306" web: build: ./web_api/ command: python manage.py runserver 0.0.0.0:8000 volumes: - .:/web_api ports: - "8000:8000" links: - db:db

demo : case1

P E R F O R M A N C E : C A S E 1


M Y S Q LL O A D T O O L

D E M O : C A S E 2

views.py def shopping_list(request): """ List all code shoppinglists, or create a new shopping. """ if request.method == 'GET': return JSONResponse({"product_code":"1005","product_title":"BILLIE GOES TO TOWN SHOPPER","product_price":28})


P Y T H O N R E S T A P IL O A D T O O L

N G I N X - G U N I C O R N

web: restart: always build: ./web expose: - "8000" volumes: - /usr/src/app/static env_file: .env command: /usr/local/bin/gunicorn docker_django.wsgi:application -w 12 -b :8000

nginx: restart: always build: ./nginx/ ports: - "80:80" volumes: - /www/static volumes_from: - web links: - web:web

N G I N X - G U N I C O R Nserver {

listen 80; server_name example.org; charset utf-8;

location /static { alias /usr/src/app/static; }

location / { proxy_pass http://web:8000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; }

}

N G I N X - G U N I C O R Ngunicorn docker_django.wsgi:application -w 2

gunicorn docker_django.wsgi:application -w 12

D E M O : C A S E 3

DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'shopping_db', 'USER': 'postgres', 'PASSWORD': 'hellopython', 'HOST': 'db', 'PORT': 5432, } }

web: build: ./web_api/ command: python manage.py runserver 0.0.0.0:8000 volumes: - .:/web_api ports: - "8000:8000" links: - db:db db: image: postgres ports: - "5432:5432" environment: - DB_USER=postgres - DB_PASS=hellopython - DB_NAME=shopping_db

D E M O : C A S E 3

D E M O : C A S E 4

settings.py CACHE = { 'default':{ 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache', 'LOCATION': 'memcached:11211' } }

views.py def shopping_list(request): """ List all code shoppinglists, or create a new shopping. """ if request.method == 'GET': data = cache.get('shoppinglists') if data is None: shoppinglists = Shopping.objects.all() serializer = ShoppingSerializer(shoppinglists, many=True) data = serializer.data cache.set('shoppinglists', data, 60) # seconds return JSONResponse(data)

db: image: mysql environment: - MYSQL_ROOT_PASSWORD=hellopython - MYSQL_DATABASE=shopping_db ports: - "3306:3306"

memcached: image: memcached:1.4 ports: - "11211:11211"

web: build: ./web_api/ command: python manage.py runserver 0.0.0.0:8000 volumes: - .:/web_api ports: - "8000:8000" links: - db:db - memcached:memcached

D E M O : C A S E 4



M Y S Q LL O A D T O O L

M E M C A C H E D

D E M O


M Y S Q L

M E M C A C H E D

W E B PA G E

H A P R O X Y

W E B PA G E

W E B PA G E

H A P R O X YP Y T H O N R E S T A P I


web: restart: always build: ./web_api/ command: python manage.py runserver 0.0.0.0:8000 volumes: - .:/web_api expose: - "8000" links: - db - memcached

weblb: restart: always image: tutum/haproxy links: - web ports: - "80:80" environment: - BACKEND_PORT=8000

db: image: mysql environment: - MYSQL_ROOT_PASSWORD=hellopython - MYSQL_DATABASE=shopping_db ports: - "3306"

memcached: image: memcached:1.4 ports: - "11211"

demo-composer.yml - haproxy - mysql - memcached - python django restframeowork

$ docker-compose build . $ docker-compose scale web=3 $ docker-compose up -d —no-recreate

P E R F O R M A N C E : D E M O

Thanks : Q&A

py conkr 20150829_docker-python

Internet