Python Workshop
Beau Williamson
Python Workshop-rev4 1
Housekeeping• Please silence cellphones
– If you must take a call, please step outside
• If we hear your cellphone go off, . . . . . . you have to buy beer for everyone after the meeting.
Python Workshop 2
Opening Statement• I am not an Expert on Python Programming
– You may ask me questions at any time but . . .• There are no bonus points for “stumping” the presenter
– Other people here my have some/more experience with Python• If so, please speak up• We want to benefit from your knowledge too
Python Workshop 3
Equipment Prerequisites• Windows 7 or 8.1 or 10 Laptop• Intel 64-bit processors are preferred.• 8 GB or more of RAM• Browser Requirements: Chrome• Administrator Rights on Laptop
– Or at least ability to install new software• Software Installers
– Links listed in March 2018 Newsletter• If you didn’t bother to pre-stage those files, look over someone’s shoulder that
did
Python Workshop 4
Objectives• Setup a quality Python IDE programming environment
on your Laptop • Understand how to leverage features of PyCharm to
streamline and improve Python scripting• Accessing Python Library packages • Accessing GitHub Python packages
Python Workshop 5
Non–Objectives• Teach you the fundamentals of programming
– This is not intended to be your first class in or exposure to programming• (Although for some of you it might just be)
– Ideally you should already know some fundamentals of programming
• Teach you programming in Python– That’s a topic that just can’t be covered in two hours– There are a LOT of good resources to learn programming in Python
Python Workshop 6
Learning Python – Online Courses• Some Basic Python Programming Courses
– Coursera Python Programming Course(s)– Udacity Python Programming Course– Code Academy Python Programming Course
Python Workshop 7
Learning Python – Books
Learn Python the Hard WayAuthor: Zed A. Shaw
– Online Html version• https://learnpythonthehardway.org/book/
– Highly recommended– Fun to ready. Easy to follow.
DFWCUG Python Study Group– Any volunteers to start one?
Python Workshop 8
Workshop Agenda• Lesson 1: Basic Python Tools• Lesson 2: PyCharm IDE• Lesson 3: Python Libraries & Resources• Lesson 4: Accessing Network Devices• Lesson 5: YDK-py
Python Workshop 9
LESSON 1Basic Python Tools
Python Workshop 10
Lesson 1: Basic Python Tools• Agenda
– Installing Notepad++ for Windows– Installing Python 2.7 for Windows– Introduction to IDLE– Using pip to install/manage Python Packages from PyPI– Installing Python 3.6 for Windows– Using py to install/manage Python Packages for 2.7 & 3.6– Installing GIT for Windows– Cloning Packages from GIT repositories to your laptop– Installing cloned Python packages– Other Recommended Tools
Python Workshop 11
Installing NotePad++• Notepad++
– A source code editor and Notepad replacement– Supports several languages including Python– https://notepad-plus-plus.org/download/
• Download the 64-bit x64 version
Python Workshop 12
Installing NotePad++
Python Workshop 13
• Take all the Defaults
Installing NotePad++• Install the Plugin Manager
https://github.com/bruderstein/nppPluginManager/releases– Copy PluginManager.dll file to Notepad++ plugin folder
• C:\Program Files\Notepad++\plugins – Requires Admin Privs
– Restart Notepad++ (if running)
Python Workshop 14
Installing NotePad++
Python Workshop 15
Launch Plugin Manager• Plugins-->Plugin Manager-->Show Plugin
Manager• Check the “Compare” plugin• Click “Install”
Installing Python 2.7 for Windows• https://www.python.org/downloads/release/python-2713/
Python Workshop 16
Installing Python 2.7 for Windows
Python Workshop 17
Recommendation:– Install in “C:\Python27\” (default)
Take all other defaults Add Environment Variable
– From Windows Powershell type [Environment]::SetEnvironmentVariable("Path", "$env:Path;C:\Python27", "User")
– May not be necessary– Type “python” at command prompt
• If it comes up, you are good to go.
Introduction to IDLE
Python Workshop 18
From Start Menu Find “IDLE (Python GUI)”
Introduction to IDLE
Type <Tab> for Auto-Completion
Python Workshop 19
Introduction to IDLE• Type <Tab> for Auto-Completion
Python Workshop 20
Introduction to IDLE
Python Workshop 21
Using pip to install/manage Python Packageshttps://docs.python.org/2.7/installing/index.html
• List installed Packagespython -m pip list
• Search for a Packagepython -m pip search SomePackage
• Install a Packagepython -m pip install SomePackage
• Install a specific version of a Packagepython -m pip install –upgrade SomePackage==1.0.4
Python Workshop 22
Using pip to install/manage Python Packages• Install a minimum version of a Package
python -m pip uninstall SomePackage>=1.0.4
• Show information about an installed Packagepython -m pip show SomePackage
• Upgrade a Packagepython -m pip install –upgrade SomePackage
• Remove a Packagepython -m pip uninstall SomePackage
Python Workshop 23
Using pip to install/manage Python Packages• Package location
– Packages are installed in Lib\site-packages directory of each Python version
• Examples:– C:\Python27\Lib\site-packages– C:\Python36\Lib\site-packages
• PIP Documentation– https://pip.pypa.io/en/stable/
Python Workshop 24
Installing Python 3.6 for Windows
https://www.python.org/downloads
Python Workshop 26
Installing Python 3.6 for Windows
Recommendations:• Choose Customize Installation• Add Python 3.6 to your PATH
Python Workshop 27
Installing Python 3.6 for Windows
Take the default options• Click “Next”
Python Workshop 28
Installing Python 3.6 for Windows
Recommendations:• Install for all users on your Laptop• Change location to “C:\Python36”
Click “Install”
Python Workshop 29
Installing Python Packages in 2.7 & 3.6• Problem:
– pip will default to first version (2.7 or 3.6) in your %PATH% variable– Trying to install packages in both versions can be challenging
• Installing Packages the Hard Way– cd to C:\PythonXX\Scripts
– Where XX is Python version– Execute pipn.m.exe to install into version n.m of Python– Examples:
cd C:\Python36\Scriptspip3.6 install SomePackage
cd C:\Python27\Scriptspip2.7 install SomePackage
Python Workshop 30
Using py to install/manage Python Packages • Installing the Easy Way
– Use the Windows py Python launcher with the -m switch• Specify the Python version using the version switch (e.g. -3.6, -2.7)
– Example – Install SomePackage in Python3.6:py -3.6 –m pip install SomePackage
– Example – Install SomePackage in Python2.7:py -2.7 –m pip install SomePackage
Python Workshop 31
Git, Git Repositories and GitHub• Git – A Distributed Version Control System
– Allows groups of people to work on the same documents (often code) at the same time, and without stepping on each other's toes.
– Often used to publish code
• Git Repositories– Contains a directory structure of the files/documents.
• GitHub– A public set of Git Repositories– Contains many Python packages not necessarily on PyPi
Python Workshop 32
Git, Git Repositories and GitHub
Python Workshop 33
Installing Git for Windows
https://git-scm.com/download/win
Python Workshop 34
Installing Git for Windows
Launch Installer Click “Next”
Python Workshop 35
Installing Git for Windows
Recommendation:• Accept the defaults Click “Next”
Python Workshop 36
Installing Git for Windows
Recommendations:• Accept the Defaults
Click “Next”
Python Workshop 37
Installing Git for Windows
Recommendation:• Accept Defaults
Click “Next”
Python Workshop 38
Installing Git for Windows
Recommendation:• Accept Defaults
Click “Next” Add slide for OpenSSH
Python Workshop 39
Installing Git for Windows
Recommendation:• Accept Defaults
Click “Install”
Python Workshop 40
Git Basics• Download a git project and version history to local
directorygit clone url
• Download a git project to different local directory namegit clone url local-dir-path
• Examples:git clone https://github.com/adlhancock/morse.git
git clone https://github.com/adlhancock/morse.git MyMorse
Python Workshop 41
Git Basics• Check to see if a git project has updated files
cd <local git directory>git fetch
– It will say “Your branch is behind . . . ” if not up-to-date
– Otherwise it will return to command promptPython Workshop 42
PS C:\Users\bwilliam\Documents\git\kicad-library> git fetchremote: Counting objects: 273, done.remote: Total 273 (delta 152), reused 152 (delta 152), pack-reused 121Receiving objects: 100% (273/273), 1.78 MiB | 2.50 MiB/s, done.Resolving deltas: 100% (172/172), completed with 55 local objects.From https://github.com/KiCad/kicad-library
40685cc6..bb856328 master -> origin/master* [new branch] revert-1875-MCP1825S -> origin/revert-1875-MCP1825SPS C:\Users\bwilliam\Documents\git\kicad-library> git statusOn branch masterYour branch is behind 'origin/master' by 81 commits, and can be fast-forwarded.(use "git pull" to update your local branch)
Git Basics• Update your copy of a git project with the latest updates
cd MyMorsegit pull
– Will first perform a “fetch” to update status and then download changed files
Python Workshop 43
Git – Learning more
Try Git 15 minute walk-through of Git
https://try.github.io/levels/1/challenges/1
Python Workshop 44
Git – Reference
https://progit2.s3.amazonaws.com/en/2016-03-22-f3531/progit-en.1084.pdf
Excellent Book on Git Keep a copy on your bed stand
• Makes a great children’s bedtime story“Once upon a time, Fuzzy-wuzzy cloned a repository”
Python Workshop 45
Cloning your first package using GIT
Create a “git” folder under “Libraries->Documents” Right-Click on the “git” folder
and select “Git-Bash Here”
Python Workshop 46
Cloning your first package using GIT• At the bash prompt, type:
git clone https://github.com/CiscoDevNet/coding-skills-sample-code.git
Python Workshop 47
Cloning your first package using GIT• Display cloned directory structure, by typing:
cd coding-skills-sample-code
ls -la
Python Workshop 48
Installing cloned Python packages• Using Python “wheel” (.whl) file
python –m pip install somepackage.whl– Wheel files often found under a “dist” directory in the repository
• Using setup.py– Find directory in repository with “setup.py”
• Typically top-level directory– From within that directory type:python –m pip install .orpy -2.7 –m pip install .
Python Workshop 49
Other Highly Recommended Tools• Cygwin
– Large collection of GNU and Open Source tools (ls, cat, vi, ...) which provide functionality similar to a Linux distribution on Windows.
– Home Page: https://cygwin.com/– 64-bit Installer: https://cygwin.com/setup-x86_64.exe
Python Workshop 50
LESSON 2PyCharm
Python Workshop 51
Lesson 2: PyCharm• Installation, Overview and Setup of PyCharm (Videos: QT, 1, 2)
https://www.jetbrains.com/pycharm/download/#section=windows
• Running Code (Video: 3)– Interpreter Selection– Install/Manage Python Packages with PyCharm
• Coding and Code Navigation (Videos: 4, 5)– Quick Documentation
• Debugging (Video: 6)– Breakpoints, Single-Step, Watch-points
Python Workshop 52
LESSON 3Python Language Libraries & Resources
Python Workshop 53
Libraries (Bookmark these!)
• Python Tutorialhttps://docs.python.org/2.7/tutorial/index.html
– Good basic “go-to” Python Language reference– You should already know most of this but may need to refresh your memory
• Python Language Referencehttps://docs.python.org/2.7/reference/index.html#reference-index
– Guide to just about everything related to the Python Language• Syntax, Grammar, Expressions, etc.
• Python Library Referencehttps://docs.python.org/2.7/library/index.html
– Think of this as your “Python Power Tools”– Keep it under your virtual pillow
Python Workshop 54
Libraries (Bookmark these!)
• Python HOWTO’shttps://docs.python.org/2.7/howto/index.html
– Be sure to at least check out:• Logging HOWTO• Regular Expressions HOWTO• HOWTO Fetch Internet Resources Using urllib2
– Especially useful when formatting REST API calls• Argparse Tutorial
– Used to parse command line arguments
• Python FAQ’s– https://docs.python.org/2.7/faq/index.html– You may wish to check out “Python on Windows FAQ”
Python Workshop 55
Python Library Reference – A Closer Look• Keep these “Python Power Tools” handy while coding
– Built-in Types• str, set, dict and File Objects
– String Services• string – Common string operations• re – Regular Expressions operations
– Numeric and Mathematical Modules• math – Mathematical functions• random – Generate pseudo-random numbers
– Internet Data Handling• json – JSON encoder and decoder• random – Generate pseudo-random numbers
• Many others– Explore and develop your list of favorite “Python Power Tools”
Python Workshop 56
Resources• PEP8
– https://www.python.org/dev/peps/pep-0008/– Style Guide for Python Code
• PyCharm will flag your code that doesn’t follow PEP8• Get in the habit of following PEP8 coding style early in your coding
• Google– Enter “Python <some programming task> example”
• Don’t get me started on how we had to write code in the old days before Google
Python Workshop 57
LESSON 4Accessing Network Devices
Python Workshop 58
Accessing Network Devices• Paramiko• Netmiko
– (Coming DFWCUG Presentation by K. Byers)
• TextFSM• CiscoConfigParse
Python Workshop 59
ParaMiko• Overview
– A Python implementation of SSH with a whole range of supported features.• SSH• SFTP
• Pros– Extensive set of API’s covers a broad range of SSH/SFTP features & functions
• http://docs.paramiko.org/en/2.1/• Cons
– No built-in “Expect” functionality– No built-in Device prompt/response knowledge– Typically requires use of time.sleep() calls or complex prompt recognition code
Python Workshop 60
Paramiko – Simple Cisco Router Example# Use invoke_shell to establish an 'interactive
session'remote_conn = remote_conn_pre.invoke_shell()print "Interactive SSH session established"
# Strip the initial router promptoutput = remote_conn.recv(1000)
# See what we have for initial router promptprint output
# Turn off pagingdisable_paging(remote_conn)
# Now let's try to send the router a commandremote_conn.send("\n")remote_conn.send("show ip int brief\n")
# Wait for the command to completetime.sleep(2)
output = remote_conn.recv(5000)print output
import paramikoimport time
def disable_paging(remote_conn):'''Disable paging on a Cisco router'''remote_conn.send("terminal length 0\n")time.sleep(1)
# Clear the buffer on the screenoutput = remote_conn.recv(1000)return output
if __name__ == '__main__':# VARIABLES THAT NEED CHANGEDip = '1.1.1.16'username = 'testuser'password = 'password'
# Create instance of SSHClient objectremote_conn_pre = paramiko.SSHClient()# Automatically add untrusted hosts (make sure okay for security policy in your environment)remote_conn_pre.set_missing_host_key_policy(paramiko.AutoAddPolicy())
# initiate SSH connectionremote_conn_pre.connect(ip, username=username, password=password, look_for_keys=False,
allow_agent=False)
print "SSH connection established to %s" % ip
Python Workshop 61
NetMiko• Features
– Successfully establish an SSH connection to the device– Simplify the execution of show commands and the retrieval of output data– Simplify execution of configuration commands including possibly commit
actions– Do the above across a broad set of networking vendors and platforms
Python Workshop 62
Netmiko• Platform support (partial list)
Platform Device_Type
Cisco IOS cisco_ios
Cisco IOS-XE cisco_xe
Cisco ASA cisco_asa
Cisco NX-OS cisco_nxos
Cisco IOS-XR cisco_xr
Cisco WLC cisco_wlc
Arista EOS arista_eos
HP ProCurve hp_procurve
Platform Device_Type
HP Comware hp_comware
Juniper juniper
Juniper JUNOS juniper_junos
Brocade VDX brocade_vdx
Brocade brocade_nos
F5 LTM f5_ltm
A10 a10
Alcatel SROS alcatel_sros
Platform Device_Type
Linux linux
OVS_Linux ovs_linux
Extreme extreme
Fortinet fortinet
Dell Force10 dell_force10
Aruba aruba_os
Huawei Huawei
generic termserver generic_termserver
Python Workshop 63
TextFSM – Text Finite State Machine• Description (courtesy of Jason Edelman’s Blog)
TextFSM is a state machine (basically like a templating engine) that is purpose built to simplify working with regular expressions and getting structured data out of traditional network devices.
• Using TextFSM– Obtain network device output (NetMiko, Paramiko, etc.)– Process using a TextFSM template– Produces structured data.
Python Workshop 64
TextFSM Information• TextFSM Overview
– Overview of TextFSM and its Finite State Machine Template files– https://github.com/google/textfsm/wiki/TextFSM
• Code Lab– Tutorial on writing TextFSM Templates
• Covers important Value Options (Filldown, Required, List, . . .) – https://github.com/google/textfsm/wiki/Code-Lab
• CLI Tables– https://github.com/google/textfsm/wiki/Cli-Table
• Jason Edelmann’s Blog– http://jedelman.com/home/programmatic-access-to-cli-devices-with-textfsm/
Python Workshop 65
CiscoConfParse• Parses Config Files
– Uses “Parent/Child” relationships of configuration
• Documentationhttp://ciscoconfparse.readthedocs.io/en/latest/
Python Workshop 66
CiscoConfParse• Example
#!/usr/bin/env pythonfrom ciscoconfparse import CiscoConfParse
parse = CiscoConfParse("../configs/sample_01.ios")
# Return a list of all ATM interfaces and subinterfacesatm_intfs = parse.find_lines("^interface\sATM")
# Return a list of all interfaces with a certain QOS policyqos_intfs = parse.find_parents_w_child( "^interf", "service-policy" )
# Return a list of all active interfaces (i.e. not shutdown)active_intfs = parse.find_parents_wo_child( "^interf", "shutdown" )
Python Workshop 67
LESSON 5YDK-py
Python Workshop 68
YDK-py• Ok, We Got YANG Data Models, Now What?
Santiago Alvarez, NANOG 68; Dallas, TX– Slides
• https://www.nanog.org/sites/default/files/20161017_Alvarez_Ok_We_Got_v1.pdf– Video
• https://www.youtube.com/watch?v=2oqkiZ83vAA&list=PLO8DR5ZGla8hcpeEDSBNPE5OrZf70iXZg&index=36
Python Workshop 69
YDK-py• Getting Started
– Git-Hub• https://github.com/CiscoDevNet/ydk-py.git
– Documentation• http://ydk.cisco.com/py/docs/
– Cisco dCloud Lab• https://dcloud2-
rtp.cisco.com/content/demo/308588?returnPathTitleKey=content-view
Python Workshop 70