harnessing the power of python in arcgis using the conda … · 2017. 7. 11. · harnessing the...
TRANSCRIPT
HarnessingthePowerofPythoninArcGISUsingtheCondaDistributionShaunWalbridge
https://github.com/scw/conda-uc-2017
HighQualityPDF(2MB)
Python
WhyPython?Accessiblefornew-comers,andthe
Extensivepackagecollection(56thousandon ),broaduser-baseStronggluelanguageusedtobindtogethermanyenvironments,bothopensourceandcommercialOpensourcewithliberallicense—dowhatyouwant
mosttaughtfirstlanguageinUSuniversites
PyPI
WhyPython?Inthebox:
TheSciPyStack(NumPy,SciPy,Pandas,matplotlib,sympy)
xlrd,netCDF4,requests,PyPDF,pytz
ScientificProgrammingwiththeSciPyStack
GettingtoPackages
WhyPackages?Softwareiscomposedofmanysmallercomponents,oftencalledpackagesorlibraries.It’softenbettertoreusecodethatsolvesaproblemwellratherthanrecreatingitBut,sharingcodeisahardproblem.Doyouhavethesamepackagesofthesameversionsasthedeveloperdid?
PackageManagementforPythonWhynotpip,wheels,virtualenvs?
Don’thandletheharderproblemofsystemdependencies,consideredoutofscopebyPythonpackagers—doesitendupinsite-packages?Packagedevs:OnOSXandLinux,‘easy’togetthedeps!Useasystempackagemanager(e.g.apt,brew,yum)andtheincluded
compiler(e.g.clang,gcc).It’sstillnoteasytomakereproduciblebuilds,andwhataboutWindows?
WhataboutWindows?WeareparticularlystuckonWindowswhichlacksbroadlyusedpackagemanagementOnlydevelopershaveaCcompilerontheirmachineAhardproblem
EnterConda
WhyConda?
ScientificPythoncommunityidentifiedthattherewasagapnotbeingaddressedbythecorePythoninfrastructure,limitingtheirabilitytogetpackagesintothehandsofusers
Industrystandardbuiltbypeoplewhocareaboutthisspace—ContinuumAnalytics
WhyConda?
Itsolvesahardproblem:
Handlesdependenciesformanylanguages(C,C++,RandofcoursePython)BuiltforPythonfirst,butitreallysolvesamuchbroaderinfrastructuralissue.
E.g.UseitforisolatedRinstallations,whichareusuallytricky
Conda
Cross-platform:simplydeveloprecipesforbuildingandinstallingsoftwareonLinux,OSXandWindows.Opensource:Esriisusingit,youcanuseitinyourownprojectsforothercontexts
Whatcanitinstall?Notjustscientificpackages.Itcanhelpwith:
GUItoolkits(PyQt,TKinter)C++Libraries(Boost)IDEs(Spyder,Juptyer)
CondainPro
Environments—AcollectionofpackagesandPythoninstalliscalledanenvironment,thebuildingblockformanagingPythonwithCondaFlexiblymakechangeswithoutaffectinginstalledsoftwareCancreatemultipleenvironmentsandswitchseamlessly
Requirements—includeexplicitstateinformation,notjustthepackagename.
CondaBasicsActivatingenvironments,acoupleways:
UsetheshortcutsincludedinProManuallyactivatetheenvironment:
cd C:\ArcGIS\bin\Python\Scripts activate arcgispro-py3
CondaBasics
Condainfoisthestartingpoint—ittellsyouthestateoftheenvironment.
conda --help
conda info
CondaBasicsDemo
CondaBasicsOnceyou’reinanenvironmentgetdetailswithinfo:
Condainfoisthestartingpoint—ittellsyouthestateoftheenvironment.
conda info
CondaBasicsconda info
Current conda install:
platform : win-64 conda version : 4.0.6 conda-build version : not installed python version : 3.5.1.final.0 requests version : 2.9.1 root environment : C:\ArcGIS\bin\Python (writable) default environment : C:\ArcGIS\bin\Python\envs\arcgispro-py3 envs directories : C:\ArcGIS\bin\Python\envs package cache : C:\ArcGIS\bin\Python\pkgs channel URLs : https://conda.anaconda.org/esri/win-64/ https://conda.anaconda.org/esri/noarch/ https://repo.continuum.io/pkgs/free/win-64/ https://repo.continuum.io/pkgs/free/noarch/ config file : C:\ArcGIS\bin\Python\.condarc
CondaBasicsconda list
# packages in environment at C:\ArcGIS\bin\Python\envs\arcgispro-py3:##colorama 0.3.7 py35_0 defaultscycler 0.10.0 py35_0 defaultsfuture 0.15.2 py35_0 defaultsmatplotlib 1.5.3 np111py35_0e [arcgispro] esrimpmath 0.19 py35_1 defaultsnetcdf4 1.2.4 py35_0e [arcgispro] esrinose 1.3.7 py35_1 defaultsnumexpr 2.6.1 np111py35_0e [arcgispro] esrinumpy 1.11.2 py35_0e [arcgispro] esripandas 0.19.0 np111py35_0 defaultspip 8.1.2 py35_0 defaultspy 1.4.31 py35_0 defaultspyparsing 2.1.4 py35_0 defaultspypdf2 1.26.0 py_0 esripytest 2.9.2 py35_0 defaultspython 3.5.2 0 defaultspython-dateutil 2.5.3 py35_0 defaultspytz 2016.6.1 py35_0 defaultsrequests 2.11.1 py35_0 defaultsscipy 0.18.1 np111py35_0e [arcgispro] esrisetuptools 27.2.0 py35_1 defaultssympy 1.0 py35_0 defaultswheel 0.29.0 py35_0 defaults
CondaBasicsCreatingnewenvironments:
Afewdifferentways.Canmanuallyspecifythedependencies:
Canalsouseafilewhichincludesallthedependencies:
Thesecancontainexplcitinformationaboutchannels,toensurethatthenewenvironmentpreciselymatchestherequirements.
conda create --name my_env python=3.5 numpy flask dask
conda create --name my_env --file my_sweet_depends.txt
Condavs…Name Means Will
Ship?
Conda Thecommanditself ✓Miniconda AminimumsetofPythonpackagesto
buildandrunConda.✓
Anaconda Adistribution200+packagesbuiltwithConda
AnacondaServer
Hostthefullinfrastructureinternally
DeeperDive
Demo:CondaPackage
MultiplePythonsCurrently:
Platform Pythonversion
Desktop Python2.7(2.7.13)
Pro Python3.5(3.5.3)
MultiplePythonsUpgradecode?
Doit!Youcansupport2+3withoutthatmuchworkStillneedtochangearcpy.mappingtoarcpy.mpwhenmovingfrom
DesktoptoPro,butnoPythonlanguagelevelchangesneeded....
But…thiscanbecostly.Formanyorganizations,asignificantburden,evenifthelanguagechangesarerelativelysmall.MultiplePythonsis
asolutiontothis.
PythonmigrationforArcGISPro
ChallengesHavetomakesureyou’rerunningtherightPython(whathappens
whenyoutypepythonatthecommandline?)
WorkingtomakethiseasyaspossibleIt’llbeeasytotellinapp
Isolatedinstallationfixesavarietyofissues
Requiressomeusereducationoverthe“onlyonePythononthebox”model
Upgrades,whathappens?
WhatDoIGetOutoftheBox?CondacommandandaCondarootPythoninstallNewmodules(e.g.requests)CondaenvironmentwithalloftheArcGISProdependenciesasCondapackages
HowcanIusethis?ArcGISshipswiththeSciPystack—powerfulandoutoftheboxinallproductsCondacommandandaCondarootPythoninstallNewmodules(e.g.requests),environmentwithPro
PythonPackageManagerinPro...
Getpackages,expandyourpossibilityspace
Packageyourwork:thisisanopportunitytodistributeit
WhereCanIRunThis?
ArcGISPro1.3:firstwithcondaArcGISPro2.0:
PythonPackageManagerPython3.5withcurrentpackageset
ArcGISEnterprise10.5ArcGISAPIforPython
from future import *EffectivelymanagecomplexsoftwaredependencieswithConda.Thousandsofpackagesexisttoday,canintegrateitintoyourorganization’sneeds.
Closing
ThanksEsriCondaTeam:
ContinuumAnalyticsforcreatingandopensourcingConda
RateThisSessionPleasetakeoursurvey,findsessioninappandprovidereview