harnessing the power of python in arcgis using the conda ... · getting data science with r and...
TRANSCRIPT
HarnessingthePowerofPythoninArcGISUsingtheCondaDistributionShaunWalbridge
ClintonDow
https://github.com/scw/conda-devsummit-2017-talk
HighQualityPDF
Conda
GettingtoPackages
WhyPackages?Softwareiscomposedofmanysmallercomponents,oftencalledpackagesorlibraries.It’softenbettertoreusecodethatsolvesaproblemwellratherthanrecreatingitBut,sharingcodeisahardproblem.Doyouhavethesamepackagesofthesameversionsasthedeveloperdid?
PackageManagementforPython
Whynotpip,wheels,virtualenvs?
Don’thandletheharderproblemofsystemdependencies,consideredoutofscopebyPythonpackagers–doesitendupinsite-packages?Packagedevs:OnOSXandLinux,‘easy’togetthedeps!Useasystempackagemanager(e.g.apt,brew,yum)andtheincludedcompiler(e.g.clang,gcc).It’sstillnoteasytomakereproduciblebuilds,andwhataboutthatotherplatform?
WhyConda?
ScientificPythoncommunityidentifiedthattherewasagapnotbeingaddressedbythecorePythoninfrastructure,limitingtheirabilitytogetpackagesintothehandsofusers
Industrystandardbuiltbypeoplewhocareaboutthisspace—ContinuumAnalytics
WhyConda?
Itsolvesthehardproblem:
HandlesdependenciesformanylanguagesBuiltforPythonfirst,butitreallysolvesamuchbroaderinfrastructuralissue.
Gatewaytodatascience—scientific,analytics,integratedsoftwareecosystemfororganizations
Interlude:ReviewingReviews
ReviewingReviews
ReviewingReviewsscore=(ratings.sum(axis=1)+inverse_ratings.sum(axis=1)*5)"""104130.0103430.0103629.0103927.0103826.0103726.0104025.0104224.0103515.0"""#1035,let'sseeiftheyleftusacommentdf_conda.loc[1035]['Comments']
Therewasnorealbasicexplanationofwhatcondadoes…justapackageandenvmanager.
ReviewingReviewsIfyoudon’tagreewithsomeone,trywalkingamile
intheirshoes…
…becausethenyou’llbeamileawayfromthem,andyou’llhavetheirshoes.—TerryPratchett
Fundamentals
FundamentalsConsistentbuildingofpackages(Windows,Linux,MacOS),publicandprivatesharingCrossplatform,andcross-langauge—handlesC/C++,R,Java,Scala,JavascriptandmanymoreUltimatelyrealizedasacollectionoffilesinanarchive,andruleswhichdictatepackagedependenciesOpensource:Esriisusingit,youcanuseitinyourownprojectsforothercontexts
Environments—AcollectionofpackagesandPythoninstalliscalledanenvironment,thebuildingblockformanagingPythonwithCondaFlexiblymakechangeswithoutaffectinginstalledsoftwareCancreatemultipleenvironmentsandswitchseamlessly
Requirements—includeexplicitstateinformation,notjustthepackagename.
WheredoIgetpackages?Condapackagescancomefromavarietyoflocations:
PublicrepositorieshostedonPublicrepositoriesself-hostedPrivaterepositories
OndiskChannels—
AcollectionofpackagesownedbyauserororganizationConfigureCondatolookattheselocations(.condarcfile)
AnacondaCloud
AnacondaEnterprise
Demo:UsingPackages
CondaBasicsActivatingenvironments,acoupleways:
UsetheshortcutsincludedinProManuallyactivatetheenvironment:
cdC:\ArcGIS\bin\Python\Scriptsactivatearcgispro-py3
CondaBasics
Condainfoisthestartingpoint—ittellsyouthestateoftheenvironment.
conda--help
condainfo
CondaBasicscondalist
#packagesinenvironmentatC:\ArcGIS\bin\Python\envs\arcgispro-py3:#colorama0.3.7py35_0defaultscycler0.10.0py35_0defaultsfuture0.15.2py35_0defaultsmatplotlib1.5.3np111py35_0e[arcgispro]esrimpmath0.19py35_1defaultsnetcdf41.2.4py35_0e[arcgispro]esrinose1.3.7py35_1defaultsnumexpr2.6.1np111py35_0e[arcgispro]esrinumpy1.11.2py35_0e[arcgispro]esripandas0.19.0np111py35_0defaultspip8.1.2py35_0defaultspy1.4.31py35_0defaultspyparsing2.1.4py35_0defaultspypdf21.26.0py_0esripytest2.9.2py35_0defaultspython3.5.20defaultspython-dateutil2.5.3py35_0defaultspytz2016.6.1py35_0defaultsrequests2.11.1py35_0defaultsscipy0.18.1np111py35_0e[arcgispro]esrisetuptools27.2.0py35_1defaultssympy1.0py35_0defaultswheel0.29.0py35_0defaults
DeeperDive
CondaPackagingOK,sohowdowemakeanewpackage?
whichdescribestheinstructionstobuildthesoftware
WheredoIgetthecode?Whatarewebuilding,andwhatdoesitdependon?Runcondabuildtocreateapackagefromthisrecipe
Createarecipe
CondaPackagingmeta.yaml
package:name:conda-devsummit-2017-talkversion:"1.0"
source:git_url:https://github.com/scw/conda-devsummit-2017-talk.git
requirements:run:-python-scikit-learn
CondaPackagingBuildthepackage:
Uploadandshare:
condabuildc:\example\mypackage
anacondauploadmypackage
Demo:CondaPackaging
PythonPackageManager
HowcanIusethis?WealreadyshipyoutheSciPystack—powerfulandoutoftheboxinallproductsCondacommandandaCondarootPythoninstallNewmodules(e.g.requests),environmentwithPro
PythonPackageManagerinProGetpackages,expandyourpossibilityspacePackageyourwork:thisisanopportunitytodistributeit
WhereCanIRunThis?
ArcGISPro1.3CondaisthePythoninstall,includedforall
ArcGISPro1.4PythonPackageManagerPython3.5withcurrentpackageset
ArcGISEnterprise10.5ArcGISAPIforPython
fromfutureimport*ArcGISPro2.0:
ManageenvironmentsandchannelsResettableenvironment“μConda”
Resources
OtherSessionsWeds4:00PM,SanJacinto
Weds4:00,DemoTheater1
Weds4:30PM,DemoTheater1
Thurs10:30AM,MesquiteG-H
GettingDataSciencewithRandArcGIS
DeployingYourGeoprocessingToolsasPythonModules
PythonPackageManagementUsingConda
ContinuumAnalytics:ExploringContinuumAnalytics’Open-SourceOfferings
Condavs…Name Means Included?
Thecommanditself ✓AminimumsetofPythonpackagestobuildandrunConda.
✓
Adistribution200+packagesandrunConda
Self-hosted,distributedandHPCadditions
Conda
Miniconda
Anaconda
AnacondaWorkgroup
ResourcesCondaCheatsheetAnaconda.orgCondaRecipesHarnessingthePowerofPythoninArcGISUsingtheCondaDistributionUnderstandingCondabyJakeVanderplasSecurityupdatesforPythondepdendencies
Closing
ThanksEsriCondaTeam:
ContinuumAnalyticsforcreatingandopensourcingConda
RateThisSessioniOS,Android:Feedbackfromwithintheapp
Bewarnedthatwemayincorporatefeedbackintonextyear’ssession