Download - Python and the CERR Challenge Process
E L I Z A B E T H M C D O W E L L
M E S A C O U N T Y G I S
PYTHON & THE CERR CHALLENGE PROCESS
SEVERANCE TAX FUND
Fifty percent of the State’s receipts from the severance tax on minerals and mineral fuels are credited to the Local Government Severance Tax Fund. The Department of Local Affairs (DOLA) is directed to allocate 70% of these funds to local governments through discretionary grants and loans under the Energy and Mineral Impact Assistance Program. The remaining 30% is distributed directly to the municipalities and counties economically and socially impacted by mineral production based on certain measurable factors determined by the General Assembly.
Colorado Employee Residence Reports (CERR) are submitted by reporting parties and are used as one of the factors to determine the severance tax allocations to and within counties.
(State of Colorado. Department of Local Affairs. Federal Mineral Lease and State Severance Tax Direct Distribution: Program Guidelines, June 2011)
FOR 2013 REPORTING YEAR
• 237 CERRs (with Mesa County addresses)
• 233 Companies
• 16,088 Total Employees
• 3,261 Employee Addresses within Mesa County
• 356 Not Challengeable
• 106 Invalid Addresses (Incl. PO Boxes, bad addresses)
• 879 Challenged Addresses
Jurisdiction Company Reported
Employees Post-Challenge
Employees Net Gain
City of Fruita 328 298 -30 City of Grand Junction 1769 1150 -619
Mesa County 994 1689 695 Town of Collbran 28 18 -10
Town of De Beque 56 45 -11
Town of Palisade 86 72 -14 Garfield County 4 0 -4
Town of Parachute 7 0 -7 TOTAL 3272 3272 0
+21%
MESA COUNTY ALLOCATION
Direct Distribution Payment - 2014
$1,723,565.20
COLORADO EMPLOYEE RESIDENCE REPORTS
• Employers submit lists of employees involved in
mineral extraction that meet the following:
• Worked ≥ 500 hours in Colorado in any 6 months during the
reporting year
• Worked for the purpose of extraction within Colorado
• Submission guidelines:
• Report only employees during reporting year period
• Report only Colorado employees
• Report the PHYSICAL LOCATION of employee’s residence in
Colorado (not mailing address)
• No PO Boxes
WHAT WE GET
• Lists of employees that do not meet mineral
extraction requirements
• PO Boxes
• Incorrect / invalid addresses
• Addresses listed in the wrong jurisdiction
Communication from Local Governments back to Employers can
help improve the overall quality of employee reports
MESA COUNTY GIS
• Processes CERRs for municipalities within Mesa County
• City of Grand Junction
• City of Fruita
• Town of Palisade
• Town of De Beque
• Town of Collbran
• Include surrounding Counties/Municipalities in process
• Garfield County
• Town of Parachute
APPLICATIONS USED
• CERRs provided as Microsoft Excel workbooks • VBA Script to merge multiple workbooks
• Manually add, populate and rename fields
• Work in batches as CERRs become available
• Maintain data in SQL database • Scripts to prep data for geocoding in ArcGIS
• Union batch tables/views
• Reporting
• ESRI ArcMap 10.1 and PythonWin 2.7.2 • Geocoding
• Intersects, Spatial Joins
• Supporting Documentation
WHY PYTHON?
• ESRI Marketing
• Online Python class
• Python based sessions at the ESRI Developer
Conference, March 2014
• Data Driven Pages enhanced with Python
GOAL:
Use python scripting with DDP to create supporting
documentation for address challenges
CHALLENGE PROCESS OVERVIEW
1. Download / Prep CERRs for Processing
2. Geocoding / Address Verification
3. Supporting Documentation
4. Submit Challenges on DOLA’s Website
In previous years supporting documentation was
created from Assessor’s Property Detail
- Individually created per challenge address
- For 2012 Reporting Year: >1000
http://emap.mesacounty.us/assessor_lookup/
SUPPORTING DOCUMENTATION
• State Requirements:
• A current map or property tax record showing the address is within the municipal limits or located in unincorporated
county territory may be used. In addition, a letter from both
the municipality and the county agreeing to the address
location may be used. (DOLA Direct Distribution Frequently Asked Questions (FAQ) July 2014)
• ArcMap Data Driven Pages
• Solution to combine a current map with property tax table
for multiple properties
• Leverages existing County GIS data
• Greatly reduces tedium of individually creating PDFs from website
DATA DRIVEN PAGE SETUP
• Input Layer • Spatial join of geo-coded address layer with parcel data
• Copy of input layer used as the Page Definition Query to only show the active address on each page
• DDP Extent • Since parcel size varies greatly used “Best Fit” with a 1500%
Margin to get suitable surrounding areas around small parcels.
• Inset Maps • One of each municipality and one County-wide
• Extent indicator on main data frame to show where property is located in reference to municipal boundary
PYTHON SCRIPT COMPONENTS
• General Settings
• Map Document Variables
• Looping through Pages
• Map Layout
• Dynamic Layout Elements
• Export to PDF
Python script available on github:
https://github.com/emcdowell/Python-CERR.git
GENERAL SETTINGS
• Set Directory Paths / Output PDF
• Script Parameters
• Used to select different input without having to change
code
inputLayer = arcpy.GetParameterAsText(0)
• Allow script to overwrite outputs
arcpy.env.overwriteOutputs = True
MAP DOCUMENT VARIABLES
Set up map document variables mxd = arcpy.mapping.MapDocument(mxdpath)
ddp = mxd.dataDrivenPages
dataframes = arcpy.mapping.ListDataFrames(mxd, '')
actDataframe = arcpy.mapping.ListDataFrames(mxd, '')[0]
Define input table and temporary table variables inputTable = r'S:\IT\GIS\Liz\Python\CERR\Data\Python_CERR.gdb\MC_TAC_AgencyCodes'
tempTable = r'S:\IT\GIS\Liz\Python\CERR\Data\Python_CERR.gdb\tempTable'
Define field variables
LOOPING
Script has multiple loops • Primary Loop:
• Loop through each page using page number for pgNum in range(1, ddp.pageCount + 1):
ddp.currentPageID = pgNum
• Secondary & Tertiary Loops:
• Script loops through scenarios within each page
• Search & Insert Cursor
• Layout element positioning
• if…elif statements with
nested for…in statements
# Set variables to set data frame insets
reportedDistrict = ddp.pageRow.getValue(qMuni_County)
newDistrict = ddp.pageRow.getValue(qDistrict)
if reportedDistrict == 'City of Grand Junction':
lblGrandJunction.elementPositionX = 8.27
lblGrandJunction.elementPositionY = 2.63
lblFruita.elementPositionX = 10.0
lblCollbran.elementPositionX = 10.0
lblDebeque.elementPositionX = 10.0
lblPalisade.elementPositionX = 10.0
lblCounty.elementPositionX = 12.0
txtCounty.elementPositionX = 3.18
txtCounty.elementPositionY = 1.07
txtGrandJunction.elementPositionX = 10.0
txtFruita.elementPositionX = 10.0
txtCollbran.elementPositionX = 10.0
txtDebeque.elementPositionX = 10.0
txtPalisade.elementPositionX = 10.0
for frames in dataframes:
if frames.name == 'df_GJT':
frames.elementPositionX = 5.7
frames.elementPositionY = 0.5
elif frames.name == 'df_Address':
frames.elementPositionX = 0.2745
frames.elementPositionY = 2.9452
else:
frames.elementPositionX = 8.8
frames.elementPositionY = 0.7
elif reportedDistrict == ‘Mesa County’:
MAP LAYOUT ELEMENTS
• Each layout item MUST be uniquely named
• Size and Position Tab
• Element Name
• Create and do preliminary setup in ArcMap then
use Python to further manipulate the element
SCALE BAR
# Set scale bar based on page extent
if actDataframe.scale < 4000:
scale1.elementPositionX = 0.33
scale1.elementPositionY = 2.98
scale2.elementPositionX = 11.6
scale3.elementPositionX = 11.6
scale4.elementPositionX = 11.6
elif 4001 <= actDataframe.scale <= 10000:
scale1.elementPositionX = 11.6
scale2.elementPositionX = 0.33
scale2.elementPositionY = 2.98
scale3.elementPositionX = 11.6
scale4.elementPositionX = 11.6
elif 10001 <= actDataframe.scale <= 20000:
scale1.elementPositionX = 11.6
scale2.elementPositionX = 11.6
scale3.elementPositionX = 0.33
scale3.elementPositionY = 2.98
scale4.elementPositionX = 11.6
elif actDataframe.scale > 20001:
scale1.elementPositionX = 11.6
scale2.elementPositionX = 11.6
scale3.elementPositionX = 11.6
scale4.elementPositionX = 0.33
scale4.elementPositionY = 2.98
• Create four scale bars in
ArcMap
• Set Division Value for each
• Determine placement on layout
• Use Python to change X,Y
location based on the scale of
the active data frame
SEARCH & INSERT CURSORS
• Search Cursor
• Use For … In… to match record in input table to page variable
• Insert Cursor
• Populate temporary table with matching records
cursor = arcpy.da.SearchCursor(inputTable, ('TAC_Code', 'Year_', 'Agency_Name'), ''' "Year_" = 2013 ''')
insertcursor = arcpy.da.InsertCursor(tempTable, ('TAC_Code', 'Year_', 'Agency_Name'))
for row in cursor:
if row[0] == pgTAC:
insertcursor.insertRow(row)
del row
del insertcursor
del cursor
cursor = arcpy.da.SearchCursor(inputTable, ('TAC_Code', 'Year_', 'Agency_Name'), ''' "Year_" = 2013 ''')
insertcursor = arcpy.da.InsertCursor(tempTable, ('TAC_Code', 'Year_', 'Agency_Name'))
DYNAMIC TABLE
• Built using Python’s Search and Insert cursors
• Lists taxing authorities per challenge address • Verifies that a parcel is located inside or outside of municipal boundary
• Based on the number of rows in the table, script will adjust text size,
row height, grid line positioning
#Set and clone cell text elements
cellTxt.fontSize = rowHeight / 0.0155 y = upperY - headerHeight rows = arcpy.SearchCursor("in_memory\sort1") for row in rows: x = upperX + 0.05 col1CellTxt = cellTxt.clone("_clone") col1CellTxt.text = row.getValue("Agency_Name") col1CellTxt.elementPositionX = x col1CellTxt.elementPositionY = y
col2CellTxt = cellTxt.clone("_clone") col2CellTxt.text = row.getValue("TAC_Code") col2CellTxt.elementPositionX = x + 2.3 col2CellTxt.elementPositionY = y y = y - rowHeight if col1CellTxt.elementWidth > 2: col1CellTxt.elementWidth = 2
EXPORT TO PDF
# Set Output PDF (Beginning of script)
pdfPath = outDir + r"\FinalMapBook.pdf" if os.path.exists(pdfPath): # Check to see if file already exists, delete if it does os.remove(pdfPath) finalPdf = arcpy.mapping.PDFDocumentCreate(pdfPath) …
# Name and export individual pages (Inside Loop through each page) ddp.exportToPDF(pgLabel + ".pdf", "CURRENT") finalPdf.appendPages(pgLabel + ".pdf") …
# Commit Changes and delete variable references (Outside of loop) finalPdf.saveAndClose() del finalPdf
THREE HOURS I WAS ABLE TO SPEND WORKING ON SOMETHING ELSE!
For batch size between 150 – 200 addresses
MOVING FORWARD
• Significant time-saver (not counting the time it took
to put the script together)
• Will continue to work on script
• Incorporate intersect and spatial join
• Do more with parameters
LINKS
• DDPwithDynamicTablesAndGraphs_10.1_v1 http://www.arcgis.com/home/item.html?id=3a525b986b774a3f9cbbd8daf2435852
• Merge Excel Workbooks http://msdn.microsoft.com/en-us/library/office/gg549168(v=office.14).aspx
• My full script: https://github.com/emcdowell/Python-CERR.git