2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman
TRANSCRIPT
![Page 1: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/1.jpg)
Spatial Analysis at the Continental Scale: A Practical Approach
Dan GetmanGeospatial Big Data SolutionsDigitalGlobe9/22/2016 GISITR
![Page 2: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/2.jpg)
~ 7.6M sqkm~ 24M people
~ 13M structures
https://www.psma.com.au/sites/default/files/styles/image_800x534/public/shutterstock_138152432.jpg?itok=GLja4tlG
![Page 3: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/3.jpg)
What are we delivering?
![Page 4: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/4.jpg)
![Page 5: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/5.jpg)
![Page 6: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/6.jpg)
http://i1.wp.com/altigator.com/wp-content/uploads/Multispectral-vs-Hyperspectral1.png?w=916
![Page 7: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/7.jpg)
http://missionscience.nasa.gov/ems/08_nearinfraredwaves.html
![Page 8: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/8.jpg)
http://s3.amazonaws.com/content.satimagingcorp.com/media/cms_page_media/66/stereo-ikonos-data-collection-page_1.jpg
![Page 9: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/9.jpg)
http://content.satimagingcorp.com/static/galleryimages/3m-dsm-combined-web.jpghttp://s3.amazonaws.com/content.satimagingcorp.com/media/cms_page_media/66/stereo-ikonos-data-collection-page_1.jpg
http://www.ltdwatch.com/jls-in-3d/
![Page 10: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/10.jpg)
10To Update Footer, Find Header & Footer in Insert Tab and Apply to All
Zoom In
![Page 11: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/11.jpg)
11To Update Footer, Find Header & Footer in Insert Tab and Apply to All
![Page 12: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/12.jpg)
https://www.psma.com.au/geoscape
![Page 13: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/13.jpg)
https://www.psma.com.au/geoscape
LULCLULCTree
Masked DSM
Tree Masked
DSM
Roof Materials Classification
Roof Materials Classification
Roof Materials Classification
Swimming Pools and PV Systems
LULC
![Page 14: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/14.jpg)
VNIR Images SWIR ImagesPan Images
![Page 15: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/15.jpg)
Digital Surface Model
VNIR Images SWIR Images
Land Use Land Cover
Tree Height Assessment
Tree MaskedDigital Surface Model
Pan Images
![Page 16: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/16.jpg)
Digital Surface Model
VNIR Images SWIR Images
Land Use Land CoverBuilding Footprints
Tree Height AssessmentRooftop Height Assessment
Tree MaskedDigital Surface Model
Pan Images
![Page 17: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/17.jpg)
Solar Panels and Pools
Digital Surface Model
VNIR Images SWIR Images
Land Use Land CoverBuilding Footprints
Tree Height AssessmentRooftop Height Assessment
Tree MaskedDigital Surface Model
Pan Images
![Page 18: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/18.jpg)
Solar Panels and Pools
Composite ImageDigital Surface Model
VNIR Images SWIR Images
Land Use Land CoverBuilding Footprints
Roof Top Material Classification
Tree Height AssessmentRooftop Height Assessment
Tree MaskedDigital Surface Model
Rooftop Material Assessment
Pan Images
![Page 19: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/19.jpg)
Data Provider
Analysis Paradigms (a few anyway)
Cloud Based Organized
Image StoreCloud Based
Scalable Compute
Cloud Based WMS or Tile
Store Cloud Based
Scalable Compute
![Page 20: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/20.jpg)
Analysis Paradigms: Scalable (GBDx)
• Analyst never touches or purchases imagery, just information
• Analyst can run their own algorithms or anyone else's• Leverages compute size needed for each process• Parallelized on nodes and through data distribution
across nodes
• Configured for processing at the state, national, continental scale
• Configured for processing all imagery that meets certain specifications as it is collected
Cloud Based Image Processing Framework
Image Catalog
User Defined
Processing
Scalable Compute
Provider Defined
Processing
![Page 21: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/21.jpg)
Analysis Paradigms: Scalable (GBDx) Raw Image
Orthorectify
Compensate for
Atmosphere
Pan Sharpen
Other Standard Function
User Defined Function
Create Output
Rest Endpoint
User Defined Function
User Defined Process
Specified Through the
APIOutput can be
imagery, vector, tabular
data
Starts with
“Raw” Image
![Page 22: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/22.jpg)
GBDx
AWS S3
100s of Source Datasets
EC2 Instance
Postgres and Python
Many EC2 Instances
Many Python Process
AWS S3
Result Datasets
![Page 23: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/23.jpg)
GBDx
AWS S3
100s of Source Datasets
EC2 Instance
Postgres and Python
Many EC2 Instances
Many Python Process
AWS S3
Result Datasets
Main module reads in
metadata and creates
optimized number of runs
![Page 24: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/24.jpg)
GBDx
AWS S3
100s of Source Datasets
EC2 Instance
Postgres and Python
Many EC2 Instances
Many Python Process
AWS S3
Result Datasets
Each worker pulls needed
data and distributes
across its cores
![Page 25: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/25.jpg)
GBDx
AWS S3
100s of Source Datasets
EC2 Instance
Postgres and Python
Many EC2 Instances
Many Python Process
AWS S3
Result Datasets
Main module aggregates results and submits the final aggregation process on GBDx, which pushes the final
result to the customer S3 bucket
![Page 26: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/26.jpg)
GBDx
AWS S3
100s of Source Datasets
EC2 Instance
Postgres and Python
Many EC2 Instances
Many Python Process
AWS S3
Result Datasets
• Imagery is never purchased or downloaded, just analyzed
• Processing is automated and event driven• Customer gets information, not imagery
![Page 27: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/27.jpg)
![Page 28: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/28.jpg)
We Get Captured at
Different Times
![Page 29: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/29.jpg)
![Page 30: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/30.jpg)
We All Run In Parallel
As Data Arrives
![Page 31: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/31.jpg)
![Page 32: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/32.jpg)
![Page 33: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/33.jpg)
![Page 34: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/34.jpg)
![Page 35: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/35.jpg)
![Page 36: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/36.jpg)
![Page 37: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/37.jpg)
![Page 38: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/38.jpg)
![Page 39: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/39.jpg)
![Page 40: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/40.jpg)
![Page 41: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/41.jpg)
![Page 42: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/42.jpg)
• Ground elevation is higher than eave or roof elevation
• Roof elevations have non-standard distribution
• Eave is below 2.5m (one story building)
Issues Impacting Building Height Accuracy
![Page 43: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/43.jpg)
Issue 1: Ground elevation is higher than eave or roof elevationArea used to estimate
ground elevation(purple, not including
inside of polygon)
Area used to estimate eave elevation
(pink, not including inside of polygon)
Area used to estimate roof elevation
(green, including inside of polygon)
![Page 44: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/44.jpg)
Issue 1: Ground elevation is higher than eave or roof elevation
In considering the area covered by this doughnut, note that
the ground is is almost completely
obscured by trees or other buildings
Vegetation is masked out of the DSM, which means that this point is likely to be our only
“ground” measurement…
![Page 45: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/45.jpg)
Issue 1: Ground elevation is higher than eave or roof elevation
This “building” has the same issue with surrounding trees, but as it’s really an empty pool that just looks like a building, the ground
elevation actually should be higher than the “eaves”
![Page 46: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/46.jpg)
Issue 2: Roof elevations have non-standard distribution
Trees are intentionally
masked from the elevation data
![Page 47: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/47.jpg)
Issue 2: Roof elevations have non-standard distribution
Unfortunately, mainly due to mixed pixels, some trees are
missed in the masking
![Page 48: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/48.jpg)
Issue 2: Roof elevations have non-standard distribution
Q3 to Q4 range is large
Looks like a building
![Page 49: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/49.jpg)
Issue 2: Roof elevations have non-standard distribution
Q3 to Q4 range is large
Looks like a tree
![Page 50: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/50.jpg)
Issue 2: Roof elevations have non-standard distribution
Trees are removed and the whole distribution looks like a building
![Page 51: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/51.jpg)
• Types of occurrences • Eave is < 2.5m and Roof is < 2.5m• Lots of sheds and water storage
in this group• Is it an error or a correct
estimate?
Issue 3: Calculated eave heights are below a one story building (2.5m)
![Page 52: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/52.jpg)
Issue 3: Calculated eave heights are below a one story building (2.5m)Eave is < 2.5m and Roof is < 2.5m:
![Page 53: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/53.jpg)
• Anyone can create a process that operates on imagery and add it to the ecosystem
• Processes can operate on every square meter of the globe
• Processes can operate on imagery as it is collected for an AOI
• Low resolution imagery can be used to determine which high resolution imagery is needed to save costs
• This can be driven by other data science applications
• Examples• Detect when an airport is operating
abnormally• Detect when air/auto traffic patterns change• Detect when there is more or less activity in
certain areas
Enormous Opportunities
![Page 54: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/54.jpg)
![Page 55: 2016 asprs track: spatial analysis at the continental scale: a practical approach by dan getman](https://reader036.vdocument.in/reader036/viewer/2022070516/5870706a1a28ab48378b7769/html5/thumbnails/55.jpg)
Questions and Comments?