![Page 1: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/1.jpg)
Buying lidar dataRalph Haugerud
U.S. Geological Surveyc/o Earth & Space Sciences
University of WashingtonSeattle, WA 98195
[email protected] / [email protected]
October 2009, Geological Society of America Annual Meeting, Portland, Oregon
![Page 2: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/2.jpg)
How to buy lidar data• You know somebody…
A known quantity. Easiest• Advertise for bids
Maybe you don’t know as much as you think• NCALM (http://www.ncalm.org/)
NSF sponsored, partly subsidized, limited capacity• Participate in a consortium
Economies of scale, in-place contracting structure, community of experienceGeographically limited
• Start your own consortiumAdvertise for bids…
![Page 3: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/3.jpg)
![Page 4: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/4.jpg)
Specifications• Where• When (acquisition, lag
to delivery)• Spatial reference
framework• Ground control
(procedures, #, quality)• Instrument• Data density and
completeness
• Accuracy• Data products to be
delivered (kind, file formats, file naming)
• Acquisition conditionsSky: PDOP, # satellites in view, solar flares, airport operationsGround: leaves, standing water, snow cover, tides
• Data ownership
see A proposed specification for lidar surveys in the Pacific Northwest, in the course materials
![Page 5: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/5.jpg)
• $21K mobilization
• $400/mi2 raw cost
• Half-swath selvedge
• 2:1 rectangles
Costs for an ideal survey
![Page 6: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/6.jpg)
survey area, mi20 100 200 300 400 500 1000
$200
$400
$600
$800
$1000
2.6 pulse/m2
3 pulse/m2
1.4 pulse/m2, didn’t read RFP
1 pulse/m2, client pays weather
standby
2 pulse/m2
8 pulse/m2
Responses to a 2005 RFP
cost per mi2
![Page 7: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/7.jpg)
Recent PSLC contract with Watershed Sciences
• 50-100 sq mi2: $943/mi2 $420/km2
• 100-150 mi2: $704/mi2• 150-200 mi2: $592/mi2• 200-250 mi2: $521/mi2• > 250 mi2: $472/mi2 $210/km2
no mobilization fee includes 5% to Kitsap County
5% to Regional Council
![Page 8: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/8.jpg)
Data qualitya Puget Sound Lidar Consortium
perspective
![Page 9: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/9.jpg)
Outline
• Usability, Completeness, Accuracy
• A QA protocol: 3 analyses– Test against ground control– Examine images of bare-earth surface model– Evaluate internal consistency
• What accuracy do we need? Effects of correlated errors
![Page 10: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/10.jpg)
Usability
• Report of Survey is complete and correct
• Formal metadata are complete and correct
• Consistent, correct, and correctly labeled spatial reference framework
• Consistent file names and file formats
• Usable tiling schemecan calculate names of adjoining tiles
![Page 11: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/11.jpg)
Usability, continued
• Fully populated data attributesGPS week OR Posix time
• Consistent calibration
• Consistency between data layers
• No unnecessary artifacts in surface models
![Page 12: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/12.jpg)
Completeness
• Complete coverageFilling gaps requires remobilization, thus is very expensive
• Adequate data density
• Adequate swath overlap
These are what we pay for
![Page 13: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/13.jpg)
Accuracy is complicated
• Accuracy of point measurements What the vendor can be held responsible for
Evaluation requires abundant, expensive GCPs
• Accuracy of DEM What we care about
Evaluation requires abundant, expensive GCPs
![Page 14: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/14.jpg)
DEM error
DEM error ≈
[ (measurement error)2
+ (classification error)2
+ (interpolation error)2 ]1/2
small, ≤10 cmlarge in forested terrain ditto
To make bare-earth DEM:1) Measure XYZ of points2) Classify points as ground or not-ground 3) Interpolate ground points to continuous surface
Rule of thumb:internal DEM reproducibility = 1.5–2 x Z measurement reproducibility
![Page 15: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/15.jpg)
Accuracy is complicated
• Accuracy of point measurements What the vendor can be held responsible forEvaluation requires abundant, expensive GCPs
• Accuracy of DEM What we care aboutEvaluation requires abundant, expensive GCPs
• Reproducibility (consistency) of point positions Can be cheaply evaluated from swath overlapsProvides lower bound on measurement accuracy
• Reproducibility (consistency) of DEM Can be cheaply estimated from swath overlapsProvides lower bound on surface model accuracy
![Page 16: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/16.jpg)
PSLC QA protocol: 3 analyses
1. Test against ground control points (GCPs)
2. Look at large-scale shaded-relief images
3. CONSISTENCY analysis of swath to swath reproducibility, with completeness inventory
Extensive automated processing effectively tests for consistent file formats and file naming
![Page 17: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/17.jpg)
Test against GCPs• We use existing GCPs
(cost / benefit for new GCPs is too high)
• Must filter for landscape change, GCPs on corners, GCPs in vaults, etc.
![Page 18: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/18.jpg)
Test against GCPs
What do we learn?• Confirm absolute accuracy, typically with low
confidence• Identify undocumented and misdocumented
spatial reference frameworks
![Page 19: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/19.jpg)
Look at large-scale shaded-relief images
Too-high points (positive blunders)
Stretched data along tile boundary
![Page 20: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/20.jpg)
• Highest-hit surface, voids = NoData.
• 6-ft raster
• Voids for open water (specular reflection) and in forest (off-nadir shadowing by canopy)
3,000 ft
GOOD
![Page 21: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/21.jpg)
Observation: Voids end at tile bound-aries, not only at swath margin.
Inference: Data omitted on tile-by-tile basis
3,000 ft
BAD
![Page 22: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/22.jpg)
Tile-boundary artifacts
points scalped off corners
swath-boundary fault
points scalped off bluff corners
corduroy
Poor veg penetration, swath mismatch, bad point classification
![Page 23: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/23.jpg)
Observation: 3 ft step in freeway at tile boundary
Inferred cause: tile to tile variation in inclusion of data, “calibration”, and/or range-walk correction
![Page 24: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/24.jpg)
CONSISTENCY analysis• Start with tile of multiple-swath data• Sort on time. Split into swaths at time breaks. For
each swath– Identify data areas– Build surface (1st-return points TIN lattice)
• Subtract swath surfaces, spatially merge results• Calculate curvature to identify smooth areas where
interpolation is valid• Make image
– Saturated color = smooth area with overlap– Unsaturated color = rough area with overlap– Gray = no overlap– White = no returns
![Page 25: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/25.jpg)
1 km
Difference (Z) between overlapping swaths
No off-nadir returns from open water
Faint striping reflects
imperfections in pyramidal scanner?
Aircraft roll
Blue to green Blue to green transition across transition across swath: relative tiltswath: relative tilt
Striping of pitched roofs: X-Y shift between swaths
No returns from No returns from roofs of large roofs of large
buildingsbuildings
![Page 26: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/26.jpg)
1st-return (measurement) reproducibility
ground-return (DEM) reproducibility
pulse density
analysis is
completely
automated
![Page 27: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/27.jpg)
50% of data
95% of data
99% of data wiggly black line: RMS Y for each X
white line: least-squares best fit quadratic
n ≈ 25 million
P1 RMSEz = 8.6 cm(from Y intercept)
P2 RMSExy = 14 cm(calculated from value at
x=100)
TerraPoint 2005 Lower Columbia survey
25% sample of two 7.5- minute quadrangles, 2 pulse/m2
![Page 28: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/28.jpg)
Observation: errors in lidar measurements show strong spatial
correlation
1 km
![Page 29: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/29.jpg)
Hypothesis: errors in photogrammetric DEMs have little spatial correlation
Why?
• Aerotriangulation and image orientation done with greater care (and more redundancy) than identification of corresponding image points
• Largest source of error has no spatial correlation beyond that imposed by structure of target region
![Page 30: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/30.jpg)
Minimum Contour Interval
max
imu
m a
llow
able
rm
s Z
err
or
2 ft
20 c
m
ASPRS vertica
l acc
uracy
standard
National M
ap Accura
cy
Standard
This relationship between measurement accuracy and intended use is (I hypothesize) empirical and based in
experience with photogrammetric DEMs
![Page 31: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/31.jpg)
• Uncorrelated errors disappear upon spatial averaging
• Drawing contours (and cut-and-fill calculations) involves spatial averaging
• Contouring minimizes errors in photogrammetric DEMs by averaging them away
• Contouring a lidar DEM, with its highly correlated errors, does NOT minimize errors by averaging
![Page 32: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/32.jpg)
Lidar surveys for contouring (and other averaging operations) should be more accurate than suggested by ASPRS and NMAS standards
Lidar surveys for feature recognition (e.g., finding fault scarps, counting trees) can be significantly less accurate than experience might suggest, provided adequate XY resolution
![Page 33: Buying lidar data Ralph Haugerud U.S. Geological Survey c/o Earth & Space Sciences University of Washington Seattle, WA 98195 rhaugerud@usgs.govrhaugerud@usgs.gov](https://reader035.vdocument.in/reader035/viewer/2022081514/56649d1f5503460f949f3f35/html5/thumbnails/33.jpg)
Lidar data quality has 3 dimensions– Usability– Completeness– Accuracy
Evaluate lidar data quality by – Testing against ground control– Looking at big images– Quantifying swath to swath reproducibility and
completenessStandards for required mean accuracy
need revision– Inundation modeling requires better absolute accuracy
than we expect– Geomorphic mapping (feature recognition) requires
less absolute accuracy