![Page 1: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/1.jpg)
Best Practices for Publishing Online ServicesMichael DangermondPaul DoddKeith VanGraafeiland
![Page 2: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/2.jpg)
Agenda
• Feature Services• Image Services• Multidimensional Image Services
![Page 3: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/3.jpg)
Section SubheadFeature Services
![Page 4: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/4.jpg)
Keith VanGraafeiland
Hosted Feature Services Examples
![Page 5: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/5.jpg)
Feature Services
• Overwrite Feature Service – from web content• Hosted Feature Views• Collaborative Group
• Examples- Coral Reef Stations- USA Weather Watches and Warnings- Arctic Sea Ice Extent
![Page 6: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/6.jpg)
Overwrite Feature Services
• Keeping it updated• There’s a script to do that?!?
- https://www.arcgis.com/home/item.html?id=d45f80eb53c748e7aa3d938a46b48836
• Append functionality coming soon!• Link to webinar
![Page 7: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/7.jpg)
Updating Hosted Feature Services
• Append- More Flexible / Efficient loader- Stream to Database – Bulk Load- Preferred over
- Batch: Add / Update / Apply Edits
• When is Overwriting practical?- High Row Replacement- No Unique Identifier (ex. weather data)
- Append / Delete too expensive
• Enterprise Equivalent to Overwrite- Disable Indexes- Truncate Data- Append Updates- Rebuild Indexes!
• Overwrite will impose Downtime!- Features are inaccessible during update- Enabling CDN cache helps!- Doesn’t have to be! (more later)
![Page 8: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/8.jpg)
Leveraging Hosted Feature Views
• Create from Hosted Feature Service- Never Duplicate Data- Not effected by Service Overwrite!- Points to Host Service Layers
• Provides High Level of Control- Rename Layers- Alter Aliases- Control Caching- Limit access to sensitive data Fields- Limit Feature access by extent- Control Sharing
File Item
Hosted Feature Service & Item
Hosted Feature Views & Items
Internal View
External View
Public View
![Page 9: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/9.jpg)
Feature Service ‘A’
Overwrite Feature Services – Layer Swizzling
• To Swizzle:1. To alter a reference by changing what it
points to.2. To present or change the appearance of
something based on something else.Ex: Lookup Table or Coded Value Domain
• Simply put:- Change what your Layers point to!
• Overwrite w/o Downtime!• Frequent / Extended Update Times
Feature Service ‘B’
Private Items
Private Items
Shared View
View Template
View Template
Active Service Active ServiceUpdate Target Update Target
Update, Swizzle, Repeat!
![Page 10: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/10.jpg)
Enterprise Example – Table Swizzling
• National Water Model – Time Series- 2.7 Million Features (river reaches)
- x 80 - 3hr Forecasts (medium range, 4/day)
- x 18 - 1hr Forecasts (short range, 24/day)
- = 2+ Billion Rows Updated Daily!
Public Map Service
Feature to Table Join
(view)Line
Featureclass
DB Table ‘A’ DB Table ‘B’
Active Table Active TableUpdate Target Update Target
Update, Swizzle, Repeat!
![Page 11: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/11.jpg)
Miscellaneous Tips
• Time Series Layers- REST admin console to refine Time details- Stick to Date and Numeric fields for Time!
• GeoDatabase Attribute Fields- Use Appropriate Type & Length for Data- Limit to a practical number, set visibility- Only add Indexes to Query-Likely Fields- Leverage Coded Value Domains!- Use Reasonable Names and Aliases- Add Units to Alias of Numeric Fields
- Ex: ‘Speed (km/h)’ or ‘Temperature (kelvin)’
• Geometries- Watch your Feature complexity!
- 10k vertices / feature ok- Too many Parts can slow load / draw times!
- Use ‘Dice’ GP tool to break up Features!
• GeoDatabase Feature Datasets- Don’t use as a content organizer!
- Locks applied to every object when accessing one!
![Page 12: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/12.jpg)
Image Services
![Page 13: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/13.jpg)
Michael Dangermond
Thematic Image Services
![Page 14: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/14.jpg)
![Page 15: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/15.jpg)
![Page 16: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/16.jpg)
![Page 17: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/17.jpg)
Mosaic Dataset+ overviews
Raster
Image Service
Clients
![Page 18: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/18.jpg)
Thematic Image Served From a Mosaic
Easier to Maintain• Upgrading can be this easy:
Turn off the service.Replace the TIFF image in the mosaic with one with the same name.Turn the service back on.
• In other words, make any changes within the mosaic and refresh the service.• The ID, content item, popup, URL stays the same.
![Page 19: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/19.jpg)
Mosaic Dataset
Rasters
Image Service
Clients
Mosaic Datasets+ overviews
BLM NPS USFS
AlaskaHawaiiCONUS
Puerto Rico
USA Federal Lands Service
![Page 20: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/20.jpg)
Mosaic Dataset
Rasters
Image Service
Clients
Mosaic Datasets+ overviews
BLM NPS USFS
AlaskaHawaiiCONUS
Puerto Rico
ProcessingTemplatesServer Functions and
Definition Queries
USA Federal Lands Service
![Page 21: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/21.jpg)
![Page 22: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/22.jpg)
![Page 23: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/23.jpg)
Attribute Table Function
• ClassName field appears in legend• Red, Green, Blue fields color the map• Other text fields display
in an online popup• Maximum 2000 Records
in attribute table
![Page 24: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/24.jpg)
Time-EnabledMosaic Dataset
2018Rasters
Time-EnabledImage Service
Time-AwareClients
Mosaic Datasets+ overviews
20162017 2018
AlaskaHawaiiCONUS
Puerto Rico
USA NLCD Land Cover Service
![Page 25: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/25.jpg)
![Page 26: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/26.jpg)
Creating a Thematic Image Service
1. Organize or Create Rasters2. Create an Attribute Table3. Create a Mosaic4. Publish an Image Service
Download the Checklist
Step by step instructions help you through the details as you publish services.https://tinyurl.com/tichecklist
![Page 27: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/27.jpg)
Organize rasters
• Common projection, nodata symbol, bands, bit depth, type, etc.• Strip off tables and colormaps from rasters.
Apply attribute table function on the mosaic using a processing template.
![Page 28: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/28.jpg)
Organizing Your Rasters
• Recommended workflow: Use CopyRaster() to createTIFF images with LZW compression. Use the same NoData symbol for each raster.
• Use an equal area projection so client area counts are accurate.USA – Albers ProjectionsWorld – Mollweide Projection has the least distortion. We checked.
![Page 29: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/29.jpg)
Prepare an Attribute Table
• Geodatabase table• Value and count fields are required• Add ClassName, Red, Green, Blue
fields for the legend and map colors,plus additional fields to drive popups,if desired.
![Page 30: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/30.jpg)
Create mosaic
• Same projection, nodata symbol, bands, bit depth, etc.• Before creating the mosaic decide:
- Mosaic’s name- Service name- Projection
![Page 31: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/31.jpg)
OverviewsBest practices
• Overviews are a lower resolution version of your rasters.
• Edit the mosaic footprint table,change all MaxPS and HighPS values to the maximum value in the table.
• Define Overviews• Build Overviews
![Page 32: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/32.jpg)
Thematic Image Service ChecklistHere’s where to download the checklist:
https://tinyurl.com/tichecklist
![Page 33: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/33.jpg)
Image Services
• Simple Example - Topobathy• More Complex Example – USA Landcover? Cropscape?• Raster Function Templates• Using Image Services in Pro
![Page 34: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/34.jpg)
Keith VanGraafeiland
Multidimensional Image Services
![Page 35: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/35.jpg)
Oceanographic Information for the World
• Five variables using on-the-fly calculations from HYCOM
AT YOUR FINGERTIPS
• Sea Surface Elevation
• Temperature, Salinity, Eastward Velocity, Northward Velocity
- 40 Standard Depth Levels
• 7 Day Forecast
• 30 Day Hindcast
![Page 36: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/36.jpg)
Multidimensional Image Services
• What we see:- Global high resolution data- Temporal component- Continuous updates
• How it works- S3 used for storage- ALF Processor / ALF Deployer- Applying RFT’s for additional
functions
![Page 37: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/37.jpg)
HYCOM CapabilitiesTemperature Salinity Eastward
VelocityNorthward
Velocity
2.9 GB 2.9 GB37 MB
Elevation• Sea Surface Elevation
• Temperature
• Salinity
• Water U
• Water V
Files are downloaded and converted to Meta Raster Format (MRF).
Placed in Amazon s3
Referenced to mosaic
Additional Raster Functions applied for unit conversion
![Page 38: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/38.jpg)
HYCOM by the Numbers
• 1 x Sea Surface Elevation
• 40 x Temperature
• 40 x Salinity
• 40 x Water U
• 40 x Water V
• 40 + 40 + 40 + 40 +1 = 161 file sets
Every 3 hour Interval
644 files (4 files per MRF 161 x 4)2.43 GB
Each day
5,152 files (644 x 8 intervals)19.44 GB
Entire Volume
190,624 files (5,152 x 37 days)719.28 GB
Elevation Temperature Salinity Eastward Velocity
Northward Velocity
![Page 39: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/39.jpg)
Future Enhancements
• Multi-Dimensional MRF or CRF- Consolidating down to 5 files per time slice (one for each variable)
- Limits the amount of requests to s3- Distributing files between multiple s3 buckets (sharding)- Reduce s3 as the “bottleneck”- Programmatic
![Page 40: Best Practices for Publishing Online Services · 2019-08-28 · Overwrite Feature Service – from web content ... Placed in Amazon s3 Referenced to mosaic Additional Raster Functions](https://reader031.vdocument.in/reader031/viewer/2022011910/5f85041aba11d5709f24689b/html5/thumbnails/40.jpg)
Please Share Your Feedback in the App
Download the EsriEvents app and find
your event
Select the session you attended
Scroll down to “Survey”
Log in to access the survey
Complete the survey and select “Submit”