Tommy Fauvell

@CartoRedux

Caching Maps and Vector Tile Layers: Best Practices

Session Goals

• Introduction to Caching and terminology

• Differences between raster tiles and vector tiles

• Sharing / Publishing best practices

• Most efficient way to generate cache

• Framework for tackling basemap projects

Cache smarter, not harder…

• Informative

• Responsive

• Flexible

- Combine with other data / basemaps

- Usable across various clients (mobile / web / desktop)

• Current with timely updates

• Cartographically rich

Fast and responsive maps keeps eyeballs on your content and users engaged

Overview | What makes a good web map?

WebmapFile

Shapefile

Tables

Raster

ArcGIS Data Store

Dynamic Data can take a long time to render

Caching is an investment of time and infrastructureto deliver cartographically rich data in an efficient manner

Tile Cache Webmap

Raster

ArcGIS Data Store

File

Shapefile

Tables

Overview | Raster Tiles and Vector Tiles

Raster Tiles

• Pre-rendered snap-shots of your data

- JPEG’s, PNG’s, LERC

• Smart tile types:

- Mixed and PNG

• Most flexible client support

• Compute and storage intensive

Vector Tiles

- Tiled containers of data

- Features and attributes

- Rendering instructions

- Style file

- Additional map components

- Fonts and Icons

• Very fast to cook at a fraction of raster tile storage costs

• Only supports VECTOR datasets

• Rendered by client devices

- Not ArcMap…

MIXED or JPEG-65Mosaic Dataset footprints

LERC-0.1Mosaic Dataset Footprints

JPEG-90Custom AOI

Vector Tile

ArcGIS Server ArcGIS Enterprise ArcGIS Online

Federated Server Hosting Server

ArcGIS Pro

Overview | Software and Architecture Terminology

Overview | Software and Architecture Terminology

By Reference

You keep using that word…I don’t think it means what you think it means

Navigating the Esri CacheEcosystem

Overview | Cache is everywhere!

Design

• Requirements

• Datasets

• Intended use

• Constraints

• Deadlines / Milestones

Author

• ArcGIS Pro Project

• Prototype

• Finalize design

• Cartography / Multiscale Mapping

Cache

• Publish to Development

• Test Cache

• Generate Cache

• QC

Deploy

• Publish to Staging and validate

• Publish to Production

• Feedback

Overview | Basemap Project Lifecycle

Overview | Raster Tiles - Create Map Tile Package

• Improved at Pro 2.5

- Parallel Processing Factor

• Good for:

- Direct to TPK / TPKX

- Regular extents

• Limitations:

- Uses envelope extents vs detailed AOI

- No partial updates

Overview | Raster Tiles - Manage Tile Cache

• Improved at Pro 2.5

- Better use of Parallel Processing Factor

• Good for:

- ArcGIS Server or Enterprise unavailable

- AOI / irregular extents

- Partial updates

• Warning:

- Uses current map extent!

Overview | Raster Tiles - Manage Map Server Cache Tiles

• Good for:

- Caching on a deadline

- Most scalable option

- AOI / irregular extents

- Partial updates

- Doesn’t tie up workstation / runs asynchronously

• Warning:

- Will make servers cry, scale responsibly

Overview | Vector Tiles – Create Vector Tile Package

• Good for:

- DIY approach to creating vector tile tilesets

- Short to no-notice deadlines

- Mobile / offline support

- Only uses local system resources (single process)

- Does not require parallel processing

• Warning:

- Modern web browsers and supported clients

- Not ArcMap

- Requires ArcGIS Enterprise or ArcGIS Online

Overview | Vector Tiles – Create Vector Tile Index

• Good for:

- DIY approach to creating vector tile tilesets

- Use first for complex and vast basemap projects, then use the output in Create Vector Tile Package

• Warning:

- Don’t modify max vertex count…unless you have a really good reason

Cache smarter not harder…

Optimizing Cache Generation

Optimizing | Data

• Use a local FGDB copy / extract of your data

• Clean your data

- Eliminate duplicates

- Check/fix geometry errors

• How dense is your data?

- Set reasonable scale dependencies

- Generalize

Optimizing | Cartography

• Don’t start from scratch, import your MXD

• Set your scales according to the tiling scheme you select

• Remember scale logic in Pro is different from ArcMap

• Convert representations to unique value symbols

• Limit…

- number of layers

- duplication of content

- inclusion of additional fields / data in the tileset

Optimizing | Cartography

• Avoid…

- group layers

- complex symbols and unsupported symbol effects: hatched / gradient fills

- unsupported layer types: annotations, basemaps

• Be mindful of users that want to re-style your maps

Cache smarter not harder…

Optimizing Raster Tile Cache Generation

• Don’t use Fine, Verbose, or Debug logging.

• Size your Caching Tools Instances:

- N = # of cores per machine

- Min and Max = N

- 2 – 4GB of RAM x N

- Decrease N if necessary

Raster Tiles | ArcGIS Server Configuration

• 10.7+

• Cached map services are good candidates for shared instances

• Most popular dynamic map services as shared instances could be considered for caching

• Monitor usage: server statistics / logs

• User feedback / complaints

Shared Instances

Raster Tiles | ArcGIS Server Configuration

• Only cache what is necessary

• Use AOI’s with decreasing coverage as you increaseLOD’s

• Break your basemap project into multiple cache jobs by bracketing LOD’s

- Each job can / should have a unique AOI

• Only update what has changed

- You don’t need to re-cache everything if you have partial updates to your data

147M72M36M18M9M4M2M1M

577K288K144K72K36K18K9K4K2K1K

147M72M36M18M9M4M2M1M

577K288K144K72K36K18K9K4K2K1K

L00L01L02L03L04L05L06L07L08L09L10L11L12L13L14L15L16L17

Raster Tiles | Cache Jobs

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

0 2 4 6 8 10 12 14 16 18

Number of Host Machines

Millions of Tiles Per Minute

Actual

Expected (90,000 T/m)

0

60

120

180

240

300

0 2 4 6 8 10 12 14 16 18

Number of Host Machines

Completion Time in Minutes

M900 Blade: Intel Xeon E7-4820, 32 Cores, 256GB Ram

Raster Tiles | ArcGIS Server Scalability

Summary | Optimizing Cache Generation

• Optimize your data:

- spatial index, compact FGDB, copy data local

• Optimize Map and Imagery Projects:

- analyzer results, scale dependencies, Maplex when needed, Mosaic Dataset tuning

• Configure ArcGIS Server Caching instances

• Optimize raster tile cache jobs:

- Multiple jobs, AOI per LOD’s per job, only cache what is necessary

• ArcGIS Server will scale and leverage system resources for raster tile generation

• Vector Tiles are created using local resources not ArcGIS Server

Vector Tiles

Vector Basemap

• Integrated Sharing Workflow:

- Share - Publish Web Layer

- Copy all data

- Vector Tiles

- Configure parameters

- Cooks, uploads, publishes the hosted tile layer

Caching | Vector Tiles

Author Map CachePublish Hosted

Tile Layer

• Geoprocesing Tools:

- Create Vector Tile Index

- Create Vector Tile Package

- Share Package (publish tile layer option*)

- Publish hosted tile layer (Web)

Raster Tiles

Vector Basemap

• Publish then Cache

• Share – Publish Web Layer

- Configure cache parameters*

- Always “cache manually”

• Cache

- Manage Map Server Cache gptool

Caching | Vector as Raster Tiles

Author Map

Publish Cache

Cooking | Manage Map Server Cache Tiles

• Only cache what is necessary

• Use AOI’s with decreasing coverage as you increaseLOD’s

• Break your basemap project into multiple cache jobs by bracketing LOD’s

- Each job can / should have a unique AOI

• Only update what has changed

- You don’t need to re-cache everything if you have partial updates to your data

147M72M36M18M9M4M2M1M

577K288K144K72K36K18K9K4K2K1K

147M72M36M18M9M4M2M1M

577K288K144K72K36K18K9K4K2K1K

Raster Tiles | Cache Jobs

ArcGIS Pro

Imagery Project

• Publish then Cache

• Share – Publish Web Layer

- Initiated from Mosaic Dataset / Raster Dataset

- Configure cache parameters

- Always “cache manually”

• Cache

- Manage Map Server Cache gptool

Caching | Imagery as Raster Tiles

Author Map or Mosaic Dataset

Publish Cache

ArcGIS Pro

Terrain Layer Project

• Publish then Cache

• Share – Publish Web Layer

- Initiated from Scene and TOC

- Configure cache parameters

- Always “cache manually”

• Cache

- Manage Map Server Cache gptool

Caching | Terrain as Raster Tiles

Author Scene

Publish Cache

Summary• Design:

- Operational / base map content

- Clients (desktop/ mobile)

- Updates

• Caching:

- Use Pro for Authoring basemaps

- Vector tiles for vector content

- Raster tiles for imagery/hill shades

- Raster tiles for vector data (if your clients do not support WebGL)

- Use hosted feature service for information

• Optimize cache generation

- Keep data close

- Tune platform / simplify data

- Use mosaic dataset for imagery

Print Your Certificate of Attendance

Print Stations Located in 150 Concourse Lobby

Tuesday12:30 pm – 6:30 pmExpoHall B

5:15 pm – 6:30 pmExpo SocialHall B

Wednesday10:45 am – 5:15 pmExpoHall B

6:30 pm – 9:30 pmNetworking ReceptionSmithsonian National Museumof Natural History

Download the EsriEvents app and find

your event

Select the session you attended

Scroll down to “Survey”Log in to access the

surveyComplete the survey and select “Submit”

Please Share Your Feedback in the App

And maybe some answers?

Questions?