enterprise gis: performance and scalability · 2015-07-30 · •multiple administrators •...
TRANSCRIPT
![Page 1: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/1.jpg)
Enterprise GIS: Performance and Scalability
Andrew Sakowicz, [email protected] Pizzi, [email protected]
![Page 2: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/2.jpg)
Esri UC 2014 | Technical Workshop |
Definitions
![Page 3: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/3.jpg)
Performance
• Speed, e.g. response time (seconds)
![Page 4: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/4.jpg)
Scalability
• The ability to increase output and maintain acceptable performance
![Page 5: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/5.jpg)
Capacity
• The maximum level of output the system can produce, e.g.• X cars/sec• X maps/sec
At capacity Over capacity
![Page 6: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/6.jpg)
Bottleneck
• Resource(s) limiting the performance or capacity
Not bottleneck bottleneckThink of :Lanes -as CPU processorToll -as ArcGIS Server instancesCars -as map requests
![Page 7: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/7.jpg)
Capacity
Throughput(req/hr)
CPU Utilization (%)
Memory used (Mb)
Network used (Mbps)
Content length (bytes)
Response Time (sec)
Time
User load
Capacity (~ 85% utilization)
![Page 8: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/8.jpg)
Esri UC 2014 | Technical Workshop |
Process and Tools
![Page 9: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/9.jpg)
Process and Tools
Type Presentation Title Here
![Page 10: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/10.jpg)
System Tools overview
• http://www.arcgis.com• owner:EnterpriseImp• Show ArcGIS Desktop Content
![Page 11: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/11.jpg)
Relationship between System Tools
User LoadCPU%
Capacity models
Performance Tests
![Page 12: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/12.jpg)
System Tools frameworkSystem Tools are not just tools
Tool
Patterns
Discipline
![Page 13: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/13.jpg)
Esri UC 2014 | Technical Workshop |
Infrastructure Capacity Planning
![Page 14: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/14.jpg)
Provide sufficient hardware resources
GIS Systems are bound by:1. CPU - typically2. Memory – when large number of services3. Disk – Image Service, Synchronization4. Network – low bandwidth deployment5. Poorly configured virtualization can result in 30% or higher performance degradation
Most systems are CPU bound
Most well-configured and tuned GIS systems are CPU bound.
![Page 15: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/15.jpg)
InfrastructureMemory requirements
Wide ranges of memory consumptions
Item Low HighArcSOC Map 50 MB 500 MB
ArcSOC Image 20 MB 1,024 MB
ArcSOC GP 100 MB 2,000 MB
XenApp Session 500 MB 1.2 GB
Database Session 10 MB 75 MB
Database Cache 200 MB 200 GB
![Page 16: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/16.jpg)
InfrastructureHyper threading does not double throughput
• System Test case: 4 physical, 8 logical core
![Page 17: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/17.jpg)
System DesignerSolution Architecture design methodology
• Gathering requirements
• Designing
• Capacity: CPU, Network, Memory
• Reporting
![Page 18: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/18.jpg)
System Designer output
![Page 19: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/19.jpg)
DemoSystem Designer
![Page 20: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/20.jpg)
Esri UC 2014 | Technical Workshop |
Performance Testing
![Page 21: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/21.jpg)
Testing Objectives
- Meet Service-Level Agreement (SLA)- Bottlenecks analysis- Capacity planning- Benchmarking different alternatives
![Page 22: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/22.jpg)
Testing process
Application
GIS Services
Infrastructure: Hardware and Software
![Page 23: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/23.jpg)
Required skill set Configuration, Tuning, Testing
Testing
Configuration
Tuning
![Page 24: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/24.jpg)
Tuning methodologyProfile each tier starting from the top
Browser
Web Server
ArcGIS Server
ArcSOC
Total Response Time (t1-t2)
Wait Time
Search & Retrieval Time
Usage Time
ArcSDE/DBMS
t1 t2
![Page 25: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/25.jpg)
FiddlerFiddler measurement approximately 5.2 seconds
![Page 26: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/26.jpg)
ArcGIS Server 10.3.1 Statisticshttp://server.arcgis.com/en/server/latest/administer/windows/about-server-statistics.htm
• Total requests• Average response time• Maximum response time• Timeouts• Maximum running instances• 30 min resolution reports
![Page 27: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/27.jpg)
ArcGIS Server Logshttp://www.arcgis.com/home/item.html?id=90134fb0f1c148a48c65319287dde2f7
![Page 28: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/28.jpg)
System Monitor –ArcGIS Server Statistics
• https://systemmonitoring-emcs.esri.com/#/arcgis/ESLSRV12• User: esridemo• Password: esridemo
![Page 29: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/29.jpg)
Mxdperfstathttp://www.arcgis.com/home/item.html?id=a269d03aa1c840638680e2902dadecac
![Page 30: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/30.jpg)
Oracle TraceCompare elapsed time
Elapsed time slightly changed due to different test runs
![Page 31: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/31.jpg)
Oracle Execution plan
Inefficient spatial index
![Page 32: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/32.jpg)
System Test Tool featuresGIS Test Automation
• ArcGIS Services- Mapping- Feature Service- OGC- Geocoding- Image Service - Network Analyst- Geoprocessing- Tile Cache
• Application Testing• Discipline relevant report
Application
GIS Services
Infrastructure: Hardware and Software
![Page 33: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/33.jpg)
Test tools feature comparison
Tool Cost LearningCurve
OS Metrics GIS Data Generation
GIS Test Automation
Load Runner High High Windows/Linux No No
Visual Studio Medium High Windows No No
JMeter Free High Requires additionalplugin
No No
System Test Free Low Windows/Linux Yes Yes
Tech Support by Esri PS as part of consulting support
![Page 34: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/34.jpg)
Performance testingValue
• Identify bottlenecks• Determine system capacity• Demonstrate performance SLA
![Page 35: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/35.jpg)
DemoSystem Test
![Page 36: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/36.jpg)
Esri UC 2014 | Technical Workshop |
System Monitoring
![Page 37: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/37.jpg)
Monitoring Enterprise GISChallenges
• Multiple administrators • Multiple disparate monitoring/diagnostic tools• Data collected in a reactive fashion: on demand and for limited time• Correlation of data with different timestamp is difficult• ArcGIS administrators do not have access to all tools, data and reports• Challenging to quickly identify the root cause and take appropriate measures
![Page 38: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/38.jpg)
• Many excellent monitoring tools on the market• Few provide GIS dashboards• System Monitor can be used as reference implementation
Standards for effective GIS monitoring
![Page 39: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/39.jpg)
Enterprise GIS effective monitoring“PIECE” of mind with System Monitor
• Proactive• Integrated
- Dashboards across all tiers
• End-to-End- All tier monitoring
• Continuous- %Coverage provided
• Extendable- Custom queries
Network
Hardware
Web Server
ArcGIS Server
Geodatabase
RDBMS
![Page 40: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/40.jpg)
Monitoring Value
• Proactive validation: Configuration
Resource Utilization
Usage Trends
Performance SLA
Uptime SLA
![Page 41: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/41.jpg)
Demohttps://systemmonitoring-emcs.esri.com/Portaluser: esridemopassword: esridemo
![Page 42: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/42.jpg)
Esri UC 2014 | Technical Workshop |
Applied use of System Monitor and Test tools
Use Cases
![Page 43: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/43.jpg)
DemoSimulate CPU spike (e.g. Antivirus scan)
![Page 44: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/44.jpg)
DemoSimulate ArcGIS user load
![Page 45: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/45.jpg)
DemoIntermittent slow performance: Networks latency (SM extensions)https://systemmonitoring-emcs.esri.com/#/url/eslsrv12_6080_arcgisoutput_1mb_file
![Page 46: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/46.jpg)
DemoArcGIS Server services stopped
![Page 47: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/47.jpg)
DemoArcGIS Server service usagehttps://systemmonitoring-emcs.esri.com/apps/dashboard_sample/#/ArcGIS
![Page 48: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/48.jpg)
DemoUnbalanced resources:ArcGIS Server clustershttps://systemmonitoring-emcs.esri.com/apps/dashboard_sample/#/ArcGISConfig
![Page 49: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/49.jpg)
DemoArcGIS Server slow serviceshttps://systemmonitoring-emcs.esri.com/apps/dashboard_sample/#/ArcGIS
![Page 50: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/50.jpg)
DemoArcGIS Server service usagehttps://systemmonitoring-emcs.esri.com/apps/dashboard_sample/#/ArcGIStuning
![Page 51: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/51.jpg)
DemoArcGIS Server instance configurationhttps://systemmonitoring-emcs.esri.com/apps/dashboard_sample/#/ArcGISInstances
![Page 52: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/52.jpg)
Thank you…
• Please fill out the session survey in your mobile app• Select [enter session title here] in the Mobile App
- Use the Search Feature to quickly find this title
• Click “Technical Workshop Survey”• Answer a few short questions and enter any comments
![Page 53: Enterprise GIS: Performance and Scalability · 2015-07-30 · •Multiple administrators • Multiple disparate monitoring/diagnostic tools • Data collected in a reactive fashion:](https://reader031.vdocument.in/reader031/viewer/2022011816/5e739493b0994163773053ef/html5/thumbnails/53.jpg)