ArcGIS Pro SDK for .NET: Pro Application Architecture Overview

and API Patterns

RUSSELL LOUKS

Motivations

Desktop in 2012• Separate non-integrated applications

• 32 bit, single threaded architecture

• Slow 2D/3D graphics

• SDI, context free UI with toolbars

• Monolithic, COM heavy, Admin Only

• Complex API – poor integration with .NET / Java

• Deficiencies in cartographic data model and project management

ArcGIS Pro - Synthesis

Managed / Native hybrid application

• 64 bit Engine level ArcObjects

• ArcGIS Runtime

• BGL Graphics Engine

• CIM (Cartographic Information Model)

• WPF

• New .NET API

• Extensibility via Add-Ins

Enhanced GUI

• Ribbon / Tabs

• Multiple detachable/dockable views

• Flexible docking framework

• Galleries

• Contextual UI

Contextually driven application

• Functional areas are loaded on demand

• UI elements are shown and enabled when relevant

• Statically specified rules can be used to control context

UI Elements

• Buttons, Tools, Checkboxes

• Combo Boxes, Edit Boxes, Spinners

• Menus, Context Menus, Dynamic Menus

• Galleries, Button and Tool Palettes, Split Buttons

• Tabs, Groups, Contextual Groups

• Property Pages/Sheets

• Wizards

• Views and Docking Panes

• Custom (XAML) controls

• Backstage Tabs, Backstage Button

MVVM – Model, View, ViewModel

• Design pattern used to separate implementation aspects

• Flexible, Testable

• Well suited for threaded applications

• Central to the ArcGIS Pro SDK

Extensibility

• Python scripts and tools

• .NET/WPF Add-In model

- DAML extensibility framework

- Modern, simplified .NET API

• User/Scenario based configuration/restrictions

- Configurations

Modules

• Similar to Extension objects in 10.x

• Singleton component that forms a functional hub

• Groups Add-In elements

• API Entry point to various subsystems

• Bindable

• Extension info, licensing

<DAML>

• An XML markup language for extensibility and UI compositing

• Controls how Add-In UI elements are logically arranged within shared regions such

as the ribbon and docking areas.

• Controls how and when Add-In functionality is loaded

• Categories: declarative extensibility mechanism

• Uses insert, update, delete scheme

<insertModule id="acme_mod" >

<controls>

<button id="example1" className="acme_mod:OnClick" caption="Test"/>

</controls>

</insertModule>

Context - Conditions and State

• State: A named Boolean state within the application

• Condition: A named expression consisting of one or more states

<condition name="myCondition">

<and>

<state name="esri_mapping_rasterLayerSelectedState""/>

<state name="esri_mapping_singleTOCItemSelectedState""/>

</and>

</condition>

<button id="myButton" condition="myCondition""/>

Context – Defined Behaviors

• Built-in behaviors driven by Context

- Contextual Tab and Tab group visibility

- Enabled state

- Property Page visibility

- Wizard Page visibility

Context - States and State locality

• Implicit States

- Active Pane type

- Active Tab

- Active Tool

- Module Loaded

• Explicit StatesState.Activate("<some state>")

State.Deactivate("<some state>")

• Locality of state (Global and Pane)

• All esri conditions are listed in the Pro SDK on github

Events

• Traditional event model

• Event Aggregator

- Publishers

- Subscribers

Events - subscribing

• SubscriptionToken _token = null;

• protected override bool Initialize()• {• _token = MapSelectionChangedEvent.Subscribe((MapSelectionChangedEventArgs args) =>• {• // event handling logic goes here…• });• }

• protected override void Uninitialize()• {• MapSelectionChangedEvent.Unsubscribe(_token);• }

Threading Architecture

Threading – common uses

Use case Example within ArcGIS Pro

Improving UI Responsiveness Primary worker thread

Independent background operations Background GP, Background thread pool

Capitalizing on I/O latency, Interleaving BGL loader threads

Multi-processing (divide and conquer) BGL, GeostatisticalAnalyst, Raster functions

Technology Example within ArcGIS Pro

SIMD Raster, various

GPU Compute Spatial Analyst

Threading Architecture – general characteristics

• ArcGIS Pro is a stateful application

- Most operations are order dependent and atomic

- User initiated actions are queued to the primary worker thread

- Multi-processing is leveraged within esri’s implementation

• The GUI thread is never used to perform operations

• Background threads used for independent* operations

• Only simple non-mutable types passed between threads

• Public API based on TPL/TAP with enhancements

Threading - complexities

• Operation ordering

• Concurrency control

• Thread affinity constraints

• Threading and Events

• Data Binding

• Context switches in loops

• UI state changes

• Progress & cancellation

Threading – Standard TPL / TAP

// Implementing FooAsyncTask<double> FooAsync(double[] vec){return Task.Run<double>(()=>{return Foo(vec);

});}

// Calling asynchronous functionspublic async void OnClick(){var tasks = new Task<double>[]{ FooAsync(), BarAsync() };await Task.WhenAll(tasks);

// Return here when tasks complete.ShowResults(tasks[0].Result, tasks[1].Result);

}

Threading - Operation Ordering

Standard TAP : Unregulated execution

Task.Run(()=>{ Foo();});

Task.Run(()=>{ Bar();});

MS Thread Pool

Thread

Thread

Foo(){ ...});

Bar(){ ...});

T1 T2T0 T3

Logical

Conflict

Time

Threading - QueuedTask

Task<double> FooAsync(double[] vec){return QueuedTask.Run<double>(()=>{return Foo(vec);

});}

• Ensures operations are ordered and atomic

• Controls UI enabled state

• Provides mechanism for progress and cancellation

• Integrated diagnostic facilities

Threading - Operation Ordering

QueuedTask : Coordinated execution

Time

Threading - Operation Ordering

QueuedTask : Hardware leveraged internally

Threading - Thread affinity

• Exceptions (wrong thread) when violated

• Some entities that have thread affinity

- HWNDs, timers, input queues

- WPF GUI elements (DependencyObject derived)

- COM objects

- Database objects

- Bitmaps (Freezable objects)

QueuedTask.Run(() =>{BitmapImage image = processData();var freezable = image as System.Windows.Freezable;if (freezable != null && freezable.CanFreeze)freezable.Freeze();

});

Threading - Concurrency Control

• Mutable shared data must be protected by a lock

• Fine grained locking expensive

• Thread affinity as pseudo lock (apartment model)

• Deadlock risk

• Don't loiter within a lock

Threading - Events

• Events are usually invoked on consistent threads

- Events associated with GUI actions: GUI thread

- Events associated worker thread operations: worker thread

• GUI thread event handlers should not block

Threading: WPF data binding

ObservableCollection<T>

Exception!

Threading - Thread safe binding

• WPF collection synchronization using a lock

• Both parties agree to entering the lock when reading or writing collection

internal class ExampleVM

{

private ObservableCollection<string> _vmList = new ObservableCollection<string>();

private Object _lock = new Object();

public ExampleVM()

{

BindingOperations.EnableCollectionSynchronization(_vmList, _lock);

}

Task UpdateListFromData(){return QueuedTask.Run(() =>{

lock (_lock) { _vmList.Add(getData()); }});

}}

Threading - WPF data binding with concurrency control

ObservableCollection<T>

Threading - Thread safe binding

• Minimize time in lock

• Perform lengthy operations outside lock and batch update in lock

• Beware of combining with collection changed events

- ArcGIS.Desktop.Internal.Framework.Utilities.BulkObservableCollection

Threading: Context switching in loops

• High frequency context switching

impacts performance.

• QTR will in-line async calls if possible.

• Call synchronous method in QTR if

available.

Threading: Context switching in loops

private async void OnClick(){

while(true){

// Thread context switch on eachRecord rec = await ReadNextAsync(); if (rec.End)break;

}}

private async void OnClick(){

await QueuedTask.Run(async ()=>{

while (true){// Inlined (no context switch)Record rec = await ReadNextAsync(); if (rec.End)

break;}

});}

High Overhead

Low Overhead

Threading: Context switching in loops

private async void OnClick(){

await QueuedTask.Run( ()=>{while (true){Record rec = ReadNext(); if (rec.End)break;

}});

}

No Overhead

Threading – UI State changes

• Consider appropriate state while your operation is running

- ArcGIS.Desktop.Framework.RelayCommand

- DAML attribute disableIfBusy = "true" (default)

• Do not assume that the UI state (selected items, etc.) will remain static

- Task may remain queued for a while

- Always take a "state snapshot" of whatever you need

Configuring Progress & Cancellation

• Progress State

- Message, Status, Value

• ProgressorSource

• CancelableProgressorSource

Implementing an asynchronous method with progress and cancellation

public Task<long> CalcFactorialAsync(int x, CancelableProgressor progressor)

{

return QueuedTask.Run<long>(() =>

{

long result = 1;

for (int i = 1; i < x; ++i)

{

if (progressor.CancellationToken.IsCancellationRequested)

throw new OperationCanceledException();

progressor.Message = $"Working on step:{i}";

result *= i;

}

return result;

}, progressor);

}

Using an asynchronous method with progress and cancellation

public async void OnClick()

{

CancelableProgressorSource cps = new CancelableProgressorSource("Computing n!", "Cancelling...");

long result = -1;

try

{

result = await CalcFactorialAsync(20, cps.Progressor);

}

catch(OperationCanceledException)

{

MessageBox.Show("Task responded to cancellation request.");

return;

}

MessageBox.Show($"Task completed, result = {result}");

}

Progressor Options

ProgressorSource(Action<Progressor> callback)

ProgressorSource(ProgressDialog progDlg)

ProgressorSource(string message, bool delayedShow = false)

• Progressor.Current

• Visibility heuristics

• Progress Dialogs are not shown when debugging

Threading: Embedded diagnostics

ArcGIS Monitor (ArcMon)

• Application counters

• Task list

• GUI Hang log

• Thread information

• HTTP information

• Left Ctrl + Left Alt + M

ArcMon – Important Indicators

• Hung

- Lit if the GUI thread has become unresponsive for more than 350ms.

• Task Rate

- Lit if the current queuing frequency has exceeded ten Tasks/second.

• HTTP

- Lit if an one or more HTTP requests are currently waiting.

• Task Busy

- Lit if a Task is currently executing on the main foreground worker thread.

Experimental - Internal APIs useful for threading problems

Not documented but making their way towards a public release

• Binding + CollectionChangedEvents

ArcGIS.Desktop.Internal.Framework.Utilities.BulkObservableCollection<T>

• Background thread pool compatible with ArcGIS.Core components

ArcGIS.Core.Internal.Threading.Tasks.BackgroundTask

• Idle action queue

ArcGIS.Desktop.Internal.Framework.FrameworkApplication.QueueIdleAction

Further Reading

• Task Asynchronous Pattern - MSDN

• https://msdn.microsoft.com/en-us/library/hh873175%28v=vs.110%29.aspx

•

• Best practices in Asynchronous Programming – Stephen Cleary

• https://msdn.microsoft.com/en-us/magazine/jj991977.aspx

•

• The MVVM design pattern - MSDN

• https://msdn.microsoft.com/en-us/library/hh848246.aspx

ArcGIS Pro SDK for .NET Tech Sessions

Date Time ArcGIS Pro SDK for .NET Tech Sessions Location

Wed, Mar 072:30 pm - 3:30 pm Pro Application Architecture Overview & API Patterns Mesquite G-H

4:00 pm - 5:00 pm Advanced Editing and Edit Operations Santa Rosa

Thu, Mar 089:00 am - 3:30 pm Getting Started Hands-On Training Workshop Mojave Learning Center

5:30 pm - 6:30 pm Working with Rasters and Imagery Santa Rosa

Fri, Mar 09

8:30 am - 9:30 am An Overview of the Utility Network Management API Mesquite G-H

10:00 am - 11:00 am Beginning Pro Customization and Extensibility Primrose A

1:00 pm - 2:00 pm Advanced Pro Customization and Extensibility Mesquite G-H

ArcGIS Pro SDK for .NET Demo Theater Sessions

Date TimeArcGIS Pro SDK for .NET Demo Theater

PresentationLocation

Tue, Mar 061:00 pm - 1:30 pm Getting Started Demo Theater 1 - Oasis 1

4:00 pm - 4:30 pm Custom States and Conditions Demo Theater 2 - Oasis 1

Wed, Mar 075:30 pm - 6:00 pm New UI Controls for the SDK Demo Theater 2 - Oasis 1

6:00 pm - 6:30 pm Raster API and Manipulating Pixel Blocks Demo Theater 2 - Oasis 1

ArcGIS Pro Road Ahead Sessions

Date TimeArcGIS Pro SDK for .NET Demo Theater

Presentation Location

Tue, Mar 06 4:00 pm – 5:00 pm ArcGIS Pro: The Road Ahead Oasis 4

Thu, Mar 08 4:00 pm – 5:00 pm ArcGIS Pro: The Road Ahead Primrose B