geo database replication

Geodatabase Geodatabase ReplicationReplication

An Introduction to

An Introduction to

Jay Meehl Hennepin County GIS DivisionJay Meehl Hennepin County GIS Division

Geodatabase (GDB) replication is a data Geodatabase (GDB) replication is a data distribution method offered in ArcGIS 9.2. distribution method offered in ArcGIS 9.2. When GDB replication takes place data is When GDB replication takes place data is distributed (copied) across 2 or more GDBs by distributed (copied) across 2 or more GDBs by extracting all or part of your dataset. extracting all or part of your dataset.

Definition:Definition:

SDE GDBSDE GDBSDE GDBSDE GDB

PSNL PSNL GDBGDB

••The replica in the source GDB is called the The replica in the source GDB is called the parentparent replica and the replica in the target replica and the replica in the target GDB is called the GDB is called the childchild replica.replica.

••GDB Replication is built on the GDB Replication is built on the versioningversioning technology replicationtechnology replication……

••The Create Options requires a ArcGIS Editor The Create Options requires a ArcGIS Editor licensinglicensing or betteror better……

Good to Know:

Good to Know:

Reasons.. Reasons..

1.1. Distribution of single Distribution of single organizationorganization--wide wide GDB across different GDB across different offices or locations.offices or locations.

Why Replicate?

Why Replicate?

2. Used as central hub for both readers and 2. Used as central hub for both readers and viewers of the GDB. Assists with load viewers of the GDB. Assists with load balancing.balancing.

Graphics from ESRI publication

Reasons continuedReasons continued……

3. For disconnected editing by 3. For disconnected editing by mobile users.mobile users.

4. Failover Systems4. Failover Systems

5. Working with Contractors5. Working with Contractors

Why Replicate?

Why Replicate?

Graphics from ESRI publication

In order to replicateIn order to replicate……

1. All data must be registered as versioned. 1. All data must be registered as versioned. The data cannot be versioned with the option The data cannot be versioned with the option to move edits to base.to move edits to base.

2. The database user must have write access 2. The database user must have write access to the data.to the data.

Requirements:

Requirements:

In order to replicateIn order to replicate

3. Each dataset must have a 3. Each dataset must have a Global IDs Global IDs columncolumn. This column is used to maintain row . This column is used to maintain row uniqueness across geodatabases.*uniqueness across geodatabases.*

4. All spatial data must be stored in a 4. All spatial data must be stored in a high high precision precision spatial reference.*spatial reference.*

* Required for some types of replication.* Required for some types of replication.

Requirements:

Requirements:

Global IDs?

Global IDs?

The Global ID field type contains a The Global ID field type contains a unique string of characters which is unique string of characters which is necessary for maintaining object necessary for maintaining object uniqueness across replicas. All feature uniqueness across replicas. All feature classes and tables participating in classes and tables participating in oneone--way or twoway or two--way replication must way replication must contain the Global ID field. contain the Global ID field.

Warning!Warning!

You must strip the You must strip the Global ID off before Global ID off before exporting a GDB FC exporting a GDB FC to shapefile.to shapefile.

What can be replicated?What can be replicated?•• Full data setFull data set•• Selected setSelected set•• Definition QueriesDefinition Queries•• Just the schemaJust the schema•• Related data (relationship class)Related data (relationship class)•• DomainsDomains

What?What?

RequirementsRequirements1.1. Database Connections within Database Connections within

ArcCatalog ArcCatalog

•• LAN (Local Area Network)LAN (Local Area Network)

2.2. WAN (Wide Area Network)WAN (Wide Area Network)

3.3. Internet Internet

4.4. Internet with VPN (Virtual Private Internet with VPN (Virtual Private Network)Network)

5.5. Export to XML Document for Export to XML Document for transmissiontransmission

Connections:

Connections:

ArcGIS ArcGIS Server Server Geodata Geodata ServicesServices

Geodata Services Geodata Services provide a way to provide a way to perform GDB perform GDB replication operations replication operations on remote GDBs by on remote GDBs by accessing ArcGIS accessing ArcGIS Server through the Server through the LAN, WAN, or Internet.LAN, WAN, or Internet.

ArcGIS Server:

ArcGIS Server:

Graphics from ESRI website

Replica Types:

Replica Types:

1.1. Checkout / Check InCheckout / Check In

•• PSNL/FILE/SDE/PSNL SDEPSNL/FILE/SDE/PSNL SDE

2.2. One WayOne Way

•• SDE/PSNL SDESDE/PSNL SDE

3.3. Two WayTwo Way

•• SDE/PSNL SDESDE/PSNL SDE

The Parent The Parent must be a must be a SDE GDBSDE GDB

Server with Server with SDE SDE GeodatabaseGeodatabase

PC with SDE, PC with SDE, PSNL, or File PSNL, or File GeodatabaseGeodatabase

Check Out/ Check Out/ Check In Check In

ReplicationReplication

1.1. Replicate GIS Layer From Replicate GIS Layer From Parent to ChildParent to Child

ChildChild

Check OutReplicateReplicate

The data or portion of the The data or portion of the data is copied to the data is copied to the child.child.

ParentParent

Check In




Check OutReplicateReplicate

2. Make edits in the field2. Make edits in the field……


ParentParent

ChildChild

Check In



ReplicationReplicationSynchronize

Synchronize

3. Synchronize Child to Parent3. Synchronize Child to Parent


ParentParent

ChildChild

Once the data from the child has been synchronized Once the data from the child has been synchronized additional edits can not be synchronize without additional edits can not be synchronize without creating a new check out replica.creating a new check out replica.

Check Out

Check In



ReplicationReplicationSynchronize

Synchronize

4. Possible reconcile of data 4. Possible reconcile of data conflictsconflicts


ParentParent

ChildChild

Check Out

Check In

Server with Server with SDE GDBSDE GDB

One Way One Way ReplicationReplication

1. Create a replica or copy of 1. Create a replica or copy of the parent data and send it to the parent data and send it to the child only.the child only.

SDE or PSNL SDE or PSNL SDE GDBSDE GDB

ChildChildCreate a Replica Create a Replica (send original data)(send original data)

ParentParent

Check Out

Check In



Two Models..Two Models..

Full Full –– Complex data types Complex data types and child data versioned.and child data versioned.

Simple Simple –– Simple data types Simple data types and child is not versioned.and child is not versioned.



ParentParent

Check Out

Check In





DisconnectedDisconnected


2. Parent data can also be copied 2. Parent data can also be copied to child through disconnected to child through disconnected replication using XML Documentsreplication using XML Documents

ParentParent

Check Out

Check In




ChildChildSynchronize Synchronize ChangesChanges

ExportExport

ChangesChanges

3. Parent data is edited and synchronized 3. Parent data is edited and synchronized with the child. For disconnected replicationwith the child. For disconnected replication changes are exported in XML.changes are exported in XML.

ParentParent

Check Out

Check In




ChildChildSynchronize Synchronize ChangesChanges

3. Synchronization can be done between two 3. Synchronization can be done between two disconnected GDBs via the internet using disconnected GDBs via the internet using ArcGIS Server running Geodata Services.ArcGIS Server running Geodata Services.

ParentParent

GeodataGeodata

ServicesServices

Check Out

Check In



4. There is no limit on the 4. There is no limit on the number of one way number of one way synchronizations that take place.synchronizations that take place.


ChildChild

SynchronizationSynchronization

ParentParent

SynchronizationSynchronizationSynchronizationSynchronization

Check Out

Check In


Two Way Two Way ReplicationReplication

1. Replicas can be sent from 1. Replicas can be sent from Parent to Child or Child to Parent to Child or Child to Parent.Parent.


ChildChild

ParentParent

Create a Replica Create a Replica (send original data)(send original data)

Check Out

Check In


4. Disconnected synchronization 4. Disconnected synchronization can be done with XML or Geodata can be done with XML or Geodata Services.Services.


ChildChild

ParentParent

Xml or Geodata Xml or Geodata ServicesServices

Two Way Two Way ReplicationReplication


EXAMPLEEXAMPLE

Replication can be Replication can be done from ArcCatalog, done from ArcCatalog, ArcMap, or scripts. ArcMap, or scripts. Specify data (or Specify data (or subsets), the replica subsets), the replica type, where you would type, where you would like to replicate the like to replicate the data to, name, and data to, name, and optional parametersoptional parameters


EXAMPLEEXAMPLE

Make Make changes to changes to the geometry the geometry or attributes.or attributes.


EXAMPLEEXAMPLE

Synchronize the Synchronize the changes. In other changes. In other words send the words send the parent changes to the parent changes to the child. In the case of child. In the case of two way or check out two way or check out replication it could be replication it could be sending changes to sending changes to the parent.the parent.


EXAMPLEEXAMPLE

In this one way example changes were In this one way example changes were sent to the child and reflected in the sent to the child and reflected in the other geodatabase.other geodatabase.


EXAMPLEEXAMPLE

If disconnected If disconnected replication was replication was necessary then necessary then changes would be changes would be exported to XML exported to XML documents and documents and sent.sent.


EXAMPLEEXAMPLE

To change the To change the child geodatabase, child geodatabase, the changes would the changes would need to be need to be ““importedimported”” from from the XML the XML document.document.


EXAMPLEEXAMPLE

In either case you must specify how In either case you must specify how conflicts are handled.conflicts are handled.


EXAMPLEEXAMPLE

The XML changes are incorporated into the The XML changes are incorporated into the data.data.

1. Determine what replicas are needed1. Determine what replicas are needed2. Determine what type of replication to use2. Determine what type of replication to use3. Choose the toolset (models, scripts, AM/AC)3. Choose the toolset (models, scripts, AM/AC)4. Integrate replication in workflows.4. Integrate replication in workflows.5. Define the data to replicate5. Define the data to replicate6. Consider replica creation options6. Consider replica creation options7. Decide on Connected or disconnected 7. Decide on Connected or disconnected environmentenvironment

ESRI recommends

ESRI recommends

Automate:Automate:

You can automate You can automate the replication the replication process by process by creating Python creating Python scripts and scripts and scheduling them scheduling them nightly or weekly.nightly or weekly.

Distribution:

Distribution:

Hennepin County has no current policy on Hennepin County has no current policy on implementing any form of geodatabase implementing any form of geodatabase replication. replication.

We are testing geodatabase replication in We are testing geodatabase replication in conjunction with ArcGIS Server Geodata conjunction with ArcGIS Server Geodata Services. As options are investigated Services. As options are investigated and tested we will have a better idea of and tested we will have a better idea of how and what will be served to cities.how and what will be served to cities.

Distribution:

Distribution:

There are a number of issues includingThere are a number of issues including……

1.1. SDE to SDE for One Way and Two Way SDE to SDE for One Way and Two Way ReplicationReplication

2.2. Internet data access permissionsInternet data access permissions

3.3. Geodata Services without replication for those Geodata Services without replication for those using PSNL or FILE GDBs.using PSNL or FILE GDBs.

4.4. Schedules and customized, individual Schedules and customized, individual download extents or areas. download extents or areas.

Distribution:

Distribution:

ESRI staff have indicated that in ArcGIS ESRI staff have indicated that in ArcGIS 9.3 SDE will not be required for One 9.3 SDE will not be required for One Way.Way.

““Microsoft Access personal geodatabases Microsoft Access personal geodatabases and file geodatabases can participate in and file geodatabases can participate in oneone--way replication as child way replication as child geodatabases.geodatabases. ““

http://www.esri.com/software/arcgis/geodatabase/about/whatshttp://www.esri.com/software/arcgis/geodatabase/about/whats-- coming.htmlcoming.html

Distribution:

Distribution:

What general distribution options would What general distribution options would be availablebe available……..

•• One Way One Way –– Cities required to have SDECities required to have SDE

•• One Way One Way –– Cities that run Cities that run PsnlPsnl SDE SDE using using SqlSql Server Express (free)Server Express (free)

•• One Way One Way –– Wait for 9.3 for those with Wait for 9.3 for those with PSNL GDBs or File GDBsPSNL GDBs or File GDBs

•• Full Download Full Download –– Using Geodata ServicesUsing Geodata Services

ResourcesResources

• Working with GDB Replication – ArcGIS 9.2 Help Pages

• Introduction to GDB Replication in ArcGIS 9.2 – Free Recorded Live Training Seminars November 9, 2006

• An Overview of Distributing Data with GDB – Whitepaper June 2007

• ESRI Systems Integration Technical Brief – ArcSDE GDB Replication Overview – Whitepaper May 2005

Conclusion:

Conclusion:

Any questions or comments?Any questions or comments?