data management and migration in salesforce

Data Management and Migration in Salesforce

Presented By: Sunil KumarEmail id: [email protected] : @sunil02kumarBlog: https://sunil02kumar.blogspot.in/

mailto:[email protected]

Application Building Blocks

Visual force PagesWeb Controls

Force.com Sites

Applications Tabs

Page LayoutsRecord Types

WorkflowValidation Rules

Assignment Rules

Visual force ControllersApex

Web Services API

ObjectsFields

Relationships

Web Services APIMetadata API

User Interface

Business Logic

Data Model

Declarative Programmatic

Agenda• Essential Data Management Operations• Record IDs• Object Relationships• Modifiable System Fields• Upsert using External Ids• Data Management Tools• Bulk API

Essential Data Management Operations Exploring Data -Get reference IDs for inserts/updates

-Download records for backup

Inserting Data-Load Users-Load existing Positions

Updating Data-De-duplicate existing data-Introduce Record Types

Deleting Data-Free up space used by too much legacy data-Fix mistakes!

Record IDs• Unique identifier of a record.• Analogous to a primary or foreign key field in a database

table.• Salesforce generates an ID value when a new record is

created i.e. a00D0000005iTiZ

IDs can be obtained 3 ways:•1.URL•2.Report•3.Web Services API, e.g., the Data Loader

Record IDs (cont..)• Salesforce.com Object IDs come in 2 forms:

-15digit case-sensitive form-18-digit case-insensitive form

• Reports and Object IDs-Reports(and office Edition)return 15-digit IDs-Report framework does not expose IDs for all objects

• API and Object IDs-API always return 18-digit IDs-API will accept either the 15-digit or 18-digit format.

Object Relationships• Relationships exist between objects, for example:

-All Positions have an owner-Candidates are related to Position through a Job Application-Reviews are associated to a Job Application

• Relationships are expressed through:-Related lists and lookups in the application-IDs(foreign keys) in the database

Modifiable System Fields• What Does It Do?

-Allows you to set Created Date,Created By,Last Update Date, Last Update By-Useful for migrating date from external systems and preserving history-Generally, these fields are Read-Only

• How Do You Get It?-Contact salesforce.com-Customer Support will enable

Modifiable System Fields (cont..)• Only accessible through API• Backwards compatible with all SOAP-based API’s.• Restricted to Account,Opportunity,Contact,Lead, Case, Task, and Event Standard Objects.• Works for all Custom Objects.• Read-only for existing records.

Best Practice: enable only for initial data migration

Upsert using External Ids• Upsert• External IDs• Typical Use cases Where Upsert is Useful• Upsert with Relationships

Upsert using External Ids (cont..)• Upsert is an API function that combines insert and

updates into a single call.• Upsert uses an indexed custom field or external ID to

determine whether to create a new object or update an existing object.

-If the external ID is not matched, then a new object is created-If the external ID is matched once, then the existing object is updated-If the external ID is matched multiple times, then an error is reported

• Use Upsert when importing data to prevent the creation duplicates.

Upsert using External Ids (cont..)• External ID is a flag that can be added to a custom field to

indicate that it should be indexed and treated as an ID.• Custom index on any custom field of type Text,Number or

Email.• Available on all objects that support custom fields.• User-defined cross-reference field.• Why is it important?

-Increase report and API SOQL performance-Used with upsert to easily integrate apps with other systems

• An object have three(3) External ID fields.

Upsert using External Ids (cont..)

• Typically used for migrations and integrations between Salesforce and other systems

• Useful to have external foreign key to link data in both systems

• Helps in auditing flow of information across systems

Upsert using External Ids (cont..)

• Allows use of relationships defined in legacy systems when importing data into Force.com apps.

• Configure upsert action to traverse object relationships defined in your Force.com app, but use external IDs from legacy system to discover Force.com record IDs.

• No need to know Force.com record IDs to load data!• Very convenient for integrations and migrations

-Shifts more of the work to Salesforce.

Data Management Tools• Via Application Import Wizards

Accounts Contacts Leads Solutions Custom Objects

• Via Web Services APIData Loader Partner ToolsCustom-built ToolsOpen Source Tools

Data Management Tools (cont..)• Import Wizards

-Easy to use tool to load Accounts,Contacts,Leads.Solutions, or Custom Objects.

-Load 50,000records or less.-Prevent duplicates:

Account Name and Site; Account IDs Contact Email Address; Contact Name; Contact ID Lead Email Address; Lead IDs

Data Management Tools (cont..)• The Apex Data Loader:

-Is a fully supported salesforce.com product.-Supports import from CSV or export to CSV.-Supports loading from or exporting to a database via JDBC.-Supports custom relationships for upsert.-Can be run from command line.-Load more than 50,000 records.-Schedule regular data loads such as nightly feeds.-Export data for backup.-Mass delete supported objects

Bulk APIThe Bulk API for high-volume data loads is designed to:• Improve throughput when loading large data sets into

Salesforce.• Increase stability, monitoring, and control of high-volume

data loads.• Allow better control and optimization of data center

resources during data loads.

Data is transferred The whole data set is managed The data set can be at full network speeds, in a job that can be monitored

processed faster byreducing dropped and controlled from Setup allocating multipleconnections. menu. Servers to process

in parallel.Client Job Processing Servers

.

.

.

Send all data to server in large

batches

Check Status

Retrieve results

Data batchData batchData batch

Results

Results

Results

Dequeue batch

from job

Insert or update records

Save results to

job

Bulk API (cont..)

Bulk API (cont..)

The Data Loader uses the SOAP-based Web Services API by default. To use the Bulk API instead.

1. Open the Data Loader client.2. Click Settings | Settings3. Select the Use Bulk API checkbox.4. Click OK.There is an additional Enable serial mode for Bulk API option.When the Bulk API is selected, users with the Bulk API Hard Delete permissionCan perform a hard delete.

Hard delete bypasses the Recycle Bin, so deleted records do not consume storage space and there is no way to retrieve them

Using Data Loader with the Bulk API

Using Data Loader with the Bulk API

• Monitoring bulk data loads requires the “Manage Data Integrations” permission.

• To track the status of bulk data load jobs, click Setup | Monitoring |Bulk Data Load Jobs.

• The In Progress Jobs list contains:-JobId,object, Operation,Progress,Records Processed,Start Time,Status,Submitted By.

• The Completed Jobs list contains:-End Time, Job ID, Object, Operation,Progress,Records Processed,Start Time,Status, Submitted By, Time to Complete.

Bulk API (cont..)

data management and migration in salesforce

Software