dell bhomi tool 9ways
TRANSCRIPT
-
8/17/2019 Dell Bhomi Tool 9Ways
1/15
9 Ways To Bulletproof Your Boomi Processes
-
8/17/2019 Dell Bhomi Tool 9Ways
2/15
A Bulletproof Process
I have worked with hundreds of Dell Boomi Atomsphere processes since 2010,and prior to that worked with a number of other data integration tools and ETL
processes. Over that time, my understanding of what causes errors in integration
processes and how to prevent them has grown. In this white paper, I’ll talk about the
tools that are available to the Boomi developer to prevent and detect process errors
- the tools you need to make your Boomi Processes Bulletproof.
What are the Bullets we must proof our integration processes against? In my
experience, nine times out of ten, the Bullets are Data. Data that is wrong, missing,new, out of range, not found in the destination system, or in some other way
unexpected. After all, the movement of data from one system to another as part of
an overall business process is why Integrations exist. And it is the variations in the
source data encountered that are the Bullets which threaten to cause the Integration
to Fail.
A Bullet-proof Integration process should handle as wide a variation in expected
source data as possible, and should fail gracefully when presented with Data that
cannot be handled. The Bullet-proof process should handle expected or anticipated
variation in data without Process-level failure. The Bullet-proof process should notify
a User who can x the data problem when the problem is not systemic. The Bullet-
proof process should notify the User so that the error message is closely associated
with the record or object that failed, ideally right on the failing record itself. The
Bullet-proof process should leave a trail of breadcrumbs to allow diagnosis and
debugging, and not fail quietly or in a way that does not alert responsible users.
-
8/17/2019 Dell Bhomi Tool 9Ways
3/15
Techniques & Tools
The techniques and suggestions described here are tools that you can use in Boomi
integrations to:
Prevent normal variations in source data from causing the
integration to fail. An example would be when multiple addresses
in a Netsuite Contact cause multiple output records.
Notify Users about non-fatal data problems, and support the
timely correction of the source data. An example would be a
missing required eld.
Support IT staff in diagnosing operational problems. An example
would be when a User calls, and asks if you can conrm that
order 123 was pushed over the integration yesterday. If the right
tracking elds are in place, you can search all of yesterdays
activity for order 123, nd the execution that passed the order, and
dig into the details of the source record, or even re-run the order in
debugging mode.
These techniques make the difference between a merely adequate integration, and
a robust, maintainable part of the business process.
-
8/17/2019 Dell Bhomi Tool 9Ways
4/15
Error DetectionAfter Updates
This is the basic bulletproong technique - your
process should look at the return codes from
any update step, and notify as appropriate. In
many connectors, you have an option to have the
connector return an error, rather than failing the
process. This is most applicable for API updatesinto complex cloud applications.
For instance, Netsuite has a sophisticated
connector in Boomi that gives the process access
to standard and custom objects and elds. With
the Setting “return all errors = yes”, the Netsuite
connector will return a status document from any
call, with success = false when an error occurs.You can make an xml prole to capture the return
codes from your Netsuite update call, and test the
return code in a decision step.
Then use the error messages provided in the
return document as part of your notication
message. Your process will then detect errors in
an API call, and provide useful error messages to the right people so that the error
can be corrected. It is good practice to post the error back to the source object, if
possible, to alert the End User to the problem. As most integration run-time errors
are data problems, the End User can often x the problem if notied promptly and
clearly.
-
8/17/2019 Dell Bhomi Tool 9Ways
5/15
Try Catch to Trap Errors
This is a relatively new and welcome error detection and trapping component. Try-Catch
lets you go down an error branch if any subsequent process or process step fails. Before
this feature, a process error would just stop the process, and you would need
to go in and diagnose from process logs. Many update
types including the much-used Process step do not fail
gracefully, and don’t allow error trapping. The Process
step is the most effective way to update database tables
by running SQL fragments that update specic elds.
Try Catch lets you detect process errors at any
downstream step, and send the failing document down
an error branch, where notications and updates can
occur.
Try-Catch provides a document property that allows you
to capture and report the error that caused the failure.
The only downside to Try-Catch is that you lose the
details of the failing step. This is because the logs after
the Try-Catch has switched the document to Catch reflect the processing that happened
in the Catch branch, and the document details of the error branch are lost.
-
8/17/2019 Dell Bhomi Tool 9Ways
6/15
Smart XML Profles
With cloud application process
endpoints like Netsuite and
Salesforce, the record layouts, or
proles, are often complex XML
documents. Many errors occur due
to mapping issues between complex
XML documents with multiple sub-
records. An example is the Netsuite
Contact prole, which can generatemultiple output documents from
a map if the contact has multiple
occurrences of the Address Book
sub- component. Use Qualiers and
Identiers within your XML proles to
make sure that you map to only the
occurrence of a sub-object that you
want.
Setting up a Qualier on the default flag of the Address Book object will insure
that you only map to a single, preferred Address in the Contact object. See the
Atomsphere reference articles about Identiers and Qualiers to learn about these
powerful prole tools.
Then notify the appropriate parties with information that will help them diagnose
the error situation and recover the processing. There are a few ways to do this - the
Notify step allows you to create Boomi platform alerts, warnings, and errors, and
use the Boomi platform to send these alerts.
-
8/17/2019 Dell Bhomi Tool 9Ways
7/15
Boomi Suggest
Many errors have their source in mapping problems. Should I map to the Status,
the Category or the Type eld? Which of the several description or memo elds
should I use? Does the current date/time go into that eld, or the date/time from the
incoming record?
If you are working with common prole layouts in your map, Boomi Suggest can
be very helpful in identifying and mapping common elds. It is both a time saver in
suggesting and mapping common eld connections, and a quality aid in that Boomi
Suggest will identify common mappings that you might otherwise get wrong.Boomi Suggest is statistically driven from hundreds of thousands of maps that
have been created on the Boomi Atomsphere platform. For that reason, it is
most useful working with common layouts like Salesforce object layouts. I have
found it essential in EDI mappings, where many layouts are standardized, but
documentation can be sparse and confusing. In the right situation, Boomi Suggest
can save signicant time in mapping, debugging, and error handling.
-
8/17/2019 Dell Bhomi Tool 9Ways
8/15
Business Rules Step
Based on my informal surveys of Boomi Atomsphere processes in production, the
Business Rules step is under-utilized as an error detection approach. Which is a
shame, as it is specically designed to make error detection easy.
The Business Rules step has several features that make it a powerful error detection
asset. You can set up multiple business rules, each with its own source data, logic,
and error message. If any one of the rules fails, the document is routed down the
failure branch, otherwise down the success branch.
The logic for each individual rule can be quite complex. Input parameters to the logic
can come from almost anywhere - the source prole, properties of all types, and
even connector calls. A special document property, found under Base, is created for
failing records. This property holds all of the error messages for each of the rules
dened in the step.
There are a few challenges in using Business rules: The dedicated UI is complex,
and not intuitive. The controls used to set up the logic are also a bit confusing. And
some gyrations are needed to pull out the error message for failing records.
But the Business Rules step does allow
you to test for multiple complex rules
at one time, use a map to extract theerrors, and alert with a customized error
message for each error. Well worth
learning to use.
-
8/17/2019 Dell Bhomi Tool 9Ways
9/15
Notifcations
The techniques discussed above are ways of detecting errors when they
occur in your Boomi process. When an error does occur, the rst thing youwant to do as part of error handling is to notify the appropriate parties with
information that will help them diagnose the error situation and recover
the processing. There are a few ways to do this - the Notify step allows you to
create Boomi platform alerts, warnings, and errors, and use the Boomi platform to
send these alerts.
More typically, your error branch creates an error notication message and usesan email connector to email it to the appropriate party. Your email should contain
several important pieces of information: the process that failed, the ID of the record
that failed, and the nature of the failure. Use a message component to assemble the
error message, a set properties component to set addresses and subject , and an
email connector to send it out.
You can include propertiesand prole elds in the
message body, just note that
the syntax used to construct
the message is a bit different
from the syntax used to
include parameters in a SQL
statement or a parameterbuild.
-
8/17/2019 Dell Bhomi Tool 9Ways
10/15
Error Post BackTo Source
Many lower volume integrations that are part of bigger business processes use
an error post-back technique. In this approach, when an error is detected in a
document being processed, an error message is formatted with useful information
about the error and the data. Then, rather than sending an email, the error message
is posted back to the originating record or object, to a custom “Integration Error”
eld.
The benet here is that the error is directly associated with the source record, and
can be dealt with immediately. Notications can also occur via email, but often a
scheduled report that lists objects with error messages in their Integration Error eld
is more useful.
This technique is effective with lower volume transactions that require some human
intervention as part of the larger business process. A good example is Approved
Expenses - a manager must approve the expense report manually, and is the logical
person to notify if there is an integration error.
-
8/17/2019 Dell Bhomi Tool 9Ways
11/15
Enviroment Properties
The use of these properties is an advanced, but important, part of error
management. The management of parameters across environments is afundamental part of the Boomi architecture. Environments are part of what make
Boomi Atomsphere an enterprise-class integration development and deployment
environment.
The Dell Boomi team has given lots of thought to the development process, and it
shows - the seamless flow from build to test to production, component versioning
logs, and environment properties are three examples of this. Everyone uses
environment properties in the connectors to route transactions between test and
production data sources. But you can also use environment properties for error
handling.
A simple example is routing notication emails. You probably want to have these
routed to you during build and test, but to operational contacts during production.
In a properly structured Boomi integration process, the majority of the production
errors are data problems in the source or destination system, and these aregenerally resolved by the appropriate operations people – that is, Users not IT staff.
These Users are probably not getting alerts from the Boomi platform, so your
process should give them the information that they need, via email, to diagnose and
resolve the problem.
Create a meaningful notication message as described above, and route it to them
via setting an environment property in the production environment that contains thenotication list for the To: property of the email error message.
This can be a distribution group in your email environment, and by setting it up as
an Environment Property you can route error emails to you during development and
test, and to operations contacts in Production.
-
8/17/2019 Dell Bhomi Tool 9Ways
12/15
Tracked Fields
An advanced technique in Boomi integration is the use of tracked elds in your
processes. Most connectors have a tracking tab, which allow you to push a unique
ID into the process logs for that process. The Process Monitoring interface in the
Boomi Atmosphere environment lets you look for processes that are in error, but
also lets you search for individual documents by a tracking number, or ID. Pull
down the triangle next to Executions on the Process Reporting tab to switch over to
Documents mode, and enable search by tracked elds.
This is useful in lots of
situations - often a process
will throw an error, but it was
the prior execution that caused
the problem. Or hundreds of
documents were processed,and you need to nd a specic
one. Setting a tracking number
in the process logs lets you use
a logical or business ID to look
up information in the Boomi
logs. This is so important in
diagnosing run-time problems,that no process is ready for
production until tracked elds
been properly set and tested.
-
8/17/2019 Dell Bhomi Tool 9Ways
13/15
Conclusion I’ve covered some of the most important techniques and tools.The Dell-Boomi
Atomsphere Cloud Integration Platform is deep and rich in functionality, and there
are many other tools, techniques and tips for the advanced developer.
I’d like to hear about your favorite tips, as well. Feel free to send an email to
[email protected] with your techniques, and I’ll include them in version two. Or
post your tip as a comment in the Blog at kitepipe.com.
As more and more enterprises move business processes to specialist cloud-
based application platforms, the role of integrations will increase in volume and
importance. Its my goal to provide you with the tools to build and maintain those
integrations in robust and professional manner.
-
8/17/2019 Dell Bhomi Tool 9Ways
14/15
About The Author
Larry is founder and Chief Solutions Architect at KitePipe,
a specialist cloud integration services rm that helps
customers improve operations.We do this by streamlining
functions through integration and workflows, and deploying
dashboards that track the improvement in operational
metrics.
Larry is an expert Dell-Boomi Atomsphere architect, consultant, developer, and
trainer. Larry writes and speaks about Cloud Applications and Integration topics,
and leads the services team at Kitepipe. His consulting career includes work with
numerous Fortune 100 companies, and top-tier global systems integrators. He
consults with software vendors on cloud architecture and integration product and
services strategy.
-
8/17/2019 Dell Bhomi Tool 9Ways
15/15
About Kitepipe
At Kitepipe,We assist clients with seamless business solutions across multiple
cloud-based or on-premise applications to achieve operational gain. Servicesinclude design, conguration and testing of integration components to achieve
seamless business coordination across applications. Our specialty is Boomi, and we
offer a range of consulting and structured services offerings for Customers who use
the Dell-Boomi Atomsphere platform.
Check out our Offers section for information about a services offering that can help
you:
• Jump start your Boomi Platform implementation, eliminate costly delays, and
reduce time to integration project success;
• Refactor and reformat your existing Boomi Processes to reduce maintenance
time, improve component re-use, improve performance, and minimize your
purchased connector usage;
• Multiply the value of your Netsuite investment with rich integrations to
fulllment, sales, webstore, manufacturing, or partner/customer systems.