soap web services best practices _ bmc communities

7212019 SOAP Web Services Best Practices _ BMC Communities

httpslidepdfcomreaderfullsoap-web-services-best-practices-bmc-communities 17

BMCcom gt All Places gt Products gt Remedy AR System gt Documents

ltlt Other Best Practices

ltlt AR System APIs and Integration Interfaces Overview

Context and Issue Statement

BMC provides a synthetic and straight-forward description of web services and their implementation in AR System See the ldquoIntegrationrdquo

documentation for your version (7604 or 81 ) for

A short introduction to web services

A description of the web service architecture in AR System

You can also see this document for a quick dive-in (the document is old but still valid)

Consuming (from a filter set fields action)

Publishing (from Mid-Tier)

Registering (UDDI)

The Pulse AR server the SOAP client (really) by Justin Bakker is a great introduction to SOAP web services in the AR System context

This document adds some practical information gathered from experience and discussions on these boards

Performance concerns

ITSM Web Services

Designing robust implementations

Complex web services

Web Services and SSL

Debugging tools and techniques

Most common error messages

Known limitations and issues

Consuming vs Publishing

In the following paragraphs we consider that we are on the AR System side of an interface

- Consuming means reaching for an external web service from AR System via the Web Service ARF Plugin in order to push data to or get data from

another application

- Publishing means exposing methods via the Mid-Tier for external applications to push data to or get data from AR System

Performance concerns

When allowing 3rd Party consumption of your published Web Service the Mid Tier is used as the interface platform As the Mid Tier serves User

access also dependant on the perceived load by the Web Service additional dedicated Mid Tiers should be considered for Web Service

integrations With the standard scalability of AR System this is a cost effective way to build a robust integration platform

When consuming a 3rd Party Web Service the AR System Web Service plugin is utilized Dependant on the consumption load additional

performance tuning may be required on the AR side

ITSM Web Services [publishing]

Using BMC ITSM OOB Web Services How to test the ITSM Incident Interface based Web Services There is also some example NET (VB) code

showing how to Create Query and Modify using the Incident based Web Services ( Carl Wilson)

Designing robust web service interfaces

If you are designing a simple query interface to allow your partner to search for items or get some details for them then you should only need to

publish a web service with a getEntry and a getEntryList methods directly on your form provided you keep the performance concerns in mind

Now if you are going to design a full AR System WS interface with bidirectional objects updates there are a few questions you need to answer and

points of attention

- consider that the partner application might be unpredictably unavailable or your request could be rejected for an unanticipated reason you

should handle the errors without blocking your users

- consider that your application might be unpredictably unavailable (MT or AR System down) if your partner sends his requests in bulk and

not necessarily in the right order you should be able to handle them without loosing consistency

- consider that the partner application or yours can have scheduled downtimes an admin should be able to deactivate the interface

temporarily and resume it without loosing data

SOAP Web Services Best PracticesCreated by Sylvain YVON on 24-Oct-2011 0149 Last modified by Sylvain YVON on 25-Mar-2015 0240

Version 23



For all these reasons keeping all transactions in a staging form is strongly recommended

The diagram below shows a sample design for a bidirectional interface for incidents

Remember that if the partner web service is unavailable you will receive an error from the web services plugin If you consume the WS on modify

without handling the errors on the HPDHelp Desk form the user simply wont be able to save his changes

You have two options here

- call the WS synchronously when the staging form entry is createdmodified but catch exceptions on your staging form with an error handler

- use an escalation to call the WS asynchronously after an entry in the staging form has been flagged handle exceptions too

If you can bear with a few minutes of delay the second option is the way to go If you choose the first option the WS call will be done quickly most

of the time but if the partner is malfunctioning and causes a timeout your users will be blocked until the timeout is reached

A staging form comes with a few perks too

- you keep track of every transactions and their status see the error numbers and labels

- you can add a mechanism to replay an incoming or outgoing transaction after an error

- you can make statistics about your interface

- you can build a user friendly console for the administrators empowering them with the options listed above

Complex Web Services

A lot of discussions around web services in AR System community are about consuming or publishing complex web services We call web services

complex when they include more than one occurence of an element For example you may want to send or receive informations on an update of

an incident or change along with its worklog entries and its related CI

To either build a complex web service or to consume one you will need to define the following elements

The main form on which the web service will be based (ex HPDHelp Desk)

One or more child forms that contain the multiple elements (ex HPDWorklog)

A distinguishing key (DK) the unique key (for this transaction) that will differentiate the children records (ex Worklogs ID)

A foreign key (FK) the child form field that will contain the parent forms Primary Key (ex Incidents ID)

[publishing] a complex web service

Lets say you have 2 forms

- P the parent form (ex HPDHelp Desk)

- C the child form (ex HPDWorklog)

You want to build a create operation

For P the Primary Key is the Request ID by default This primary key is obviously unique

In C you need two keys because of the sequence of which AR System creates the records First it needs a key to link the records of C to the

records of P This is the foreign key Given that you can have multiple records in C for a record in P it is not unique When AR System create

the records it pushes Ps primary key into the field you defined as foreign key You must not map this field in your input mapping

The second key will identify uniquely in the context of this request the record of C This is the distinguishing Key This field you have to set

with the input mapping So this value doesnt have to be unique table-wide but request-wide

Here is an example with values You want to push a request with 1 parent record and 2 child records

01 ltPgt

02 ltgt

03 ltCgt

04 ltidgtC1ltidgt

05 ltgt

06 ltCgt

07 ltCgt



When you send that request to your WS heres what you get

- record in P reserves a request ID for example 1

- records in C are created

- AR System sets the FK field to 1

- You mapped the id field from the input request to your DK field so the first one gets C1 and the second gets C2

You can rerun this multiple times The only thing that has to be unique in your tables is your request ID which AR System is taking care of


This part is focused on the configuration of AR System and its arplugin The commands for handling certificates and truststores depend on you

servers OS and Java version This doc by Karl Miller provides a lot of details about how to deal with certificates with Remedy 8

Please post your question in the AR System community forum if you have one on the subject

[publishing] a web service with server-side certificate

Server-side certificate to make people access your web service via https instead of http can be configured on the web server (Apache Apache

Tomcat if stand-alone IIS Web Logic ) or on the reverse proxy gateway

Either way this is not an AR System concern

[consuming] a web service via https

To encrypt the http stream and authenticate the server hosting the WS if its private key has not been compromised you can use https

instead of http For this the server needs a valid certificate that has been approved by an authority that is trusted by your server

Java comes with many world wide trusted authorities but big IT departments often use their own authority for internal certificates

its cheaper if you have many certificates to handle

its more secure (search for DigiNotar being compromised for example)

Most of the time servers are already configured to accept the internal authoritys certificates

To make the plugin validate the servers certificate you need

a truststore (if the authority is not already trusted)

to add the authoritys certificate to your trustore or simply the WS servers certificate if its self-signed

to configure arplugin to use the truststore

edit the armonitorconf file

locate the line for java arpluginsvrjar

add the following parameters to the command line

-DjavaxnetssltrustStore=ltpath to your truststoregt

-DjavaxnetssltrustStorePassword=ltpasswordgt

[consuming] a web service using a client-side certificate

After it has authenticated itself the server that publishes the web service can require that your server authenticates itself If your private key

has not been compromised it pretty much guarantees that no rogue server will access the web service by impersonating your server

This is a good idea if anonymous login has been configured for example

To make the plugin show a client certificate you need

if you need a signed certificate

a csr file (Certificate Signing Request) to send to the certification authority

the cer file the authority will return to you

else

the cer of your self-signed certificate

a PKCS keystore

to store your certificate in the keystore

to configure arplugin to use the keystore and to send the client certificate when required



add the following parameters to the command line-DjavaxnetsslkeyStore=ltpath to your keystoregt

-DjavaxnetsslkeyStorePassword=ltpasswordgt

-DjavaxnetsslkeyStoreType=pkcs12

08 ltidgtC2ltidgt

09 ltgt

10 ltCgt

11 ltPgt




Usually when you develop applications and interfaces based on AR System your best debugging friend is the server-side logging functionality

(escalations filters SQL arplugin API) For web services it will help but sometimes it wont help enough

Use these log files for debugging

MTWebService to see inbound requests to a published AR System WS

Java logging tool like log4j to see outbound requests from AR System Java Plugin to an external WS (See How to log soap messages when

consuming Web Service )

AR Systemfilters

It might be enough for the occasional debugging For the conception and development of an interface you will need a SOAP development tool toforge your sample requests do some unit testing for both inbound and outbound SOAP requests

There are a few toolsout there so take some time to check them out

SOAP UI which comes in a free version with all we need for most situations

Creates sample requests from WSDL file ([publishing] and [consuming])

Creates unit tests ([publishing])

Creates mock web services from a WSDL file ([consuming])


1 In DevStudio while loading a WSDL ([consuming])

When an error occurs while loading a WSDL within a Set Fields action in a filter the most obvious symptom is that nothing happens Clicking

the reload button leaves the ports and operations menu desperatly empty Remember that only your client is used at this stage not your ARSystem server The first thing to do is to make sure the Messages window of DevStudio is open The messages displayed there are often

obnoxious but you need to bite the bullet and take some time to read the details

PKIX path building failed

ERROR URI not found httpsmydummyserver1337wsdl

Additional Details sunsecurityvalidatorValidatorException PKIX path building failed

sunsecurityprovidercertpathSunCertPathBuilderException unable to find valid certification

path to reque sted target

You are trying to consume a web service hosted on a server that requires https The certificate provided by the server is not considered as

trustworthy What you need is to add the public key of the host of the web service to a local truststore If the publisher didnt provide it you

can get it from your browser On Firefox click the lock icon next to the URL then more informations show certificate details export

Save it into a crt file Now you can add it to a local truststore using the java keytool and change your devstudioini file to add



Restart DevStudio and if the settings are correct you should not see this error

An error occurred trying to resolve schema referenced at []

The WSDL references a schema (xsd) that cannot be locatedaccessed Make sure the URL is correct and that your workstation can resolve it

You can test this with a simple browser If you can check if your browser is accessing it via a proxy server If it does make sure your DevStudio

is also configured to use this proxy (see below for the proxySet option)

If your workstation cannot access to file by any mean you have to download the file or request it from the publisher Save the wsdl and the

xsd files in a directory and edit the wsdl file Replace the http adress with yourfilenamexsd save and load this wsdl

Unable to locate or load the specified file 0 []

This one is the most puzzling error you can get It seems like quite generic and can be seen for a number of reasons But the most commoncause for it is that the WSDL is not understandable by DevStudio in its current state See known limitations and issues below

It can also be raised when the WSDL has some references to URL that cannot be resolved either because they are just dummy URL (like

httptempuriorg ) or because your workstation itself cannot resolve them (not on the same network no access to Internet or access

through a proxy server) To solve this there are two solutions

modify your hosts file to map a dummy IP address with the FQDN (Fully Qualified Domain Name) ie 127001 tempuriorg

if you have access to Internet via a proxy server modify your devstudioini to let it know


[Consuming] Documented limitations regarding specific XML schema and WSDL structures render some W3C-compliant web services

impossible to consume by AR System plugin

01 ‐DhttpproxySet=true

02 ‐DhttpproxyHost=proxy_address

03 ‐DhttpproxyPort=8080



Average User Rating

(11 ratings)

MOST LIKED

The supported structures depend on your version of AR System See your versionrsquos Integration document for more details or here for

version 81

So how do you handle this Well get ready for some nasty XML tinkering You are going to forge your own WSDL to make AR System

understand it

Sometimes it will not be possible Eg replacing ltsanygt if the sequence never has the same structure

Sometimes it will be approximate Eg replacing an unbound limit to 10 assuming that you will never receive more than 10

Sometimes you will brilliantly reproduce the original WSDL with a supported structure after hours of pain Eg merging an external

unsupported xsd file into the wsdl

Either way you know you will have a lot of work if your interface nemesis changes This is also a major risk you have to consider when

charging for the conception of a web service interface Out of 6 web service-based interfaces I created 4 required a locally edited WSDL

[Consuming] the appropriate web service URL according to the current environment

During your development process chances are youre not consuming the production webservice of your interface mate So how do you move

to QampA or production or other environment

Starting from versions 7604 you can query the Registry to overwrite the WS location at run time but you cant set the endpoint directly from

a field See Querying the registry form more details

One solution is to modify the wsdl locally set an alias for the ldquolocationrdquo tag and configure the ldquohostsrdquo file on your server to target to correct

environment Of course it supposes that all the URL are the same except for the host name (same port same path)

Frequent terms and abbreviation

FQDN Fully Qualified Domain Name

SOAP Simple Object Access Protocol

WS Web Service

External links

Jose Huerta posted a good article here

45543 Views Categories Best Practices Developer Guides Tips and Techniques Utilities Tags api ws dev web-service

Login or Register to comment

17 Comments

Great work Sylvain YVON Thanks for updating this document

Jason Miller 23-Apr-2013 1908

Like (1)Actions

Nice detailed explaination of Web Service Integration in Remedy I would like to know if BMC has recommended any best practice for ErrorHandling while using Web Services

Vinod Gaidhani 24-Apr-2013 0558

Like (3)Actions

Hey guysThanks for your feedbacks

Vinod NameToUpdate I dont know of any white paper on the subject The doc even states In the Error Handler panel verifythat error handling is disabled

Sylvain YVON 25-Apr-2013 0151 (in response to Vinod Gaidhani)

Like (0)Actions

Anupam Wagh 15-May-2013 0521



Thanks Sylvain YVON for sharing such a great document

Like (2)Actions

Thanks Sylvain YVON

Pankaj Salpekar 21-May-2013 0752

Like (1)Actions

Glad this helps guysFeel free to post update requests here or even update the document yourself if you feel like something is missing

Sylvain YVON 21-May-2013 0944 (in response to Pankaj Salpekar)

Like (3)Actions

Merci pour la doc et feacutelicitations pour ton papier il est excellent

Claire et peacutedagogique

Andreacutes RAMIREZ 14-Nov-2013 0948

Like (1)Actions

This article was pretty useful It would be nice to see more detail on how to consume a web service using client certs though When creatinga Set Fields web service filter action I dont know where you would define the cert fingerprint that you would be placing in the cert keystoreso you could load it on call to the external web service Is that something someone has documented somewhere

Dana Epp 30-Nov-2013 1904

Like (0)Actions

Hi Danathe certificates are all stored in a Keystore for the ARS Web Service Plugin (as the Plugin consumes the externalinternal WebService) You then define the Plugin (armonitor) with additional parameters where the Keystore is locatedThe document that is referenced ( Remedy 8 - Digital Certificates) explains in detail about the Certs and Keystores Once thePlugin has been setup to reference the Keystore you place all certificates required in the Keystore and the system willautomatically pick these up and use them at runtime

The Keytool and Keystore tools are used to importexportgenerate certificates Typically you will obtain the certificate from theexternalinternal source of the Web Service you are consuming httpsdocsbmccomdocsdisplaypublicars81Configuring+to+consume+over+SSL+or+with+client+certificates

No additional configuration is required on a Filter or Filter ActionLet us know if this does not cover your question (and if I completely missed the question) or you require additional information CheersCarl

Carl Wilson 01-Dec-2013 0422 (in response to Dana Epp)

Like (3)Actions

Sylvain YVON

Velan Ramakrishnan 04-Dec-2013 0724

Like (1)Actions

Thanks for sharing document

Subhash Sonwane 09-May-2014 0011

Like (2)Actions

Great document Sylvain Currently Im working with Complex Webservices in the same way youve explained and it Works as expected forCREATE operation However when using MODIFY operation it doesnt update the entries in the child form but seems all entries relate tothe parent form are deleted and it creates again only the information is sent during the MODIFY operation Do you know how to handlethis

Thank you

Nancy Ramirez 16-Sep-2014 0053

Like (0)Actions

Sylvain YVON 16-Sep-2014 0207 (in response to Nancy Ramirez)



Find People Community Help

Support Login Worldwide About BMC BMCcom

copy Copyright 2005-2016 BMC Software Inc Use of this site signifies your acceptance of BMCs Terms of Use Privacy Policy and Cookie NoticeBMC BMC Software the BMC logos and other BMC marks are trademarks or registered trademarks of BMC Software Inc in the US andor certainother countries

Manage Cookies

Hi NancySorry I never experienced this myself and cant test it right now Perhaps you could ask this on Remedy AR System then updatethe doc with your findings

Like (0)Actions

Thanks Sylvain YVON

Guillermo Rubio Blanco 16-Sep-2014 0226

Like (1)Actions

What are the possibilities in which WSDL amp End point URLs contain different protocols any hint I observed that they should have same protocol http-http|https-https if they are not same in such cases you should go for SSL related troubleshooting RegardsVinayak

Vinayak Sadashiv Bedake 30-Mar-2015 0039

Like (0)Actions

I have problem When I run the establishment its all perfectBut when I need to perform a SET operation l get an error and I can not to identify what the problemExample Form A to Form B (Operation = SET) i get message ARERR [ 8940 ] Error parsing XML definition If I use directly the SOAP tool it usually changes the target form but the call in the form origin through the filter does not someone could help me with this problem or could tell me what could be happening remembering that the operation runs in the sameserver Edit after many tests I managed to perform SET operation But I found that a type of type selection menu field with 19 options does notwork My configuration in field Element true MaxOccurs 1 MinOccurs 0 Nillable true Type String XMLType False

im tried other options without success anyone can help me

Leonardo Jose Sebba 31-Mar-2015 1314

Like (0)Actions

Hi LeonardoSelection lists are of string type by default Try logging your filters and see what value is sent (string value alias string integervalue ) You can then adapt the field type in the xml properties or use a temporary field to store the value before it is sentIf it still doesnt work please post a question in the AR System community with your wsdl and your filter logsHTHSylvain

Sylvain YVON 01-Apr-2015 0202 (in response to Leonardo Jose Sebba)

Like (0)Actions

0 0Google + 1 5 184K



For all these reasons keeping all transactions in a staging form is strongly recommended

The diagram below shows a sample design for a bidirectional interface for incidents

Remember that if the partner web service is unavailable you will receive an error from the web services plugin If you consume the WS on modify

without handling the errors on the HPDHelp Desk form the user simply wont be able to save his changes

You have two options here

- call the WS synchronously when the staging form entry is createdmodified but catch exceptions on your staging form with an error handler

- use an escalation to call the WS asynchronously after an entry in the staging form has been flagged handle exceptions too

If you can bear with a few minutes of delay the second option is the way to go If you choose the first option the WS call will be done quickly most

of the time but if the partner is malfunctioning and causes a timeout your users will be blocked until the timeout is reached

A staging form comes with a few perks too

- you keep track of every transactions and their status see the error numbers and labels

- you can add a mechanism to replay an incoming or outgoing transaction after an error

- you can make statistics about your interface

- you can build a user friendly console for the administrators empowering them with the options listed above

Complex Web Services

A lot of discussions around web services in AR System community are about consuming or publishing complex web services We call web services

complex when they include more than one occurence of an element For example you may want to send or receive informations on an update of

an incident or change along with its worklog entries and its related CI

To either build a complex web service or to consume one you will need to define the following elements

The main form on which the web service will be based (ex HPDHelp Desk)

One or more child forms that contain the multiple elements (ex HPDWorklog)

A distinguishing key (DK) the unique key (for this transaction) that will differentiate the children records (ex Worklogs ID)

A foreign key (FK) the child form field that will contain the parent forms Primary Key (ex Incidents ID)

[publishing] a complex web service

Lets say you have 2 forms

- P the parent form (ex HPDHelp Desk)

- C the child form (ex HPDWorklog)

You want to build a create operation

For P the Primary Key is the Request ID by default This primary key is obviously unique

In C you need two keys because of the sequence of which AR System creates the records First it needs a key to link the records of C to the

records of P This is the foreign key Given that you can have multiple records in C for a record in P it is not unique When AR System create

the records it pushes Ps primary key into the field you defined as foreign key You must not map this field in your input mapping

The second key will identify uniquely in the context of this request the record of C This is the distinguishing Key This field you have to set

with the input mapping So this value doesnt have to be unique table-wide but request-wide

Here is an example with values You want to push a request with 1 parent record and 2 child records

01 ltPgt

02 ltgt

03 ltCgt

04 ltidgtC1ltidgt

05 ltgt

06 ltCgt

07 ltCgt









































else


a PKCS keystore








08 ltidgtC2ltidgt

09 ltgt

10 ltCgt

11 ltPgt










AR Systemfilters













































Average User Rating

(11 ratings)

MOST LIKED


version 81


understand it

















WS Web Service

External links




17 Comments



Like (1)Actions



Like (3)Actions




Like (0)Actions





Like (2)Actions

Thanks Sylvain YVON


Like (1)Actions



Like (3)Actions




Like (1)Actions



Like (0)Actions





Like (3)Actions

Sylvain YVON


Like (1)Actions



Like (2)Actions


Thank you


Like (0)Actions







Manage Cookies


Like (0)Actions

Thanks Sylvain YVON


Like (1)Actions



Like (0)Actions




Like (0)Actions



Like (0)Actions

0 0Google + 1 5 184K









































else


a PKCS keystore








08 ltidgtC2ltidgt

09 ltgt

10 ltCgt

11 ltPgt










AR Systemfilters













































Average User Rating

(11 ratings)

MOST LIKED


version 81


understand it

















WS Web Service

External links




17 Comments



Like (1)Actions



Like (3)Actions




Like (0)Actions





Like (2)Actions

Thanks Sylvain YVON


Like (1)Actions



Like (3)Actions




Like (1)Actions



Like (0)Actions





Like (3)Actions

Sylvain YVON


Like (1)Actions



Like (2)Actions


Thank you


Like (0)Actions







Manage Cookies


Like (0)Actions

Thanks Sylvain YVON


Like (1)Actions



Like (0)Actions




Like (0)Actions



Like (0)Actions

0 0Google + 1 5 184K










AR Systemfilters













































Average User Rating

(11 ratings)

MOST LIKED


version 81


understand it

















WS Web Service

External links




17 Comments



Like (1)Actions



Like (3)Actions




Like (0)Actions





Like (2)Actions

Thanks Sylvain YVON


Like (1)Actions



Like (3)Actions




Like (1)Actions



Like (0)Actions





Like (3)Actions

Sylvain YVON


Like (1)Actions



Like (2)Actions


Thank you


Like (0)Actions







Manage Cookies


Like (0)Actions

Thanks Sylvain YVON


Like (1)Actions



Like (0)Actions




Like (0)Actions



Like (0)Actions

0 0Google + 1 5 184K



Average User Rating

(11 ratings)

MOST LIKED


version 81


understand it

















WS Web Service

External links




17 Comments



Like (1)Actions



Like (3)Actions




Like (0)Actions





Like (2)Actions

Thanks Sylvain YVON


Like (1)Actions



Like (3)Actions




Like (1)Actions



Like (0)Actions





Like (3)Actions

Sylvain YVON


Like (1)Actions



Like (2)Actions


Thank you


Like (0)Actions







Manage Cookies


Like (0)Actions

Thanks Sylvain YVON


Like (1)Actions



Like (0)Actions




Like (0)Actions



Like (0)Actions

0 0Google + 1 5 184K




Like (2)Actions

Thanks Sylvain YVON


Like (1)Actions



Like (3)Actions




Like (1)Actions



Like (0)Actions





Like (3)Actions

Sylvain YVON


Like (1)Actions



Like (2)Actions


Thank you


Like (0)Actions







Manage Cookies


Like (0)Actions

Thanks Sylvain YVON


Like (1)Actions



Like (0)Actions




Like (0)Actions



Like (0)Actions

0 0Google + 1 5 184K






Manage Cookies


Like (0)Actions

Thanks Sylvain YVON


Like (1)Actions



Like (0)Actions




Like (0)Actions



Like (0)Actions

0 0Google + 1 5 184K