(ATS3-PLAT02) Advanced SOAP and RESTful Services Integration

Jason BenedictSr. Architect, Platform R&Djason.benedict@accelrys.com

The information on the roadmap and future software development efforts are intended to outline general product direction and should not be relied on in making a purchasing decision.

Pipeline Pilot Integration Points

DBs

Pipeline Pilot Enterprise Server

Protocol Runtime Environment

VB Script (On

Client)

Run Program Java Perl Python

VB Script (On

Server)

Admin Portal Help PortalWeb Apps Web Services API

.NET SOAP & HTTP

Telnet / FTP SSH / SCP ODBC /

JDBC

VB Script Cmd Line Java Classes

Perl Scripts

.NET Classes

RESTService

SOAP Service Cmd Line

Server IntegrationExtend pipelines with new components that integrate your code, data and services.

Professional ClientRun Protocol

Command Line Client

Web Browser

Web Port

JavaScript Client SDK

JavaScript Client

.NET Client SDK

.NET Client

Java Client SDK

Java Client

Web Services API

SOAP Client

Client IntegrationBuild clients that connect to Pipeline Pilot and run protocol services.

REST API

Grid System Integration (optional)

Agenda

• Intended Audience: Pipeline Pilot Protocol Authors and Component Developers• SOAP Services

– SOAP Method– SOAP Method with WSDL Support (Queued)– SOAP Connector

• Customization of parameters and template

– Demos• RESTful services

– Reader Components– HTTP Connector– Demos

• Advanced Results handling– Data from XML, Data from JSON– Data Tree Manipulators– Demos

“Wrapped” Component

• Scientific Pipeline Pilot User

• Abstracted Component

• Web Service underneath handles complexity

SOAP Connector

• Expert Pipeline Pilot User

• Handles Complex Web Services based on WSDL

HTTP Connector

• Expert Pipeline Pilot User

• Generic HTTP component

• Great for direct handling of HTTP actions

• Swiss Army Knife for HTTP

Perl/Java/.NET Server Components

• Software Developers

• Complex but flexible

• Leverage Perl/Java/.NET libraries and tools

Web Service Access

Complexity and Flexibility

Ease of Use

• SOAP Web Services– Messages usually sent over HTTP– Always a POST, body is XML SOAP Doc– Always includes “SOAP Action” header– Always set Content-Type to “text/xml”

• WSDL – Binding defines how messages are sent

(SOAP 1.1, 1.2, style, transport)– Port Type defines operations and

messagse– XML schema defines the data types in

messages

• WS-Policy– Defines the Quality of Service features

required by the service– Security , Addressing, etc.

SOAP-based Web Services

* - http://www.w3.org/TR/soap12-part0/

Accessing SOAP ServicesComponents Use When Don’t Use When / Caveats

SOAP Method • RPC-Encoded SOAP • Probably never

SOAP Method with WSDL Support • RPC-Encoded SOAP• DOC-Lit SOAP w/ Simple Type

• DOC-LIT w/ Complex Types

SOAP Method with WSDL Support (Queued)

• Same as above, but when threading is needed for through-put

• Be careful with DOS attacks

SOAP Connector • DOC-Lit SOAP w/ Complex Types• Template flexibility is required

• RPC-Encoded SOAP

HTTP Connector • Need to create and POST your own SOAP messages• Need custom handling of outputs• Need to customize HTTP settings

Java (On Server).NET (On Server)

• WS-Policy Requirements• Attachments or MTOM• Client stubs already exist• All else fails

• Simpler methods will work• Generally requires package deployment• HTTP Proxy settings and authentication will not get picked up

SOAP Service Demos

• SOAP Method with WSDL Support (Queued)• SOAP Connector– Advanced XML Constructs with Amazon S3 Simulation

Debugging SOAP Problems

• Useful tools:– Capture SOAP Packets– TCPTrace.exe (http://www.pocketsoap.com/tcpTrace/)– SOAP UI (http://www.soapui.org/)

• Socket debugging is available on the server– Contact support for details– Use sparingly since it very quickly creates huge log files– Turn it off as soon as you are finished

RESTful Web Services

• Resources identified by URL’s– http://localhost:9004/zoo/animals– http://localhost:9004/zoo/animals/13– http://localhost:9004/chem/img/C1CCCCC1C(CCN)OH

• Operations on resourced identified by VERBS: – GET = get a representation of the resource– PUT = update the resource given this data– POST = create a new resource of this type with this data– DELETE = delete the resource– HEAD = get some meta-data about the resource

• Representation Format– Accept: application/xml– Content-Type: application/xml

Accessing RESTful Web ServicesComponents Use When Don’t Use When / Caveats

XML Reader (Generic)JSON ReaderText ReaderBinary Reader

• You want to GET a representation of a HTTP-based resource

• You want something other than GET• You want another format

HTTP Connector • You need to perform any operation but GET• You need to customize settings like timeout, headers or authorization

• You have HTTP requirements not met by this component

Java (On Server).NET (On Server)

• You have special HTTP requirements• You have a pre-built client library• All else fails

• Simpler methods will work

• Swiss army knife for all HTTP• Sends one request per Data Record• Can be used for any type of HTTP Verb– GET, POST, PUT, DELETE, HEAD, TRACE, OPTIONS, etc.– specialized verbs as well, such as PROPFIND and MKCOL

HTTP Connector

• HTTP communications from all standard components uses the same library– HTTP 1.1– Basic, NTLM, and Negotiate Security

• “Authentication Options” or username/password in URL• “Use Accelrys Credentials” allows single sign on from Pipeline Pilot Clients

– Redirects on 300 responses– Works with Proxy servers (Administration Portal setup)– Decompress and dechunk based on Transfer and Content

encodings– 100 Continue responses for requests with a body

HTTP Connector – Internal Handling

• Set Parameter names and values– Values are Pilotscript expressions

• use Data Record properties

– One HTTP request for each input data record• If no input, works like a Reader

– Add additional parameters

• FormMultipart or FormURLEncoded via “HTTP Post Method”

HTTP Connector - POST

• Add or override most HTTP request headers– Cookies, Content-Type, etc.– Cannot override “Authorization” or “Allow”

• Can now over-ride these headers in AEP 9.0

• Custom request body– Disables the ability to set “Post Parameters”

• Timeout

HTTP Connector – Custom Options

• Limit size of response via “Maximum Response Size”– Can also stream the data via “Maximum Property Size”– Think of this like a “buffer” in code– Only Data from XML and Data from JSON components understand this method

• Return response headers– Parsing Cookies, Content-Type, etc.

• Read as binary– Creates a ByteArrayValue property

• Connection Options– Keep-Alive, Expect: 100, 403 servers as though they sent 401– Allow error response codes to be treated as valid

HTTP Connector – Custom Options

Demos

• Accessing Simple RESTful Services• Using the HTTP Connector

• Useful browser plugins– Firefox: Live HTTP Headers– Firefox: RESTClient– Chrome: Advanced REST Client

• Socket debugging is available on the server– Contact support for details– Use sparingly since it very quickly creates huge log files– Turn it off as soon as you are finished

HTTP Connector –Debugging Tips

• What we learned– Connecting to SOAP Services

• SOAP Method Components• SOAP Connector• Results Handling• Debugging

– Connecting to RESTful Services• Readers• HTTP Connectors• Results Handling• Debugging• Managing Links

• Recommended Sessions– (ATS3-DEV05) Coding up Pipeline Pilot Components– (ATS3-DEV07) Pipeline Pilot as a Web Service Deployment Platform

Summary

The information on the roadmap and future software development efforts are intended to outline general product direction and should not be relied on in making a purchasing decision.

For more information on the Accelrys Tech Summits and other IT & Developer information, please visit:https://community.accelrys.com/groups/it-dev