domino: the integrationswu(o)nder€¦ · with lotus notessql, users and application developers can...

Stefan Neth – IBM Deutschland GmbH

Domino: The Integrationswu(o)nder

© 2015 IBM Corporation2

Agenda

IBM Domino Basics and Classics ● IBM Enterprise Integrator for Domino and DECS● Connectors and LSX● @DB, LS:DO and Data Connection Resource (DCR)● ODBC and JDBC● DXL and Web Services● TDI Tivoli Directory Integrator

IBM Notes on top● Plugins ● Widgets and LiveText

Web, Social and the future● OSGI and XPages● Rest Services● OpenSocial, Activity Stream and Embeded Experience● Social Business Toolkit


Do you remember ?


DECS Domino Enterprise Connections Services

Domino Enterprise Connection Services (DECS) is a visual tool and high performance server environment you can use to create Notes and Web applications that provide live, native access to enterprise data and applications. The visual tool includes an application wizard and online help to assist you to define external data source connections -- for example, DB2, Oracle, Sybase, File directory, EDA/SQL, or ODBC -- and fields within your application that automatically contain external connector data.


IBM Enterprise Integrator for Domino

Enterprise Integrator, a module ages ago called NotesPump, extends DECS functionality beyond real-time data sources to include support for high volume data transfer and synchronization. Enterprise Integrator provides visual tools to manage integration between data sources without programming, including the capability to initiate event-driven or scheduled high volume data transfers between Domino applications and relational databases and other enterprise applications. Enterprise Integrator also supports programmatic data transfers using LotusScript and Java Classes.

LEI 9.0.1 introduces the following features and supports● Support for LEI Installer with 64 bit JVM on Linux 64 bit● Support for LEI Installer with 64 bit JVM on Linux 64 bit● Support for DB2 10.1 and 10.5● Support for Sybase 15.7● Addition of Sybase 64 bit connector on Windows and AIX● Support for RHEL 6.4 64 bit


The difference between DECS and LEI

● LEI allows you to set up connection documents to a variety of different data sources, ODBC, DB2, SAP etc etc.

● LEI allows you to create a variety of activities that make use of these connections, eg Direct Transfer, Replication, Scripted, Polling, Virtual Fields, Virtual Documents, etc etc.

● The activities in LEI can use connections from eg DB2 to SAP (ie Notes is NOT one of the connections), and you are responsible for setting up how these systems interact.

● DECS REQUIRES that the 'target' connection IS Notes, nothing else. The source connection, or external connection can be ODBC, DB2, SAP etc etc...

● Also the only 'activity' DECS allows, is the equivalent of LEI's Virtual Fields (an interactive activity, since it monitors UI events such as create, open, update and delete on the target notes form).

● DECS is for free.


Domino Connectors

Domino Connectors are modules that provide native connectivity to external, relational databases. You can access these connectors through the forms-based development tool in DECS, LEI or through the Domino object classes using LotusScript or Java languages and the Connector LotusScript Extensions (LC LSX)

A connection is a document that defines how a Lotus Connector will interact with a specific data source. The connection provides specific access parameters, such as server names, user IDs and passwords. The “connected” databases are the databases that LEI or DECS can access. Because the forms used with each connection type are different, each Connection Document is different.

LEI and DECS provides the following standard Lotus Connectors for the

associated data sources:

DB2, Notes, File System (not used with Virtual Documents or Virtual Agents Activities),

ODBC, Oracle, OLE DB, Sybase, Text (not used with Advanced RealTime or DECS Activities), My SQL. Connectors for SAP and other sources are charged in addition.


LSX for Lotus Connectors

LotusScript Extensions for Lotus® Connectors (LC LSX) enhances the power of Notes® by extending its scripting to data outside of Notes through the use of Lotus Connectors. The programming model is independent of the individual connector. This eliminates the need to learn each individual system, while at the same time allowing experienced users to access the individual features of a specific system.

LSX supports access to the following connectors:

Notes DB2® File System ODBC Oracle OLE DB Sybase

In addition to these connectors, the LC LSX also supports additional premium connectors, such as the Lotus Connector for SAP Solutions. These specialized connectors may require additional methods; see the documentation that ships with each of these connectors for more information.

The LC LSX can be used alone or in conjunction with the Domino® Enterprise Connection Services (DECS) or Lotus Enterprise Integrator® for Domino (LEI). Respectively, these technologies provide programmatic and declarative access to external data for application development.

Dim LC_Field1 As New LCField(LCTYPE_TEXT,1)

Dim LC_Field2 As New LCField(LCTYPE_TEXT,1)

Dim count As Long

Dim SelectStatement As String

Dim workspace As New notesuiworkspace

Dim uidoc As notesuidocument

Set uidoc=workspace.currentdocument

On Error Goto ErrorHandler

LC_Conn.Userid="Administrator"

LC_Conn.Password="rac4you"

LC_Conn.Database="SAMPLE"

LC_Conn.Disconnect

LC_S.ClearStatus

LC_Conn.Connect

SelectStatement="SELECT * FROM EMPLOYEE ORDER

BY LASTNAME"

count=LC_Conn.Execute(SelectStatement, LC_FldLst)

If count <> 0 Then

count=LC_Conn.Fetch(LC_FldLst,1,1)

Set LC_Field1=LC_FldLst.GetField(1)

Set LC_Field2=LC_FldLst.GetField(4)

IDs=""

Messagebox "The Loop is starting"

While (count > 0) And LC_S.Status=LC_Success

IDs=IDs + LC_Field2.text(0) + "|" + LC_Field1.text(0) + ","

count=LC_Conn.Fetch(LC_FldLst,1,1)

Wend

Messagebox "The Loop is finished"

Call uidoc.FieldSetText("EmpNoList", IDs)

Call uidoc.refresh()

End If

End

Sample Code how to invoke LSX for Lotus Connectors


IBM ODBC Driver for Notes/Domino (NotesSQL)

IBM Lotus® NotesSQL is an ODBC (Open Database Connectivity) driver for IBM Lotus Notes® and IBM Lotus Domino® software. It allows ODBC-enabled data reporting tools, database tools, and application development tools to read, report, and update information that is stored in Domino databases (NSF files).

With Lotus NotesSQL, users and application developers can integrate Domino data with their applications using tools such as Crystal Decisions Crystal Reports, Microsoft® Visual Basic, Microsoft Access, Brio, and IBM Lotus Approach. Even Internet application development tools that support ODBC can access Domino data. IT professionals can enable their existing ODBC-enabled applications to access data stored in a Domino database.

A Domino database is not relational, but with Lotus NotesSQL a Domino database looks like a relational data source to an OBDC-enabled tool. This allows relational database management systems (RDBMS) such as Oracle and IBM DB2 to issue SQL (Structured Query Language) statements to Lotus Domino.

System requirements

Dokumentation

http://www.ibm.com/developerworks/lotus/documentation/notessql/


JDBC Access for IBM Lotus Domino (DomSQL)

The Domino JDBC Access (a.k.a. DomSQL) is a OpenNTF Project and exposes Domino data as relational tables and provides easy access to the tables using a JDBC driver.

JDBC stands for Java Database Connectivity. It is a set of technologies that are part of the standard Java Runtime environment, used to connect to relational database.

– Expand the query capability of Note/Domino (N/D)N/D views already provide a powerful query mechanism, with unique capabilities like categorization, responses... but native N/D views lack some key features like dynamic queries and joins between views. The features Domino views lack are exactly those at which SQL excels

– Reporting – Data analyticsSQL is the de facto standard used by reporting/data analysis tools. Making N/D data behave like relational data suddenly opens it to a large set of tools that understand JDBC.


@DB to show data from ODBC Sources

@DbCommand (ODBC data source)

Given data source information from the ODBC.INI file (or equivalent), uses this information to activate the appropriate ODBC driver. The driver then locates the specified DBMS, passes the specified command to it for processing, and returns the data retrieved by that command.@DbCommand only works with ODBC data sources and only with SELECT statements. If used with statements that don't retrieve a result set, @DbCommand simply transmits the statement. Use the ODBC capabilities of LotusScript® for more extensive interaction.

This formula uses the sample "pubs" database that is included with Microsoft SQL Server. The formula uses an ODBC driver to access the data source called PUBLISHERS, locate the table called "authors" that is owned by user "dbo," and then retrieve the list of names in the "au_lname" column for those authors who live in California and have a contract. The string CA is enclosed in single quotation marks, since it is already embedded within a quoted command string.

@DbCommand("ODBC";"PUBLISHERS";"dbo":"";"vanilla":"";

"SELECT au_lname FROM dbo.authors WHERE contract=1 AND state='CA' ")



@DbColumn (ODBC data source)

Uses data source information to activate the appropriate ODBC driver. The driver then locates the specified DBMS, table, and column, and returns all values in that column. You can optionally specify whether the returned list of values is sorted, whether duplicate values are deleted, and how null values are handled. @DbColumn can only retrieve data; it can't add, delete, or modify data.

This example uses the sample "pubs" database that is included with Microsoft SQL Server. The formula uses the ODBC SQL Server driver to access the database, locate the table called "authors" that is owned by user "dbo," and then retrieve the list of names in the "au_lname" column. The author names are sorted in ascending order; null values are discarded.

@DbColumn("ODBC";"PUBLISHERS";"dbo";"vanilla";

"dbo.authors"; "au_lname":"Discard";"Ascending")



@DbLookup (ODBC data source)

Uses data source information from the ODBC.INI file to activate the appropriate ODBC driver. The driver then locates the specified DBMS, table, and column, and returns only the values in that column belonging to records whose value in the key column matches the specified key. You can optionally specify whether the returned list of values is sorted, whether duplicate values are deleted, and how null values are handled. @DbLookup can only retrieve data; it can't add, delete, or modify data.

This example uses the sample "pubs" database that is included with Microsoft SQL Server. The formula uses an ODBC driver to access the data source called PUBLISHERS and locate the table called "authors" that is owned by user "dbo." In this table, the values in the "state" column are compared with the values "CA" and "TN." For every record whose state field contains either "CA" or "TN", the values stored in the "au_lname" field are returned. The author names are sorted in ascending order; null values are discarded.

@DbLookup("ODBC";"PUBLISHERS";"dbo";"vanilla";

"dbo.authors";"au_lname":"Discard";"state";"CA":"TN";"Ascending")


Data Connection Resource (DCR)

A Data Connection Resource is a design element you define within Designer to establish a data exchange between a Domino application and an external data source. Like other design resources, you can define a DCR and then use it in many places within an application, or use it in another application. For example, you might establish a connection to an inventory catalog stored as a table in a Microsoft Access database. You could then use that connection in a variety of forms in your application, or in several related applications.

DCRs provide a convenient alternative to using the

DECS Administrator that ships with the Domino server.


LS:DO LotusScript Data Object

This LSX-compliant module allows you to use LotusScript to write applications that incorporate data from external non-Notes data sources. Your script may perform the following steps:

● Establish a connection with the data source.● Each connection requires the data source name. If the data source name is not

already registered, you must also provide additional information.● Use SQL statements to send queries or other actions to the data source.● Retrieve and use result sets.● Read, modify, or add information into relational databases.● Disconnect from the data source.


Sample Script for LS:DO

Sub Initialize

Dim con As New odbcconnection

Dim qry As New odbcquery

Dim result As New odbcresultset

Dim firstnme As String

Dim lastname As String

Dim dsn As String

Dim userid As String

Dim parola As String

Dim msg As String

dsn="SAMPLE"

userid="Administrator"

parola="rac4you"

Call con.disconnect

If Not con.connectto(dsn,userid,parola) Then

Messagebox "Could not connect to " & dsn & " DataBase"

End

End If

Set qry.connection=con

Set result.query=qry

qry.SQL="SELECT * FROM EMPLOYEE ORDER BY LASTNAME"

result.execute

msg="Student Names:" & Chr(10)

If result.isresultsetavailable Then

Do

result.nextrow

firstnme=result.getvalue("FIRSTNME")

lastname=result.getvalue("LASTNAME")

msg=msg & Chr(10) & firstnme & " " & lastname

Loop Until result.isendofdata

result.close(DB_CLOSE)

Else

Messagebox "No Data retrieved for EMPLOYEE Table"

con.disconnect

End

End If

Messagebox msg

con.disconnect

End Sub


NotesDXL: Exporter and Importer

The DxlExporter class converts Domino data to DXL. Use the createDxlExporter method in Session to create a DxlExporter object. Use the exportDxl method to perform the export. Input to exportDxl can be a Database, Document, DocumentCollection, or NoteCollection object. Output is a String object.

The DXLImporter class converts DXL to Domino data. Use the createDXLImporter method in Session to create a DxlImporter object. Input to DxlImporter can be a String, Stream, or NotesRichTextItem object. Output is to a Database object.


Sample DXL Exporter: Notes Document

Sample DXL Exporter: XML Document<?xml version='1.0' encoding='utf-8'?><!DOCTYPE document SYSTEM 'xmlschemas/domino_9_0_1.dtd'><document xmlns='http://www.lotus.com/dxl' version='9.0' maintenanceversion='1.3' replicaid='852570360065D2D1' form='Form1'><noteinfo noteid='912' unid='C65EA84E8136C7DA85256FFE0066C337' sequence='9'><created><datetime dst='true'>20050511T144226,47-04</datetime></created><modified><datetime>20150228T132442,23+01</datetime></modified><revised><datetime>20150228T132442,22+01</datetime></revised><lastaccessed><datetime>20150228T132442,23+01</datetime></lastaccessed><addedtofile><datetime dst='true'>20050706T143212,04-04</datetime></addedtofile></noteinfo><updatedby><name>CN=Charles Connell/O=CHC-3</name><name>CN=Stefan Neth/OU=Germany/O=IBM</name></updatedby><revisions><datetime dst='true'>20050511T144226,47-04</datetime><datetime dst='true'>20050706T150727,64-04</datetime><datetime dst='true'>20050706T150928,93-04</datetime><datetime dst='true'>20050706T151208,62-04</datetime><datetime dst='true'>20050706T151213,71-04</datetime><datetime>20150228T131742,44+01</datetime><datetime>20150228T132305,48+01</datetime><datetime>20150228T132326,25+01</datetime></revisions><item name='RissStatus'><text/></item><item name='Subject'><text>Formatted body</text></item><item name='XmlDocRenderFile'><text>c:\notes_xml\Rep_852570360065D2D1_Note_912.xml</text></item><item name='Body'><richtext><pardef id='1'/><par def='1'><run>Hello. </run></par><par def='1'><run></run></par><par def='1'><run>This is the body of the message. </run></par><par def='1'><run></run></par><par def='1'><run>It has some colors, bolds, fonts, etc.</run></par><par def='1'/></richtext></item></document>


Domino and Web Services Lotus Domino V7.0 introduced the new Web service design element in Lotus Domino Designer. If you open a database in the Lotus Domino Designer V7.0 client, you see the Web Services entry just below the familiar Agents entry in the Shared Code section of the design element tree.

Lotus Domino deals with all the WSDL creation and SOAP handling for you, so all you have to do is write code in your Web service design element as though you were coding an agent. As long as you specify which class to use as the interface class for the service, Lotus Domino can publish a WSDL file, can convert incoming SOAP requests to method calls on your class, and can return the results of your method (if any) as a SOAP response.

From the coding standpoint, all you do is write a LotusScript or Java class. Lotus Domino does the rest.


TDI Tivoli Directory IntegratorTDI treats a Domino database like any other database, and allows you to integrate it with all sorts of other systems. The simple uses is reading files in many formats and driving that data into Domino. TDI even detects changes in the files between runs so that only the changed records are updated in Domino. It is included as limited use with your Domino Licence. One side of the sync must always be Domino.


Agenda





Notes Standard Client and Plugins

IBM Notes Standard Client is built on top of the Eclipse Rich Client Platform (RCP) and IBM Lotus Expeditor. You extend Lotus Expeditor by creating Eclipse plug-ins and deploying them using the provisioning included in Lotus Expeditor.

A plug-in is a component written in Java using the Eclipse or Expeditor Toolkit that provides a piece of functionality. Any given plug-in can define other plug-ins upon which it depends, thus permitting granular componentization and code reuse. A plug-in can also contribute functionality to the plug-ins upon which it depends through extension points. IBM Notes Standard Client do not run in the context of iNotes because of the missing RCP (Rich Client Platform) framework.

Plugins work very well to integrate with function or applications that are located on the client that run IBM Notes e.g. textprocessors or ERP clients.

Examples for Plugins:● Connections Chat Plugin● Connections Plugins for files and profiles


Notes and iNotes Widgets and LiveText

Widget and Live Text technology allows users to automatically recognize patterns of data in emails and any unstructured text. Widgets then provide a set of custom defined actions, linking to existing Web applications, pulling down an RSS feed, opening a gadget to pull or display information of any kind. The number of options are unlimited. The ability to build Widgets and Live Text is also unlimited, and requires only using a simple wizard and at the most writing a regular expression for data recognition. Below is a collection of references for Widgets and Live Text that you can use to get started.

Widgets can use are access technology and data like:

● XPages● Web page● Notes form, view● Feeds● Google gadgets● Any many more...


Agenda





Domino and OSGI

Open Services Gateway Initiative– OSGi™ Alliance (IBM involved)– Developed first in 1999– Continues to be improved

Module System and Service Platform– OSGi Bundles (Applications/components)– Bundles can be remotely...

• Installed / Uninstalled• Started / Stopped• Updated

– No reboot required for operation!


OSGi within IBM Notes/Domino...

IBM Domino Server 8.5.2– OSGi introduced in HTTP task– XPages runtime as OSGi plugin– Extension Points

IBM Domino Server 8.5.3+– OSGi support extended to;

• XPages Extension Library,• REST APIs,• Social Enabler• DOTS (Domino OSGi Tasklet Service)• Domino Servlet Container• OpenSocial Container• ...

8.5.2

8.5.3

9+


Why OSGi matters !!!

Extending toolbox– Designer components– Server-side implementation for components

Providing services for non-UI consumers– REST Services– Social APIs

Adapting standards– OAUTH– JDBC

Running Tasklets– DOTS


XPages: Free your mind

XPages can connect to any datasource and can mix and match any data with data that is contained in Domino or between external sources. Use the WEB Browser of your choice or your Notes Standard Client and mobile devices.

Here some examples:

● Embed data from IBM Connections via predefined Snipperts● Integrate into IBM Connections via iWidgets or OpenSocial Gadgets● Integrate to IBM Portal Server via iWidgets or any platform supporting iWidgets ● Notes/ iNotes Social Mail Embedded Experience as iFrame or OpenSocial Gadget● RDBs via JDBS● Any application via Java OSGI Bundles● Rest Services● Web Services● Feeds


Why Domino REST Services?

Separation of UI, data and business logic

Enable access from any device / OS

Extend reach beyond traditional Domino ecosystem

API

Rich Client

Appliance

Rich Client

Tablet

Phone


REST Services: Decoupled from XPages and Designer

Compliments, but doesn't require XPages

Access Domino from a web application, native mobile app, embedded system, etc.

Build your application in any IDE

Use any web application framework– Dojo, jQuery, Web components, etc.

Strategically important for integrating with other IBM products


Domino Access Services Framework

Domino Access Services (DAS) is a framework for adding REST services to Domino

Built with OSGi and Apache Wink

IBM delivers each service first to OpenNTF; then to a supported release

Customers and business partners may eventually be able to add services too

DAS Servlet

Domino Web Engine(native code)

...Mail

Service Plug-in

Calendar Service Plug-in

Apache Wink Runtime

BrowserServer

TabletPhone


Different Services and different Releases

Service name Root resource Resource types

Data service /api/data Database, View, View entry, Document

Calendar service /api/calendar Calendar entry, Calendar notice

Mail service /api/mail Mail message, Delegate, Quota

Freebusy service /api/freebusy Busy time, Free room

Service name 8.5.3 UP1 9.0 9.0.1 Extlib on OpenNTF

DAS framework Yes Yes Yes Yes

Data service Yes Yes Yes Yes

Calendar service

No No Yes Yes

Mail service No No No Yes

Freebusy service

No No No Yes


Domino and Open Social: Teminology

OpenSocial● OpenSocial is an open standard which defines APIs that can be used to build

social applications using HTML, JavaScript, and CSS. The specification includes both JavaScript and REST APIs. OpenSocial leverages many other standards, such as Activity Streams and OAuth.

● See: http://opensocial.org/

Embedded Experiences● Embedded experiences is a method of embedding the “experience” or

functionality from one application into a different, host application. The end user experiences the 3rd party application in their host application environment as if they were using it directly from the 3rd party provider.

Activity Streams● Activity Streams is an open standard which defines a data model to represent a

stream or list of actions performed by users and applications. This specification DOES NOT define any APIs, it simply defines the data model.

● See: http://activitystrea.ms/



Apache Shindig● Apache Shindig is an OpenSocial container and helps to start hosting

OpenSocial apps quickly by providing the code to render gadgets, proxy requests, and handle REST and RPC requests. Shindig is the reference implementation for the OpenSocial specification.

● See: http://shindig.apache.org/

Domino Server with Shindig● A Domino Server with the OpenSocial Component installed. This server runs

portions of the Apache Shindig project as a web application based on OSGi. This server is responsible for rendering OpenSocial gadgets.

DOTS● An acronym for “Domino OSGi Tasklet Service”, it is a server task that allows

OSGi plugins to contribute scheduled “tasklets” that perform some amount of work on the server. The Widgets framework in iNotes contributes a DOTS tasklet to push Widgets to iNotes users by policy.

http://shindig.apache.org/



OAuth ● An open standard used used for authorization. Allows the User to grant access

to private resources on a Service Provider to another site, the Consumer, without sharing user credentials

Widget Catalog● A database that is used to store metadata (i.e. a “widget”) about services for use

within Notes and iNotes. In the case of OpenSocial Component, the focus is on OpenSocial gadgets for which widgets are created and stored in the Widget Catalog.

Credential Store● A secure database that is used to store proxy rules, gadget capabilities, and

OAuth information on the Domino Server with Shindig.



Locked Domains● A mechanism to isolate individual gadgets from each other. Each gadget is

rendered in a separate iframe such that each iframe's source is unique. Any XHR requests the gadgets make to the Domino Server with Shindig occurs on a gadget's locked domain. Other resources,such as images and Javascripts, utilize a common unlocked domain.

● See: https://cwiki.apache.org/confluence/display/SHINDIG/Locked+DomainsFQDN

● Fully qualified domain name, ex. somehost.example.com

https://cwiki.apache.org/confluence/display/SHINDIG/Locked+Domains


Sample of OpenSocial

IBM Connections provides two OpenSocial Gadgets for use within IBM Notes and Domino deployments.

An Embedded Experiences gadget transforms Connections notifications into rich, interactive experiences

An Updates gadget brings the activity stream experience from the Connections homepage into your Notes client

Lets have a live look at this experience !!!


SBTK Social Business Toolkit

The SDK is targeted towards web and Java developers to easily access IBM Social Platform, including IBM Connections and IBM SmartCloud for Social Business. It can be run on many Java application servers like WebSphere Application Server, Tomcat, WebSphere Portal and Domino. The SDK provides easy to use JavaScript and Java APIs that encapsulate authentication mechanisms like OAuth, avoid JSON and Atom parsing and abstract the specific target environments.

A good example how easy to use the SBTK is: The trouble ticket applications from open NTF. Have a look at it here: TroubleTickets

http://www.openntf.org/main.nsf/project.xsp?r=project/Trouble%20Tickets%20Notes-to-XPages%20Example


The future is bright...

OpenSocial, SAML, Connections, IBM Verse, Cloud. Intenret of Things....

These are the topics that are on todays agenda and will be drivers in the future and of course there will be analytics with Watson technology and IBM BlueMix another great way to extend the reach of Domino...

Domino is right there... the right technology but easy to use....

Simply the best... for easy integration and robust applications in your context !!!


Thank you for attending the session.

If you have any questions or comments please contact me.

Stefan Neth

Client Technical Professional

IBM Enterprise Collaboration

IBM Deutschland GmbH IBM Alle 171139 Ehningen

Tel +49 7034 643-1232 [email protected]