session # bp201 – assemble 'n go: integrate data and ...domino directory - need to update ibm...

© 2012 IBM Corporation

Session # BP201 – Assemble 'n Go: Integrate Data and Directories with Tivoli Directory Integrator (#TDI)Marie Scott | Director, E-mail Services | Virginia Commonwealth University (@marie_scott)Thomas Duff | Developer | Cambia Health(@duffbert)

2 | © 2012 IBM Corporation

Agenda■ Introduction to TDI architecture/concepts

■ Discuss TDI entitlement

■ Installation

■ Examples

■ Resources

■ Question & Answers


What Is Tivoli Directory Integrator?■ Helps organizations build an authoritative data infrastructure, enabling consistent data across

multiple identity or generic data resources.■ Transforms, moves and synchronizes generic and identity data residing in heterogeneous

directories, databases, files, collaborative systems and applications, with real-time automated updates to the authoritative data source

■ Enhances the security, accuracy and integrity of generic and user identity data, while facilitating data migration, transformation to other file formats, and synchronization between two or more systems

■ Provides a simplified interface to develop point-to-point integration; ability to visually step through data processed by an AssemblyLine; providing Tivoli Directory Integrator as a service using the Representational State Transfer interface

■ Provides an intuitive graphical user interface for development, deployment and maintenance of synchronization rules, as well as a scalable, Web-based operations monitoring administrative console

■ Provides an open synchronization architecture that supports multivendor IT infrastructures with ease of use, ease of deployment, and rapid time to value, while flexibly scaling from small to very large deployments

(from: http://www-01.ibm.com/software/tivoli/products/directory-integrator/)

This may be why nobody understands or knows about this…


IBM Tivoli Directory Integrator:

This is a bit more direct…

“Tivoli Directory Integrator (TDI) is a graphical integration toolkit for accessing and detecting changes in practically any type of system, data store, protocol, or API. It also lets you transform, filter, and validate this data before driving it to the output targets of your choosing.”

Source: Redpaper IBM Lotus Domino Integration Using IBM Tivoli Directory Integrator

● Use for migration of data between IBM Lotus® Domino® and other data sources

● Use for integration and synchronization of data and directories between Domino and other data sources, such as Microsoft® Active Directory®


Remember To Think “Outside The Box”■ TDI works with these IBM/Lotus products

‒ IBM Connections®‒ IBM Lotus Notes®‒ IBM Lotus Domino®‒ IBM Lotus Sametime®‒ IBM Lotus Quickr®‒ IBM LotusLive™

So don't let the “directory integrator” fool you into thinking this tool is only for directories!

It's NOT!


TDI Isn't Only For Directories■ TDI connects to many systems:

─Domino, IBM Connections, LDAP, Microsoft Active Directory®, JDBC, RDBMS, HTTP/Rest, Oracle®, PeopleSoft®, SAP®, etc., etc.

─Connectors are the building blocks of your AssemblyLine─ TDI is a tool you’ll want to start using!


Possibilities■ Domino Directory - need to update from corporate LDAP directory (or Active

Directory)■ Domino Directory - need to update from Corporate HR directory■ Domino app - need to keep synched with Oracle database■ Domino app - need to update with xml file downloaded from app each night on

schedule■ Domino Directory - need to update IBM Connections■ Domino Directory - need to update LotusLive■ Other uses with Notes databases (we think developers might like)

─Connect to Microsoft SQL Server® database (or other relational database) to update Domino web page or database (scheduled or event driven)

─Notes db to Notes database (one time event)─SAP or Peoplesoft connectors─Output Notes to XML with data scrubbing


TDI Architecture


What Will TDI Connect To?


Conceptual View Of A TDI Job

Example of updating the Person document in the Domino Directory with phone number information from LDAP.


TDI Definitions And Concepts: Part 1■ AssemblyLine – just like a factory assembly line!

─ TDI uses AssemblyLines to push, pull, move, transform or sync data─Can have multiple AssemblyLines running at same time

■ Data Source – data system or group of data objects

■ Connector – Device used to connect Data Sources to AssemblyLines─Set up authentication─Set up attribute mapping─Connect to services like LDAP, AD, JDBC, Domino─Batch or event-driven─You can write your own


TDI Definitions And Concepts: Part 2■ Entries – Data Sources contain entries or records

■ Attributes – Each entry contains attributes or fields

■ Work Entry – is the entry that is manipulated by the AssemblyLine

■ Conn Entry – is the local work store for each connector

■ Attribute Map – links the data movement between the Conn Entry and the Work Entry for each Connector


TDI Definitions And Concepts: Part 3■ Connector Modes

─Iterator mode: iterate thru each entry, reads values of attributes, provides values of attributes to AssemblyLine

─Lookup mode: used to join data from different sources─AddOnly mode: add new data entries to a data source─Update mode: add or modify data in a data source─Delete mode: delete data from a data source─Delta mode: used to track incremental modifications to

connected system


TDI Definitions And Concepts: Part 4■ Link Criteria

─Used to tell a connector in Update, Lookup or Delete mode how to define a match between attributes so that they can perform work.

■ Hooks─Built-in “waypoints” in the AssemblyLine that allow you to add

scripts to extend your processes to add logging or parsing─Example: Hook: Before Execute

‒ You could add some JavaScript to do parsing or write a message to the log




■ Installation

■ Examples

■ Resources



So What Is The Cost Of This Great Tool?

TDI is FREE, FREE, FREE!

Image by Thiago Felipe Festa


TDI Is FREE, FREE, FREE!

■ As of Domino 8.5.3, you’re entitled to use TDI version 7.0 (most current version is 7.1)

■ If you’re using TDI for Connections, you’re also entitled to use version 7.0.




■ Installation

■ Examples

■ Resources



TDI Downloads■ C1IF0ML (Microsoft® Windows® x86-32)■ C1IF1ML (Windows x86-64)■ C1IF2ML (IBM AIX®)■ C1IF3ML (Linux x86-32)■ C1IF4ML (Linux x86-64)■ C1IF5ML (Linux zSeries)■ C1IF7ML (Oracle® Solaris® SPARC™)


Installing TDI■ TDI 7.0 and 6.1 currently supported versions■ We’re going to cover 7.0 in our slides■ Let’s get started!


Finding TDI On IBM Passport Advantage1.Select your Domino software type

2.Next screen – be sure to select the Download options – “If available would you like to see associated products included at no additional charge.”

3.Expand the “Associated products included at no additional charge” selection

4.Choose the version of TDI you’d like to download


Finding the IBM Connections TDI related files on Passport Advantage■ TDI files are packaged as part of the IBM Connections wizard


Once the Connections Wizard is run...■ The TDI related files can be found in the TDISOL directory


Starting The Install...Use the install_tdiv70_win_x86.exe instead of launchpad.exe...

It works better


You Might See This...This seems to be quite common on install, even when I think I have everything “cleaned up”. So long as you're not upgrading, you can ignore it.


Choose Your Install Directory...This is my default directory when installing onto a Microsoft Windows 7 platform on my laptop


Installing Everything (Disk Space Is Cheap!)A true server installation would leave out Javadocs, examples, and IEHS. I'm installing here so that I have access to everything locally.


Choose Your Solutions Directory...You will have a chance to change this when you start up TDI. Your AssemblyLines and jobs will be stored in this directory.


Select The Embedded Console...Again, choosing this for local installation and no existing ISC anywhere.


Taking The Default Ports For TDI...


If You Want To Have Console Start At Bootup...This would be useful if you were installing this on a server and wanted TDI to start up automatically on server startup.


Final Check...


If All Goes Well, You're Good To Go!Since we're going directly into testing to see if TDI starts up, the Start the Configuration Editor option can remain selected.


Starting Up TDI & Selecting Solution DirectoryThis is where you can change your solution directory, aka the “Workspace”. This can be useful if you want to have separate workspaces for different customers or jobs.


And We're There!If all goes well, this is the screen we'll first see. Once you select the Go to Workbench option, you'll open up to that screen when you start up in subsequent sessions.


Job #1 After Install... Install Fix Packs!http://www.ibm.com/support/fixcentral/

As I learned, admins get really peeved if you don't do this. Fix Central is very easy to navigate, and should be a “must visit” site for the IBM software that you run and maintain.


And To Connect To Domino...In order to connect to IBM Lotus Domino with TDI, you will need to copy the Notes.jar file from your IBM Lotus Notes installation...


Into The TDI Directory...… into the TDI installation under /IBM/TDI/V7.0/jars/3rdparty/IBM. Without this, you'll get Java errors when trying to connect.




■ Installation

■ Examples

■ Resources



Using TDI To Move IBM DB2® Data To Notes...■ Loosely based on an example in my work environment■ Using old version of Axceler Notrix® to move data from Oracle to Notes■ Runs nightly to load employee database that feeds other databases■ Goal is to replace Notrix with a tool we already have at no additional cost■ This would also work with Microsoft SQL Server or other RDBMS


This Is The Basic Notes Form We'll Be Populating...


Start With A Fresh TDI Workspace...


Create A New Project...


Give The Project An Applicable Name...


Project Built... Now Create An AssemblyLine...


Choose An Applicable AssemblyLine Name...


The AssemblyLine Is Created And Ready For Components...


Create The Feed Component...


Select JDBC Connector To Link To DB2...


Configure The Connector To Access The Employee Table In DB2...


Completed Connector And Test For Connection...


Now For The Notes Connector...


Configuration Of The Notes Connector...


Both Connectors Are Now In Place... Map The Data Next...


We'll Use All The Attributes From The DB2 Table...


All The Table Attributes Are Listed And Mapped To Work Attributes...


Now We Select Which Attributes From DB2 We'll Use In The Notes Database...


The Attributes Are Now Linked, But Those Are Not The Notes Attribute Names...


Update Notes Attribute Names By Clicking The Name In The Second Column And Adding Correct Name...


The Finished Result Of The Mapping...


TDI Also Does Data Transformation With JavaScript...


Making Sure The Email Address Is In Lower Case, And Then We'll Run The AssemblyLine...


The AssemblyLine Job Is Finished (1.5 Seconds)...


And The Results...


Intro To TDI And Directories

■ Directory related connectors■ LDAP configuration related details■ Only need to update corporate data?■ Use an LDAP connector

─Easy to set up, attributes easy to customize─Doesn’t provision Notes accounts


TDI Is Fantastic For Working With Directories

■ Connectors already included for Directory work:─ LDAP Connector:

‒ Use to connect to LDAP directories (including AD & Domino)─Domino Change Detection Connector

‒ Detects changes on objects in a Notes database (add, modify, delete). Includes actions on the names.nsf.

─Active Directory Change Detection Connector‒ Detects changes on AD objects.

─Domino Users Connector‒ Use when you want to create, delete or modify Notes user

information (in names.nsf or admin4.nsf). Can be used to create or delete Notes accounts.


Details For LDAP Connections■ Know your LDAP schema (or work with your LDAP team).■ You'll need the following:

─ LDAP userid that can read and/or write to the LDAP directory.‒ Organizational Unit‒ OrgClass

■ Another invaluable tool is Softerra's LDAP Browser (FREE!)http://www.softerra.com/download.htm

■ If you considering an assembly line to create Lotus Notes accounts:LDAP connection will not create full Notes users─Use Lotus Domino Users Connector in order to provision new accounts


Why Should You Care About LDAP or AD Details?■ Search Base – use the O or OU level qualifier for what you want to search for. If

you only need to run the AssemblyLine for an OU rather than O qualifier, don't use the O qualifier – extra overhead.

■ Search Filter – use a search filter so that you don't get objects you don't want (especially if your LDAP or AD contains many customized objects). The more specific the search filter, the fewer “junk” items you'll retrieve.

■ Credentials - sure you are logging in with credentials that has access to all O's or OU's and containers, so the AssemblyLine can process the work and run successfully.


A Little Domino LDAP Configuration Check:■ Confirm that domain configuration document has the following set on the LDAP

tab.■ You can control security via Security settings and/or XACL.


Domino LDAP Schema■ Domino LDAP Schema is stored in the schema.nsf■ Review classes and attributes to see how Domino might be using them and what

type of data is stored (string, domain name, etc.)


Notes Client Configuration■ Change the following security property in the Notes client Security Basics to

allow for DominoUsers connection to occur without prompting for a password each time the AssemblyLine starts


Create Domino LDAP Connector■ Create a connector for Domino – using base LDAP Connector■ Create in Iterator Mode


Configure The Domino LDAP Connector■ Add the specifics for your Domino LDAP connection (Connector Tab)


Browsing LDAP Data: How & Why■ Click the Connect button. Once you’ve established a connection you can begin

to browse your data.■ Review each record by clicking Next to review the data in each attribute.■ This allows you to determine if you need to make any editing changes when

performing an update or data match.■ You'll also see what attributes are available in the schema.


Browsing Domino Directory Data


Example: Active Directory To Domino Sync■ Create an assembly line(s) to do the following:

─Connect to Active Directory─Connect to Domino names.nsf─Detect any additional users created for Active Directory and create them in

Domino names.nsf─Detect any changes to information in Active Directory and update in Domino

names.nsf (but don't change internet address)─Detect any accounts deleted from Active Directory and delete in Domino (via

admin4.nsf)─Detect any changes in Domino and update to Active Directory.─Detect any deletes in Domino and update to Active Directory.

■ Going to use IF and ELSE IF Branches■ And some javascripting for good measure!


So How Are We Going To Do All That?■ Create a project■ Create “template” connectors■ Create a new AssemblyLine■ Create Feed connector based on template connector■ Create IF ELSE logic


Create A New TDI Project


Create Your Connector “Templates”■ Create the connectors you will use throughout the Assembly line■ ADChanges – based on ActiveDirectoryChangeDetector■ DominoUsers – based on DominoUsersConnector


Create Connector: ADChanges■ Use Active Directory Change Detection Connector template■ Use Iterator mode


Connector Configuration: ADChanges■ See the LDAP search base: “DC=topchefs,DC=com”■ LastChangeInAD parm■ Inherits from ADChangelog connector template


Create Connector: DominoUsers■ Uses the DominoUserConnector template■ Setup in Iterator Mode


Connector Configuration: DominoUsers

■ DominoUsers Connector detects the changes to users.■ Uses local client connection


AssemblyLine Next...

■ Create an AssemblyLine with data feed and data flow■ Map attributes that will match between AD and Domino■ Include any exception processes■ Create scripts or IF/Else/IF/ElseIF logic


Create AssemblyLine: SyncAD_to_Domino

■ First step – create a new AssemblyLine called “SyncAD_to_Domino”


Create Data Feed Connector: ADChanges

■ Create a connector based on ADChanges connector and include all work attributes


Connector Configuration: ADChanges

■ See LDAP Search Base■ LastChangeInAD setting■ Inherits from ADChangelog connector template


Add IF BRANCH - Delete


Create IF-Delete Script■ ret.value = work.getOperation() == “delete”;


Add ELSE Branch – Add Or Modify


Create Connector: DeleteDomino■ Use the DominoUsers Connector as your template


Create DeleteDomino Link Criteria■ Create the link criteria to match objects between AD & Domino


Create UpdateDomino Connector

■ Created based on the DominoUsers connector■ Created in update mode


Create UpdateDomino Link Criteria■ Update mode


UpdateDomino Mapping

■ Essentially the “working engine” of the AssemblyLine■ Includes JavaScript to modify the field data


Create BeforeModify Script


Create BeforeModify Script■ We're not updating the internetaddress object


Let's Start The AssemblyLine!

■ Click Run to start SyncAD_to_Domino■ Confirm that the process is “Iterating”


SyncAD_to_Domino Final AssemblyLine


Provisioning Notes Accounts

■ You can take this type of AssemblyLine and build a process to provision Notes accounts.

■ See this document for more information regarding the specific objects you need to pass to adminp to set up accounts.─ http://www.tdi-users.org/twiki/pub/Integrator/HowTo/IntegratingDominowithTDIv1.1.pdf


Setting up TDI to start as a Windows service■ There is a mechanism for allowing multiple instances of TDI to be started when

Windows starts.■ Each instance requires:

─ Its own solutions directory─ Files from the directory <TDI rootdirectory>/win32_service

‒ ibmdiservice.exe‒ ibmdiservice.props

─Unique service name


Make changes to your ibmdiservice.props file■ servicename= <unique name of Windows TDI service>

■ assemblylines=<names of AssemblyLine(s) that are to start when the Windows Service starts> ─You can can multiple start, by separating the AssemblyLines by commas in

the list


Installing the Windows Service

■ From the unique solutions directory execute the following command:─ ibmdiservice.exe -i─ This installs the service

■ To uninstall the service, issue the command from the same solutions directory:─ ibmdiservice.exe -u


Confirm installation of the Windows Service■ You can always check the Services panel to see the service was added:


IBM Connections/TDI: Property files■ Property files included with the Profiles wizard

─Profiles_tdi.properties ‒ Contains configuration valutes

─map_dbrepos_from.source.properties‒ Contains information about directory mapping (input source) to the

Profiles database.─map_dbrepos_to_source.properties

‒ Contains information about mapping from Profiles database to your directory.

─ validate_dbrepos_fields.properties‒ Used to specify custom filters.


IBM Connections/TDI: Functions files■ Functions files – functions are used to filter, modify, or validate the data prior to

import─ profiles_functions.js─ collect_ldap_dns_generator.js


IBM Connections/TDI – Database Population Wizard■ Included with Connections■ Must be run on a server/workstation where TDI is installed

─ Have the TDI install path available as this is needed during the install■ Needs access to LDAP/AD source

─ The wizard doesn't support Anonymous Bind – be sure you have credentials■ Needs access to Profile database


Database JDBC Drivers/Files■ DB2

─ db2jcc.jar and db2jcc_license_cu.jar (already included in the TDI jar directory)

■ Oracle─ ojdbc14.jar located in the jdbc/lib in your Oracle install

■ Microsoft SQL─ sqljdbc4.jar─ sqljdbc.jar─Downloadable from either MS SQL server or from Microsoft.com


Importing Connections Resources■ Select Import


Importing profiles_tdi.xml■ Browse for the profiles_tdi.xml file ■ Import into your Connections project


Connections AssemblyLines!■ Once the import is complete – you'll see that there's now a VERY long list of

AssemblyLines you can customize!


Property Files■ Import will include Property Files■ Can be updated with login/password and reused




■ Installation

■ Examples

■ Resources



Additional Resources

■ IBM Tivoli Directory Integrator Users Group - http://www.tdi-users.org■ Domino Integration Specific Info: http://sites.google.com/site/dominointegration■ IBM DeveloperWorks – Tivoli - https://www.ibm.com/developerworks/tivoli/■ http://www.ibm.com/developerworks/tivoli/library/t-tdi-lotus/■ http://www.ibm.com/developerworks/lotus/documentation/connectionstdi/■ http://www-10.lotus.com/ldd/lcwiki.nsf/dx/how-to-use-and-troubleshoot-the-profiles-

population-wizard■ TDI User Forum - http://groups.google.com/group/ibm.software.network.directory-integrator/topics?

gvc=2■ Learning TDI - http://www.tdi-users.org/twiki/bin/view/Integrator/LearningTDI■ Domino Integration with TDI - http://www.tdi-users.org/twiki/bin/view/Integrator/Domino■ TDI Redbooks/Redpieces –

http://www.redbooks.ibm.com/cgi-bin/searchsite.cgi?query=tivoli+AND+directory+AND+integrator■ And Marie and Tom have done a series of TDI articles on at: http://www.crashtestchix.com■ Thanks to Eddie Hartman “TDI Story Teller” ([email protected])

■ Thanks also to Mitch Cohen (@curiousmitch) who helped us with the Connections information


Related Sessions

■ JMP203: Getting the most out of IBM Connections Profiles and Tivoli Directory Integrator

■ SHOW300: Become a Connections Hero - Install and Configure IBM Connections

■ ID232: IBM Connections Deployment and Performance Planning■ AD201: Integrate and Customize IBM Connections 3.0.1■ GURU101: Gurupalooza




■ Installation

■ Examples

■ Resources



Questions? Comments?

Marie Scott – [email protected]: CrashTestChix - http://www.crashtestchix.comTwitter: marie_scott

Thomas “Duffbert” Duff - [email protected]: Duffbert's Random Musings - http://www.duffbert.comTwitter: duffbert

Please remember to fill out your evaluations!

BP201 – Assemble 'n Go: Integrate Data and Directories with Tivoli Directory Integrator


Legal disclaimer© IBM Corporation 2012. All Rights Reserved.

The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.

IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.

All references to topchefs.com refer to a fictitious company and are used for illustration purposes only.

session # bp201 – assemble 'n go: integrate data and ...domino directory - need to update ibm...

Documents