iconuk 2014 connections migration tips and tricks

IBM Connections Migration – Some Tips for your successful Connections Migration

Learn about your Known Unknowns and your Unknown Unknownsand where to look for them

ICON UK - Sept. 12, 2014

What Are the Two Most important Considerations?

If it’s real estate - location, location, location … (but we don’t care about real estate right now)

So we think of

IBM Connections Version, Version, Version . . . .You Have Choices And Challenges - Depending on Which Version You Are Coming From

&

Parallel or In-Place Migration


What needs to be migrated?

1. Your DB source

2. Shared Files (uploads, WIKIS, FILES, ACTIVITES, etc….)

3. Connections Settings (Connections XML files, proxy configurations, etc.)

4. Notification Settings/Strings (the emails your system sends out)

5. Media Gallery settings

6. Customizations (no matter how ugly …)

7. IHS Settings

8. WebSphere Security / Admin structure

9. Third Party Software Products / Media players

10. COGNOS … (Again - I pitty you …)

11. CCM (depending on originating version)

What do you NOT migrate:

Search indexes

Local Data Stores (are recreated upon install)


You Need a Plan

Sample Plan - Three phases:

Phase 1.

New System - WebSphere install

●Install WebSphere 8.0.0.8 on DM / Managed Node

●Install WebSphere 7.0.025 on IBM Docs server

●Create dB for Connections (new dB)

●TDI INstall - configuration - populate Profiles

●Install IBM HTTP Server

●Install IBM Connections: include CCM/Filenet

●Base configure of Connections

●Configure IHS, CCM, Cognos

●Install 3rd Party Products

Phase 2.

●Adjust configuration to match

existing Connections settings (export/Import)

●Apply any customizations

●Mail/notifications settings

●ICMail install and configuration

Phase 3.

Test migration:

●DATA CLEAN-UP on originating system

●Make copy of existing DB2 dB to new DB2 server

●Make copy of content stores from old environment to

new server

●Make backup of existing (new) V4.5 DB2 databases

●Put old DB2 (V4.x) onto new DB2 server and do test

migration / upgrade to V4.5 schema

●Start new servers and test/verify that data migrated

clean

Migration:

●Shut down V4.0 enviroment

●Shut down V4.5 environment

●re-copy DB2 dB to new server

●Copy delta of new files from V3.x to new server

●Reconfigure V4.5 to use the original url

●Change DNS to point to new server

●Migrate DB2 data

●Start new server

●Test/verify

Note: A “real” project plan

has WAAAY more details!


Your first and most important decision is HOW you intend to migrate

Parallel or In-Place

Parallel Migration

Pros:● No time limit that forces you into a specific schedule● Gives you opportunity to test and verify freely● Makes it possible to do test runs for the migration● Gives you a test bed to verify all the settings and configuration● Leaves you a working system to fall back onto

Cons:● Doubles your HW and disk requirements for the duration


In-Place Migration

Pros:● No additional HW required

Cons:● Everything else!● Requires an uninstall of Connections, upgrade of WebSphere and IHS then re-install Connections● Connections unavailable during the whole process - from deinstall to build to test● Might require an upgrade of the DB2 version● No easy fall-back should the migration not be successful● No good way to test the outcome ahead of time - scheduling is difficult● Might require OS upgrade (depending on OS)

!YWTATOAAC!

(You Want To Avoid This Option At All Costs!)


Versions and Migration Scenarios - The Ugly Ones

Originating Version

Target Version Steps

Cnx V3.0.1 Cnx V4.0.x ● Single step - use the V4 wizards to migrate directly.● If you are not V3.0.1 -> upgrade first

Cnx V4.5.x ● Two migration steps - Migrate DB from V3.1->V4 and then to V4.5. ● You need to first use the V4.0 wizard, then the V4.5 wizard. ● There will be some missing databases that are new to V4 & V4.5 that you will

need to create separately … (more below)*** In short .. I pity you ***

Cnx V5.x ● Basically the same as V3->V4.5, just that the V5 wizards are capable of migrating you from V4.0 directly to V5 without having to migrate/upgrade to V4.5 first.

*** Again, I pity you ***


Versions and Migration Scenarios - The Less Troublesome

Originating Version


Cnx V4.0.x Cnx V4.5.x ● Single step - use the V4.5 wizards to migrate directly● Cnx 4.0 needs to be at least CR2 for the Content stores to be formatted

correctly for an upgrade

Cnx V5.x ● Single step - use the V5 wizards to migrate directly

Cnx V4.5 Cnx V5.x ● Single step - use the V5 wizards to migrate directly


Your Database Migration

The most important and probably most difficult part of any Connections migration is the database.

It takes the longest, needs the most babysitting and has the most potential pitfalls.

The Connections Database Wizard supplied with each version of IBM Connections is in charge of

the migration steps. You need to use the wizard of the version you are MIGRATING TO or it will

not work.

Depending on the version you are migrating from and the version you are migrating to you could

have several steps to deal with, let’s take a look:


DB2 Migration - Continued:

Originating Version


Cnx V3.0.1 Cnx V4.0.x ● Single step - use the V4 wizards to migrate directly.● If you are not V3.0.1 -> upgrade first

Cnx V4.5.x ● Two migration steps - Migrate DB from V3.1->V4 and then to V4.5. ● You need to first use the V4.0 wizard, then the V4.5 wizard. ● There will be some missing databases that are new to V4 & V4.5 that you will

need to create separately … (more below)*** In short .. I pity you ***

Cnx V5.x ● Basically the same as V3->V4.5, just that the V5 wizards are capable of migrating you from V4.0 directly to V5 without having to migrate/upgrade to V4.5 first.

*** Again, I pity you ***


PREPARATION

It’s what for dinner ……. and breakfast, lunch … snacks … seconds …

What this means is - you will have no rest unless you prepare the data first

(note: Gandalf will not help you …..)


Data Preparation

If you have already migrated the databases once (or twice?) previously … you will likely have some garbage in the databases you need to review.

What to do?

CLEAN UP(just like Momma taught you …)

Even if you have NEVER migrated before .. there can be allot of chaff in the databases and a clean-up & review of your data is in order prior to doing ANYTHING


Data Preparation … Clean-up

Run a user sync - that usually shows up any problems between entries in PROFILES and the other applications. Your most important one is likely NEWS/HOMEPAGE - both applications use the same database and it is also the first database to be migrated. HOMEPAGE which is pretty much your most important database from an end-user's perspective.

Sync command Examples:

First Run the syncAllMembersExtIds commandswsadmin.sh/.bat -lang jython -user wasadmin -password **** -profile newsAdmin.py -c "NewsMemberService.syncAllMemberExtIds()"

Followed by the syncAllMembersByExtId with update triggers:

./wsadmin.sh -lang jython -user wasadmin -password **** -profile newsAdmin.py -c

"NewsMemberService.syncAllMembersByExtId({'updateOnEmailLoginMatch':'true'})"

Review the log files, they will tell you allot about your issues - or the lack thereof


Data Preparation … Clean-up

If you find errors ….. What do you do now?

Look at the accounts creating errors -

• LDAP accounts - Look at whether they might be different, corrupted or … not there anymore• Use a dB tool to open the Connections databases and look at the actual datasets ….• OPEN A PMR WITH IBM - you pay for support so you should use it• Often what you have is just a set of data that are missing some other related data (dB constraints)

and because they are incomplete you are running into issues.

My side story . . . . :I once found a client that had several thousand dormant profiles … all with their last update date set to

the same day ...which happened to be the day the previous system was migrated from V3.01 to V4.0 …..

The Voice of EXPERIENCE tells you:• Just about all problems can be solved with some sql statements, but you will want to have IBM’s

input on this since• Consider doing all this on a copy of your data … the last thing you need is to corrupt your running

system ….


The Database Wizard

The Database Wizard

Has two main functions

1. Creation / Deletion of Connections Databases on the DB server2. Migration/Upgrade of databases of previous releases to the

corresponding release of the Wizard

All sql scripts necessary are actually contained in a subfolder of the unpacked Wizard tself. The Wizard is just a visual front-end that lets you choose the parameters, build the DB2 (or SQL/Oracle) scripts and then executes them.

EXAMPLE ….Let’s look at the real thing!


Database Wizard and Migration

The Voice of Experience …. Some things to take into consideration

DB2: You want to execute the Wizard / SQL scripts using the same account that created the

databases in the first place. A DB2 database has allot of individual items and they all belong to some dentity. Sometimes an account added later with admin rights will not have all the rights necessary to update individual database features … maybe it is just a single field but that can be VERY painful.

If your databases are large (anything over 15 GB is large) you might want consider not using the Wizard, but running the scripts manually so that the wizard does not time out on you. DB2 scripts from the commandline will not time out - they will run to completion

The Wizard will actually create all the scripts for you, in the correct formatting and in the order they need to be run in … all bundled up in one nice old document

NOTE: if you run scripts manually, make sure you add a command to create log files, you HAVE TO REVIEW THEM to be sure everything went well . . . .


DB Migration - Manually

Example for manual scripts:Activities /opt/ibm/db2/V10.1/bin/db2 -td@ -vf connections.sql/activities/db2/upgrade-40-45.sql /opt/ibm/db2/V10.1/bin/db2 -td@ -vf connections.sql/activities/db2/appGrants.sql /opt/ibm/db2/V10.1/bin/db2 -td@ -vf connections.sql/activities/db2/clearScheduler.sql

Blogs /opt/ibm/db2/V10.1/bin/db2 -td@ -vf connections.sql/blogs/db2/upgrade-40-45.sql /opt/ibm/db2/V10.1/bin/db2 -td@ -vf connections.sql/blogs/db2/appGrants.sql

Bookmarks /opt/ibm/db2/V10.1/bin/db2 -td@ -vf connections.sql/dogear/db2/upgrade-40-45.sql /opt/ibm/db2/V10.1/bin/db2 -td@ -vf connections.sql/dogear/db2/appGrants.sql

There is much more, (EXAMPLE ON SCREEN)

A Trick from the wise . . . . . . .Look at the log files (they will be HUGE/LONG) you can’t read it all … just

search for the work “Error” … if that word does not exist you are golden . . . . .


Let’s Migrate some Configurations

• “To automate, or not to automate … that is the question”


Migrate Settings From Old to New

Starting with V4, IBM Connections comes with migration tool that exports “application artifacts” from the originating system. You can then use the same tool on the new system to import those “application artifacts”.

“What are “Application Artifacts”?All (or actually – most) of your configuration files from the WebSphere

Deployment Manager’s LotusConnections-config\ folder (and the sub-folders.)

What does NOT get migrated?• Customizations (=anything in the customizations shared folder)• Any changes you did INSIDE of applications (ear files) • Notification settings / strings 9= the wording in the mails that get sent out)• Profile lay-out settings and customized fields

I !SO! hope you did not do any of those ….


Profiles

A quick word on Profiles Design

Most environments have done some changes to the default profiles setup and lay-out, everything has changed, but some things are the same.

Any changes you made via TDI – mapping specific LDAP elements to specific Profiles fields – those all come over, if you reconfigure your TDI correctly

What has changed that you need to look at:• If migrating to V5 … EVERYTHING has changed, basically you get to do it all

over in a new system . . But I find the new way easier to deal with and to accomplish.

• If migrating from V4 -> V4.5 you are in luck, it is almost the same• If migrating from V3 .. Well, you get to do it al over again anyway• Read this in the V5 Wiki: Customizing Profiles


http://www-01.ibm.com/support/knowledgecenter/SSYGQH_5.0.0/admin/customize/c_admin_profiles_customizing.dita


How do I do this?

*** MAKE A BACKUP FIRST … I BEG OF YOU! ***

I generally do a WebSphere Backupconfg.bat/.sh

Go to your [\Connections InstallRoot\migration] folder, the command is:

[migration.sh/bat lc-export]

This exports (almost) all the files you need to the [\Connections InstallRoot\migration\work] folder. This process creates a log file -> CHECK IT!!! . You can find it in your OS account’s [HOME FLDER]. Take a copy of the [\work] folder and put it in the same location on the target system, then run

[migration.sh/bat lc-import -DDMUserid=wasadmin -DDMPassword=*******]ICON UK - Sept. 12, 2014

In reality you really want that opportunity to

review all settings. AND .. There are a few new

ones you don’t know of.


OK, the previous two slides are from the Connections WIKI, now comes something from Dr. Vic’s vast experience – this is why I have scar tissue:

Don’t Do It80% of the time it works OK.

20% of the time it screws up your environment. Those screw-ups are really painful

My most recent case … the update totally mashed my events-config.xml file (there were settings in there nobody has seen before). This can especially happen if you are dealing with an environment that was migrated previously using the same tool.

I don’t blame IBM … 80% is a real good ratio! But they just can’t test ALL scenarios and there is no accounting for human .. ahem … inventiveness. The scripts check for formatting but NOT FOR INCORRECT CONTENT.

Life all those changes by hand .. Go config file by config file. That also gives you the opportunity to review the settings and make a determination of they are valid or not.


Them Files – They have to Go Somewhere

• The “Other White Meat” or How to Migrate The Need To go


Share File Space

The “Other White Meat” refers to the share file space .. Also known as your shared data.

In essence this is simply a copy-and-paste operation. You want to move the shared file structure exactly AS IS from the originating server to the new server

Alternatively – if you have that file shared someplace – you could just re-mount that folder to the new server …but I am not a friend of this option.

Why? Hhmm …. “What if ..”

• Your migration somehow fails and now you have to recover• During your failed migration the serves “did something” to your files and

now .. You get to go back to a back-up .. Which is hopefully recent.


Files – More White Meat

How Do You Know It Worked?

• Simple .. Look for your files and make sure you can download them.

• Check if the HIS server – which you hopefully have mapped to do file downloads from the file share directly – actually gives you files. If something is off, the files you download will all have a 0 byte size …

• Also .. If something is off all those images you use to decorate your wine tasting communities and the cat videos you have secretly been hoarding in your private community will not show ….

Missing Cat Videos – A Dead Giveaway!

You might also see errors in

the WebSphere sysemOut.log

files …..ICON UK - Sept. 12, 2014

Customizations – What to Look Out For

• Don’t just throw your previous version onto the server ….


Customizations

We can’t cover ALL customizations but we can touch on two REALLY important items that everybody deals with:

header.jsp & footer.jsp

Just about EVERYBODY makes some changes to these files. Here is what to look out for:

• Header.jsp and footer.jsp are specific to each version and (sometimes) CR of IBM Connections

• Much of the functionality of IBM Connections depends on having the correct header.jsp & footer.jsp with the elements/code in them that Connections needs to run correctly

• Even when just doing a CR install, you should ALWAYS check the applications for changes and whether the header or footer jsp files have changed . . . . .

• I HOPE that you have all changes documented . . . . .


Customizations

This is what I do:

• Step 1: Compare your customized jsp’s to the non-customized file on your existing Connections install version. This will give you the changes you have in your system. You can now review them AND DOCUMENT THEM

• Step 2: Compare the vanilla versions of the jsp’s between the originating and target IBM versions. This will give you an idea of what is new and where there are changes. That way you can tell if you need to slot your changes into a different place

• Step 3: Review any custom CSS files you might be referring to and check for potential issues (files, locations, color changes …)

• Step 4:If you have many changes, port your changes over bits and pieces at a time. If you only have few or a single change, implement it and DOCUMENT IT!


Media Gallery – What is New?

Just a few words on the Media Gallery …

• If you are migrating to V4.5 -> nothing special, just port over your custom player, and custom terms (if you have any)

• Does not exist in V5 anymore, it is replaced with the Thumbnail Gallery• You can use custom media players in V5 if you want – but my suggestion is

to test it in a test environment first, to make sure whatever version of product you are using is still working well in a new Connections Version

Review this WIKI entry for V5 media gallery migrations – you basically back-up your applications and then review them.


http://www-01.ibm.com/support/knowledgecenter/SSYGQH_5.0.0/admin/migrate/t_prepare_media_gallery.dita

CCM – FileNet and the changes …..

• Don’t you just LOVE FileNet?


FileNet / CCM – The Steps Necessary

FileNet is one of the systems where the migration is not that hard .. You only really have to do these steps for V5 . .

Here your Steps:• Install FileNet – to the correct version your system needs with all FPs - as a NEW

DEPLOYMENT• When installing FileNet then point them to the dB of the V4.5 system (FNGCD & FNOS)• Make sure you use THE SAME FileNetAdmin account – it makes your life easier• You do not have to create a P8 domain, Global Configuration Data (GCD) or create an

Object Store and Add-Ons -> they all already exist in the V4.5 databases.• Back-up your Existing/New install!!!!! - area [x:\IBM\Connections\data\shared\ccm] and

save it!, also back-up the x:\IBM\Connections\addons\ccm] folder with all content• Copy the FileNet storage to the new server in the folder

[x:\IBM\Connections\data\shared\ccm] • Migrate the encryption keys from your old system to the new -> the location is on the

Deployment manager: [x:\IBM\Connections\addons\ccm\ContentEngine\tools\configure\profiles\CCM\ear]


FileNet / CCM – The Steps Necessary

Continued . . . . .

• Run the following command in the [x:\IBM\Connections\addons\ccm\ContentEngine\lib]

• java -jar BootstrapConfig.jar -e /temp1_device/Engine-ws.ear -j /temp2_device/Engine-ws.ear

• Go to the IBM WebSphere Console, Applications [FileNetEngine] and Update (replace entire application) with the NEWLY CREATED .ear file [/temp2_device/Engine-ws.ear]

• Copy the file [x:\IBM\WebSphere\AppServer\profiles\Dmgr01\config\cells\CELLNAMEl\fileRegistry.xml] from the V4.5 to the V5 server in the same location -> MAKE A BACKUP OF THE FILE YOU ARE REPLACING

• Sync the Nodes and restart the systemICON UK - Sept. 12, 2014

Cognos ….

• I Don’t Want To Talk About It …….


Cognos .. What to do

What is there to do?• For a straight forward migration – Nothing, all the data necessary is

contained in the Metrics database• You do not need to migrate the Cognos Content Store (the database) – it

does not give you anything and makes your life difficult …• When installing Connections on the new server, either already have

migrated a copy of the Cognos database over OR point Cognos to the dB on the V4/4.5 database server. -> I prefer to migrate ahead of time.

• If you have customized reports .. There is a bit more to do

Sounds simple … don’t it?The customized Reports are a bit of a pin, follow this in the WIKI …..


http://www-01.ibm.com/support/knowledgecenter/SSYGQH_5.0.0/admin/migrate/t_migrate_newly_created_cognos_content.dita

About me . . .

Victor Toal aka “Dr. Vic”

[email protected]: vtoalSkype: vtoal


mailto:[email protected]

http://www.linkedin.com/in/victortoal/

http://www.linkedin.com/in/victortoal/

http://www.toalsys.com/

http://www.toalsys.com/