mdm smart search configuration - informatica search... · page 5 of 27 4. configuring smart search...

of 27

MDM Smart Search Configuration

1. REFERENCE DOCUMENTS .............................................................................................................3

2. SMART SEARCH OVERVIEW .........................................................................................................3

3. WEBSPHERE PRE-CONFIGURATION .........................................................................................3

The following sections list WebSphere settings that must be set prior to beginning

configuring Smart Search in MDM. ....................................................................................................3

3.1 Registering Apache Solr MBeans ................................................................................................................ 3

3.2 JVM generic arguments .............................................................................................................................. 4

4. CONFIGURING SMART SEARCH ...................................................................................................5

4.1 Updating cmxcleanse.properties file .......................................................................................................... 5

4.2 Updating cmxserver.properties file ............................................................................................................ 6

4.3 Creating Zookeeper directory ..................................................................................................................... 7

4.4 Configuring Process Server ......................................................................................................................... 8

4.5 Restart WebSphere .................................................................................................................................... 9

4.6 Checking solr admin console ...................................................................................................................... 9

5. RESETTING SMART SEARCH ..................................................................................................... 11

5.1 Cleaning C_REPOS_CO_CS_CONFIG Metadata table ................................................................................. 11

5.2 Re-generating Business Entity Schema ..................................................................................................... 12

5.3 cmxcleanse.properties ............................................................................................................................. 15

of 27

5.4 cmxserver.properties ............................................................................................................................... 16

5.5 Stopping WAS service ............................................................................................................................... 17

5.6 Cleaning solr/zkp folders in MDM ............................................................................................................ 18

5.7 Stating WAS ............................................................................................................................................. 20

5.8 Checking solr admin console. ................................................................................................................... 20

5.9 Running Index Job in MDM Hub ............................................................................................................... 21

6. TURNING OFF SMART SEARCH ................................................................................................ 24

The following sections list Steps to turn off Smart Search if needed. ................................... 24

of 27

1. REFERENCE DOCUMENTS Document MDM_ProvisioningToolGuide_en.pdf MDM_Configuaration_en.pdf

You can follow the MDM configuration guide to enable the Solr search / Elastic search incase of

recent versions of MDM. Steps are mentioned in the configuration guide as well.

2. SMART SEARCH OVERVIEW

Smart Search provides the ability to perform searching within a specific searchable business entity type

or within all the searchable business entity types based on a string value. The Informatica Data Director

application displays the matching searchable fields. The results are ranked based on how closely the

results match the string value. Filters can be applied to identify more relevant results.

Smart Search uses solr as its search platform, solr is based on Apache Lucene.

In order to use smart search, configurations on the process server and running jobs for indexing the data

must be done. The following sections outline these steps.

3. WEBSPHERE PRE-CONFIGURATION

The following sections list WebSphere settings that must be set prior to beginning configuring

Smart Search in MDM.

3.1 Registering Apache Solr MBeans

To support smart search in Informatica Data Director, register Apache Solr MBeans. In the WebSphere

Console, check if following property is set as mentioned below in the custom properties of the Java Virtual

Machine in WebSphere.

javax.management.builder.initial=javax.management.MBeanServerBuilder

of 27

3.2 JVM generic arguments

The solr target directory needs to be added to the JVM generic arguments -Dsolr.solr.home=<INFAMDM root>\hub\cleanse\solr

For example: D:\infamdm\hub\cleanse\solr

The solr folder gets automatically created in the appropriate target when smart search is enabled.

of 27

4. CONFIGURING SMART SEARCH

This section describes the steps to be followed for Smart Search Index configuration.

4.1 Updating cmxcleanse.properties file

This section follows the document “MDM_1010_ProvisioningToolGuide_en.pdf”, Chapter 8, Step 1, #8. 1. On the MDM Application server, navigate to the MDM Installation directory,

D:\infamdm\hub\cleanse\resources

.

2. Using Notepad, open cmxcleanse.properties

3. Update the following existing Property values:

• cmx.ss.enabled

Update the value from false to true

• zookeeper.tickTime

Update the value from 2000 to 20000

• facet.infacount

Update the value from 1000 to 5000

4. Add the following new properties: •

solr.allowAdminConsole=true

of 27

• MAX_INITIAL_RESULT_SIZE_TO_CONSIDER =130

5. Save the changes and close the file

4.2 Updating cmxserver.properties file

This section follows the document “MDM_1010_ProvisioningToolGuide_en.pdf”, Chapter 8, Step 3, #2 1. On the MDM Application server, navigate to the MDM Installation directory,


2. Using Notepad, open cmxserver.properties.

of 27

3. Update the following existing Property values:

• cmx.ss.enabled

Update the value from false to true

4. Add the following new properties:

• cmx.server.batch.smartsearch.initial.block_size =2000

4.3 Creating Zookeeper directory

Create a folder for Zookeeper in Cleanse folder to store the in-memory snapshots of the data and the

transactional log of updates to the database.

1. On the MDM application server, navigate to the cleanse folder:

<MDM root>\hub\cleanse For example: D:\infamdm\hub\cleanse

2. Create new folder, zkp.

of 27

Note: Please make sure that the name of the directory is not too long (greater than 7 letters) and

doesn’t contain underscore/hyphens in its name.

4.4 Configuring Process Server

This section follows the document “MDM_1010_ProvisioningToolGuide_en.pdf”, Chapter 8, Step 1, #4

1. In MDM Hub console acquire write lock. 2. Navigate to the Process Server workbench 3. Click on Edit for the Process Server 4. The following properties in Process Server must be enabled: Enable Search Processing

Enable ZooKeeper

5. When ZooKeeper is enabled, continue configuring the Process Server: ZooKeeper ID: 1 ZooKeeper Client Port: 2181

of 27

ZooKeeper Leader Port: 0 ZooKeeper Follower Port : 0 ZooKeeper Directory : D:\infamdm\hub\cleanse\zkp ( created in step 4.3)

Note: Make sure that port assigned for ZooKeeper client port (2181) is open on server. This can be

checked by netstat or telnet command by server admin.

4.5 Restart WebSphere

Restart WebSphere to affect all changes for Smart Search.

4.6 Checking solr admin console

To log into the solr admin console, go to URL : http://<MDMHOST:PORT>/MDESolr/#/~cloud

http://dcamdm02:9080/MDESolr/#/~cloud

Navigate to cloud → graph and you will see all subject area shard folder and its status. This takes some

times to get created after WebSphere restart.





of 27

Once the all shard folder are created, it can be validate at below location on server:

D:\infamdm\hub\cleanse\solr

of 27

5. RESETTING SMART SEARCH

It is best practice to disable smart search while running large data volumes in MDM, such as IDL.Also

enabling Smart Search is inefficient prior to the completion of the IDL, as thousands of records will merge

away and not need to be indexed for Smart Search

While configuring Smart Search at Aflac, several issues were resolved by the IPS team and some

required further assistance with Informatica Global Customer Services (GCS). A support ticket was

opened, Informatica Support Case number 01915350 for Smart Search.

5.1 Cleaning C_REPOS_CO_CS_CONFIG Metadata table

1. Log into the CMX_ORS database 2. Check C_REPOS_CO_CS_CONFIG metadata table

3. If this table contains data, truncate it.

4. Confirm the table is empty.

of 27

5.2 Re-generating Business Entity Schema

In order to re-populate c_repos_co_cs_config table, we need to re-generate business entity schema from

IDD config screen.

1. Login to IDD config screen , the URL will be something like

http://dcamdm02:9080/config/config_login.jsf

2. Select the IDD application, Alfac IDD App,

of 27

3. Open the Application State menu and select “Not Deployed”

4. Once selected, the application will get un-deployed and deployment status will say as “Not

Deployed”.

5. Click “Generate Business entity Schema “

of 27

6. After generateingBES. Select Close in the Informatica MDM Data Director pop-up window.

7. In the CMX_ORS database, confirm C_REPOS_CO_CS_CONFIG table has been

repopulated

of 27

8. In the IDD Configuration URL, select Application State → Full Deployment

9. Once the Aflac IDD App is successfully deployed, the status will be “Full Deployment” and

status will be valid (green check mark)

5.3 cmxcleanse.properties

1. Log into MDM application server 2. Go to <MDM INSTALL>/hub/cleanse/resources folder ((D:\infamdm\hub\cleanse\resources) 3. Using Notepad, open the cmxcleanse.properties file

of 27

4. Check if smart search is enabled

5.4 cmxserver.properties

1. Log into the MDM application server 2. Go to <MDM INSTALL>/hub/server/resources folder (D:\infamdm\hub\server\resources) 3.

Using Notepad, open cmxserver.properties

of 27

4. Make sure that Smart Search properties are enabled (true):

5.5 Stopping WAS service

1. Log into the MDM application server 2. Open Control Panel > Services 3. Stop WAS service. DO NOT click Start, need to remove files from Smart Search folders

of 27

5.6 Cleaning solr/zkp folders in MDM

1. On the MDM application server, go to D:/infamdm/hub/cleanse folder and take backup of the solr &

zkp folders

2. Once the files are moved, , clean both of these folders (if nothing present in these folder , skip

this process)

Before cleaning the solrfolder:

of 27

After cleaning all files are deleted from the solr folder:

Before cleaning the zkp folder:

of 27

After cleaning zkp folder, all files are deleted:

5.7 Stating WAS

1. On the MDM application server, navigate to Control Panels > Services 2. Start WAS service This would take some time to re-start WAS and MDM

hub.

5.8 Checking solr admin console.

1. Log into the solr admin console, go to URL : <MDMHOST:PORT>/MDESolr/#/~cloud 2.

Navigate to cloud → graph and you will see all subject area folder status:

This shows that solr directory is connected to zookeeper and will get populated with shards.

Once the all shard folder are created, it can be validate at below location on server:

D:\infamdm\hub\cleanse\solr

of 27

5.9 Running Index Job in MDM Hub

1. Log into MDM Hub Console 2. Go to Batch Viewer workbench

3. Connect to cmx_ors database

of 27

4. Go to Batch Viewer → Party → Initially Index Smart Search data 5. Select Initial Index data for Search data Party Writing

6. To execute this job, select on “Initial Index data for Search data Party” and click on “Execute

Batch”

of 27

7. When executing, the status the current status will displays as “Processing”. Keep refreshing for

current status by clicking on “Refresh Status”

8. When successfully finished, it will show below details:

9. Similarly run Index job for Party Writing number:

10. Go to Batch Viewer → Party Writing Number → Initially Index Smart Search data 11. Select Initial Index data for Search data Party Writing Number

of 27

12. To execute this job click on “Execute Batch”

13. When executing, the status the current status will displays as “Processing”. Keep refreshing for

current status by clicking on “Refresh Status”

14. When successfully finished, it will show below details:

15. This completes the Smart Search configuration. After enabling smart search, login to UI and test

smart search.

6. TURNING OFF SMART SEARCH

The following sections list Steps to turn off Smart Search if needed.

1. On the MDM Application server, navigate to the MDM Installation directory,


of 27

2. Open this cmxcleanse.properties file and make sure that the below properties is set as:

cmx.ss.enabled=false

3. On the MDM Application server, navigate to the MDM Installation directory,

D:\infamdm\hub\server\resources

of 27

4. Open this cmxserver.properties file and make sure that the below properties is set as:

cmx.ss.enabled=false

5. Disable ZooKeeper in MDM Hub :

of 27

6. Restart WebSphere which will disable Smart Search.

mdm smart search configuration - informatica search... · page 5 of 27 4. configuring smart search...

Documents