mdm smart search configuration - informatica search... · page 5 of 27 4. configuring smart search...
Post on 25-Jun-2020
38 Views
Preview:
TRANSCRIPT
Page 1 of 27
MDM Smart Search Configuration
1. REFERENCE DOCUMENTS .............................................................................................................3
2. SMART SEARCH OVERVIEW .........................................................................................................3
3. WEBSPHERE PRE-CONFIGURATION .........................................................................................3
The following sections list WebSphere settings that must be set prior to beginning
configuring Smart Search in MDM. ....................................................................................................3
3.1 Registering Apache Solr MBeans ................................................................................................................ 3
3.2 JVM generic arguments .............................................................................................................................. 4
4. CONFIGURING SMART SEARCH ...................................................................................................5
4.1 Updating cmxcleanse.properties file .......................................................................................................... 5
4.2 Updating cmxserver.properties file ............................................................................................................ 6
4.3 Creating Zookeeper directory ..................................................................................................................... 7
4.4 Configuring Process Server ......................................................................................................................... 8
4.5 Restart WebSphere .................................................................................................................................... 9
4.6 Checking solr admin console ...................................................................................................................... 9
5. RESETTING SMART SEARCH ..................................................................................................... 11
5.1 Cleaning C_REPOS_CO_CS_CONFIG Metadata table ................................................................................. 11
5.2 Re-generating Business Entity Schema ..................................................................................................... 12
5.3 cmxcleanse.properties ............................................................................................................................. 15
Page 2 of 27
5.4 cmxserver.properties ............................................................................................................................... 16
5.5 Stopping WAS service ............................................................................................................................... 17
5.6 Cleaning solr/zkp folders in MDM ............................................................................................................ 18
5.7 Stating WAS ............................................................................................................................................. 20
5.8 Checking solr admin console. ................................................................................................................... 20
5.9 Running Index Job in MDM Hub ............................................................................................................... 21
6. TURNING OFF SMART SEARCH ................................................................................................ 24
The following sections list Steps to turn off Smart Search if needed. ................................... 24
Page 3 of 27
1. REFERENCE DOCUMENTS Document MDM_ProvisioningToolGuide_en.pdf MDM_Configuaration_en.pdf
You can follow the MDM configuration guide to enable the Solr search / Elastic search incase of
recent versions of MDM. Steps are mentioned in the configuration guide as well.
2. SMART SEARCH OVERVIEW
Smart Search provides the ability to perform searching within a specific searchable business entity type
or within all the searchable business entity types based on a string value. The Informatica Data Director
application displays the matching searchable fields. The results are ranked based on how closely the
results match the string value. Filters can be applied to identify more relevant results.
Smart Search uses solr as its search platform, solr is based on Apache Lucene.
In order to use smart search, configurations on the process server and running jobs for indexing the data
must be done. The following sections outline these steps.
3. WEBSPHERE PRE-CONFIGURATION
The following sections list WebSphere settings that must be set prior to beginning configuring
Smart Search in MDM.
3.1 Registering Apache Solr MBeans
To support smart search in Informatica Data Director, register Apache Solr MBeans. In the WebSphere
Console, check if following property is set as mentioned below in the custom properties of the Java Virtual
Machine in WebSphere.
javax.management.builder.initial=javax.management.MBeanServerBuilder
Page 4 of 27
3.2 JVM generic arguments
The solr target directory needs to be added to the JVM generic arguments -Dsolr.solr.home=<INFAMDM root>\hub\cleanse\solr
For example: D:\infamdm\hub\cleanse\solr
The solr folder gets automatically created in the appropriate target when smart search is enabled.
Page 5 of 27
4. CONFIGURING SMART SEARCH
This section describes the steps to be followed for Smart Search Index configuration.
4.1 Updating cmxcleanse.properties file
This section follows the document “MDM_1010_ProvisioningToolGuide_en.pdf”, Chapter 8, Step 1, #8. 1. On the MDM Application server, navigate to the MDM Installation directory,
D:\infamdm\hub\cleanse\resources
.
2. Using Notepad, open cmxcleanse.properties
3. Update the following existing Property values:
• cmx.ss.enabled
Update the value from false to true
• zookeeper.tickTime
Update the value from 2000 to 20000
• facet.infacount
Update the value from 1000 to 5000
4. Add the following new properties: •
solr.allowAdminConsole=true
Page 6 of 27
• MAX_INITIAL_RESULT_SIZE_TO_CONSIDER =130
5. Save the changes and close the file
4.2 Updating cmxserver.properties file
This section follows the document “MDM_1010_ProvisioningToolGuide_en.pdf”, Chapter 8, Step 3, #2 1. On the MDM Application server, navigate to the MDM Installation directory,
D:\infamdm\hub\cleanse\resources
2. Using Notepad, open cmxserver.properties.
Page 7 of 27
3. Update the following existing Property values:
• cmx.ss.enabled
Update the value from false to true
4. Add the following new properties:
• cmx.server.batch.smartsearch.initial.block_size =2000
4.3 Creating Zookeeper directory
Create a folder for Zookeeper in Cleanse folder to store the in-memory snapshots of the data and the
transactional log of updates to the database.
1. On the MDM application server, navigate to the cleanse folder:
<MDM root>\hub\cleanse For example: D:\infamdm\hub\cleanse
2. Create new folder, zkp.
Page 8 of 27
Note: Please make sure that the name of the directory is not too long (greater than 7 letters) and
doesn’t contain underscore/hyphens in its name.
4.4 Configuring Process Server
This section follows the document “MDM_1010_ProvisioningToolGuide_en.pdf”, Chapter 8, Step 1, #4
1. In MDM Hub console acquire write lock. 2. Navigate to the Process Server workbench 3. Click on Edit for the Process Server 4. The following properties in Process Server must be enabled: Enable Search Processing
Enable ZooKeeper
5. When ZooKeeper is enabled, continue configuring the Process Server: ZooKeeper ID: 1 ZooKeeper Client Port: 2181
Page 9 of 27
ZooKeeper Leader Port: 0 ZooKeeper Follower Port : 0 ZooKeeper Directory : D:\infamdm\hub\cleanse\zkp ( created in step 4.3)
Note: Make sure that port assigned for ZooKeeper client port (2181) is open on server. This can be
checked by netstat or telnet command by server admin.
4.5 Restart WebSphere
Restart WebSphere to affect all changes for Smart Search.
4.6 Checking solr admin console
To log into the solr admin console, go to URL : http://<MDMHOST:PORT>/MDESolr/#/~cloud
http://dcamdm02:9080/MDESolr/#/~cloud
Navigate to cloud → graph and you will see all subject area shard folder and its status. This takes some
times to get created after WebSphere restart.
Page 10 of 27
Once the all shard folder are created, it can be validate at below location on server:
D:\infamdm\hub\cleanse\solr
Page 11 of 27
5. RESETTING SMART SEARCH
It is best practice to disable smart search while running large data volumes in MDM, such as IDL.Also
enabling Smart Search is inefficient prior to the completion of the IDL, as thousands of records will merge
away and not need to be indexed for Smart Search
While configuring Smart Search at Aflac, several issues were resolved by the IPS team and some
required further assistance with Informatica Global Customer Services (GCS). A support ticket was
opened, Informatica Support Case number 01915350 for Smart Search.
5.1 Cleaning C_REPOS_CO_CS_CONFIG Metadata table
1. Log into the CMX_ORS database 2. Check C_REPOS_CO_CS_CONFIG metadata table
3. If this table contains data, truncate it.
4. Confirm the table is empty.
Page 12 of 27
5.2 Re-generating Business Entity Schema
In order to re-populate c_repos_co_cs_config table, we need to re-generate business entity schema from
IDD config screen.
1. Login to IDD config screen , the URL will be something like
http://dcamdm02:9080/config/config_login.jsf
2. Select the IDD application, Alfac IDD App,
Page 13 of 27
3. Open the Application State menu and select “Not Deployed”
4. Once selected, the application will get un-deployed and deployment status will say as “Not
Deployed”.
5. Click “Generate Business entity Schema “
Page 14 of 27
6. After generateingBES. Select Close in the Informatica MDM Data Director pop-up window.
7. In the CMX_ORS database, confirm C_REPOS_CO_CS_CONFIG table has been
repopulated
Page 15 of 27
8. In the IDD Configuration URL, select Application State → Full Deployment
9. Once the Aflac IDD App is successfully deployed, the status will be “Full Deployment” and
status will be valid (green check mark)
5.3 cmxcleanse.properties
1. Log into MDM application server 2. Go to <MDM INSTALL>/hub/cleanse/resources folder ((D:\infamdm\hub\cleanse\resources) 3. Using Notepad, open the cmxcleanse.properties file
Page 16 of 27
4. Check if smart search is enabled
5.4 cmxserver.properties
1. Log into the MDM application server 2. Go to <MDM INSTALL>/hub/server/resources folder (D:\infamdm\hub\server\resources) 3.
Using Notepad, open cmxserver.properties
Page 17 of 27
4. Make sure that Smart Search properties are enabled (true):
5.5 Stopping WAS service
1. Log into the MDM application server 2. Open Control Panel > Services 3. Stop WAS service. DO NOT click Start, need to remove files from Smart Search folders
Page 18 of 27
5.6 Cleaning solr/zkp folders in MDM
1. On the MDM application server, go to D:/infamdm/hub/cleanse folder and take backup of the solr &
zkp folders
2. Once the files are moved, , clean both of these folders (if nothing present in these folder , skip
this process)
Before cleaning the solrfolder:
Page 19 of 27
After cleaning all files are deleted from the solr folder:
Before cleaning the zkp folder:
Page 20 of 27
After cleaning zkp folder, all files are deleted:
5.7 Stating WAS
1. On the MDM application server, navigate to Control Panels > Services 2. Start WAS service This would take some time to re-start WAS and MDM
hub.
5.8 Checking solr admin console.
1. Log into the solr admin console, go to URL : <MDMHOST:PORT>/MDESolr/#/~cloud 2.
Navigate to cloud → graph and you will see all subject area folder status:
This shows that solr directory is connected to zookeeper and will get populated with shards.
Once the all shard folder are created, it can be validate at below location on server:
D:\infamdm\hub\cleanse\solr
Page 21 of 27
5.9 Running Index Job in MDM Hub
1. Log into MDM Hub Console 2. Go to Batch Viewer workbench
3. Connect to cmx_ors database
Page 22 of 27
4. Go to Batch Viewer → Party → Initially Index Smart Search data 5. Select Initial Index data for Search data Party Writing
6. To execute this job, select on “Initial Index data for Search data Party” and click on “Execute
Batch”
Page 23 of 27
7. When executing, the status the current status will displays as “Processing”. Keep refreshing for
current status by clicking on “Refresh Status”
8. When successfully finished, it will show below details:
9. Similarly run Index job for Party Writing number:
10. Go to Batch Viewer → Party Writing Number → Initially Index Smart Search data 11. Select Initial Index data for Search data Party Writing Number
Page 24 of 27
12. To execute this job click on “Execute Batch”
13. When executing, the status the current status will displays as “Processing”. Keep refreshing for
current status by clicking on “Refresh Status”
14. When successfully finished, it will show below details:
15. This completes the Smart Search configuration. After enabling smart search, login to UI and test
smart search.
6. TURNING OFF SMART SEARCH
The following sections list Steps to turn off Smart Search if needed.
1. On the MDM Application server, navigate to the MDM Installation directory,
D:\infamdm\hub\cleanse\resources
Page 25 of 27
2. Open this cmxcleanse.properties file and make sure that the below properties is set as:
cmx.ss.enabled=false
3. On the MDM Application server, navigate to the MDM Installation directory,
D:\infamdm\hub\server\resources
Page 26 of 27
4. Open this cmxserver.properties file and make sure that the below properties is set as:
cmx.ss.enabled=false
5. Disable ZooKeeper in MDM Hub :
Page 27 of 27
6. Restart WebSphere which will disable Smart Search.
top related