ds admin cmds

DataStage Administrator commands to find:

No Command Description Command

1DS Project Object counts SELECT COUNT(*) FROM DS_JOBS;

SELECT COUNT(*) FROM DS_JOBOBJECTS;

2List all jobs in a DS project SELECT JOBNO, @ID, CATEGORY, NAME FROM DS_JOBS

3Clear Log From Administrator select JOBNO from DS_JOBS where NAME='Name of the job'

CLEAR.FILE RT_Logxx where xx is JOBNO.

4CLEAR HASH FILE. CLEAR.FILE <HASHED FILE NAME>

5Find how many Jobs are using given Hashed file as INPUT

select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedInput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;

6Find how many Jobs are using given Hashed file as OUTPU

select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;

7Find how many Jobs are using given Hashed file as INPUT and OUTPUT

select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;

8View column data in a hashed file SELECT COLUMN_NAME FROM HASHED_FILE_NAME

9List metadata of hashfile SELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM

DICT hashedfile ORDER BY "FIELD.NO", "TYPE";

10Insert data into the hashfile INSERT INTO HASHED_FILE_NAME (COLUMN_NAME) VALUES ('VALUE');

11Find which user modified the job last SELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';

12List all the projects in DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;

13Reset surrogate key generated using

Keymgmt function in Datastage UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';

14List all the surrogate key variables

used in datastage proj SELECT * FROM SDKSequences;

15List Datastage error message decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';

16List all jobs using a TABLE

SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;

Using DS CLI: Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.

Steps to set Datastage Environment, to use Datastage CLI

1) Log in to Datastage Server2) set path to the directory where DSEngine exe resides3) execute command : . ./dsenv (on UNIX servers, DSJOB has a runtime dependency on the libvmdsapi.so shared library, dsenv sets required library paths for CLI commands)

1Find total number of connection made from client to Datastage server with the identification of IP address in it.

netstat -a | grep 'dsrpc'

2List all datastage processes executing on the server ps -ef|grep RUN

3Listing projects, jobs, and parameters 1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob

List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs <projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams <projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations <projectname> <jobname>

4Generate XML report of job , its stages and links @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>

5Do a pattern based search for job in the project

@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -matches -sub -oc -oj <projectname> <jobname>

List Sequencer jobs calling a specified job in the project

@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -usedby -r -oc -oj <projectname> <jobname>

List all jobs called in specified Sequencer job

@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r -oc -oj <projectname> <SeqJobname>

6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -wait -jobstatus -param

<name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop <projectname> <SeqJobname>

7Start and stop Datastage Server

1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)./uv -admin -stop ./uvregen ./uv -admin -start

8Unlock locked/hung datastage processes

Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director

9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import <projectname> <DSXfilename> -

overwrite

10Troubleshoot Client connectivity from datastage server unix prompt

1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q














select DS_JOBS.NAME AS JOB_NAME FMT '35L' from DS_JOBOBJECTS, DS_JOBS where DS_JOBOBJECTS.OLETYPE in ('CHashedOutput','CHashedInput') and DS_JOBOBJECTS.OBJIDNO =

DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'Hashed file name' group by JOB_NAME;


9List metadata of hashfile SELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";


11Find which user modified the job last SELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';

12List all the projects in DataStage..

With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;

13Reset surrogate key generated using Keymgmt function in Datastage UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';

14List all the surrogate key variables

used in datastage proj SELECT * FROM SDKSequences;

15List Datastage error message decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';



Using DS CLI: Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.






3Listing projects, jobs, and parameters 1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob

List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs <projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams <projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations <projectname> <jobname>

4Generate XML report of job , its stages and links @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>







6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -wait -jobstatus -param

<name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop <projectname> <SeqJobname>




Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenv

bin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director

9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import <projectname> <DSXfilename> -

overwrite


















9List metadata of hashfileSELECT "FIELD", "FIELD.TYPE", "FIELD.DEF", "FIELD.CONV", "FIELD.HEAD", "FIELD.FMT", "FIELD.ASSOC" FROM DICT hashedfile ORDER BY "FIELD.NO", "TYPE";


11Find which user modified the

job lastSELECT INSTANCE, DTM, MODIFIER FROM DS_AUDIT WHERE CLASS = '2' AND INSTANCE LIKE 'JOB NAME';

12List all the projects in

DataStage.. With Owner SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;

13Reset surrogate key generated using Keymgmt function in Datastage

UPDATE SDKSequences USING DICT VOC SET F1 = '1' WHERE @ID = 'Your_Sequence_Name';

14List all the surrogate key variables used in datastage proj

SELECT * FROM SDKSequences;

15List Datastage error message

decsription, given error me SELECT * FROM SYS.MESSAGE WHERE @ID = '081002';



Using DS CLI:Datastage Command line interface or CLI is a set of commands you can run on the Datastage server machine, @Unix prompt, with a username with sufficient privilaes and after setting datastage environment.






3Listing projects, jobs, and parameters

1)set fully qualified path for dsjob exefor example, DsServe > /etl/Ascential/Datastage/DSEngine/bin/dsjob

List Projects @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lprojectsList Jobs in a project @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -ljobs

<projectname>List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams

<projectname> <jobname>List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations

<projectname> <jobname>

4Generate XML report of job , its stages and links

@DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -report <projectname> <jobname>





List all jobs called in specified @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dssearch -ljobs -uses -r

Sequencer job -oc -oj <projectname> <SeqJobname>

6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>

Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop <projectname> <SeqJobname>




Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data

Stage Director

9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import <projectname> <DSXfilename> -overwrite

































6Run datastage job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -run -mode <NORMAL/RESET/VALIDATE> -

wait -jobstatus -param <name1=val1> -param <name2=val2> <projectname> <SeqJobname>Stop a running job @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -stop

<projectname> <SeqJobname>


1)first check that no one is connected to DataStage (netstat -a | grep dsrpc) 2)User Datastage administrator userid3)

./uv -admin -stop

./uvregen

./uv -admin -start



9Import Datastage jobs @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -import

<projectname> <DSXfilename> -overwrite


1. Login to UNIX host using NIS+ account2. . ./dsenv3. cd $DSHOME4. bin/uv If error "Unable to open unohist" Then file permissions issues exist

5. logto <DataStage project name> If successful Then client connectivity problems are not due to server config.6. q






































Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows:

ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
























12List all the projects in SELECT @ID, NAME FMT '40L' FROM UV.ACCOUNT;

DataStage.. With Owner


















<projectname>

List job parameters @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -lparams <projectname> <jobname>

List job invocation ids @DsServe >> /etl/Ascential/Datastage/DSEngine/bin/dsjob -linvocations <projectname> <jobname>















Data Stage throws error with "Unable to run job", it means there are hung processes are sitting for the particular Job. We need to kill those hung processes and clear the job locks in UV. Follow the steps to resolve this issue: Step 1: 1. Find the processes corresponding to the job name that fails with this error. For instance if the job is MHIS_Extract2, then the command is as follows: ps -ef | grep MHIS Then kill those processes using kill command. Step 2: Clear the job locks in UV. Login to UNIXcd /opt/etl/Ascential/DataStage/DSEngine. ./dsenvbin/uvshlogto uvlist.readu everyLocate the job that appears to be lockedWrite down the associated number from the 4th column (Userno)

unlock user ##### all (where ##### is the Userno from the above step) When running the above command you should see output similar to the following: Clearing Record locks.Clearing GROUP locks.Clearing FILE locks. To exit out of UV type q . Step 3: Clear the Status file and clean up resources of the job from Data Stage Director
































16List all jobs using a TABLE SELECT DS_JOBS.NAME AS JOB_NAME, DS_JOBS.CATEGORY, DS_JOBOBJECTS.NAME AS OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, EVAL

DS_JOBOBJECTS."if index(upcase(@RECORD),'TABLE_NAME',1) > 0 then 'FOUND' else ''" AS FOUND FMT '5L' FROM DS_JOBS, DS_JOBOBJECTS WHERE DS_JOBS.JOBNO = DS_JOBOBJECTS.OBJIDNO and FOUND = 'FOUND' GROUP BY JOB_NAME, DS_JOBS.CATEGORY, OBJECT_NAME, DS_JOBOBJECTS.OLETYPE, FOUND;


































6Find how many Jobs are using select DS_JOBS.NAME AS JOB_NAME FMT '35L' from

given Hashed file as OUTPUDS_JOBOBJECTS,DS_JOBS where DS_JOBOBJECTS.OLETYPE = 'CHashedOutput' and DS_JOBOBJECTS.OBJIDNO = DS_JOBS.JOBNO and EVAL DS_JOBOBJECTS."@RECORD<6>" = 'hashed file name' group by JOB_NAME;
















