1 cms user jobs submission with the usage of asap natalia ilina 16/04/2007, itep, moscow

1

CMS user jobs submission

with the usage of ASAP

Natalia Ilina

16/04/2007, ITEP, Moscow

2

What’s ASAP?

ASAP: Arda Support for CMS Analysis Processing

ASAP is a job creation and submission framework for CMS Analysis jobs:

command line tool to create and control jobs

optional server part which can assume responsibility for the users jobs resubmitting in case of failure

3

What’s ASAP?

2 types of jobs can be submitted by ASAP:

standard user job - data are registered

in CMS DBS/DLS system

private user job - any job

(the limitation is that ASAP uses scramv1 environment)

4

STEP I

Starting to work with ASAP

5


create the tracking directory which is opened for r/w for everybody:

setup enviroment:

create proxy:

> mkdir user_path/ASAP> mkdir user_path/ASAP > chmod a+w user_path/ASAP> chmod a+w user_path/ASAP

> > voms-proxy-init -voms cms -valid 192:0voms-proxy-init -voms cms -valid 192:0

> > sourcesource /afs/cern.ch/sw/arda/install/CMS/asap3/setup.{sh,csh}/afs/cern.ch/sw/arda/install/CMS/asap3/setup.{sh,csh}

6


create CMSSW project area

Usage of default library:

Changed library:

> scramv1 project CMSSW CMSSW_1_2_0> scramv1 project CMSSW CMSSW_1_2_0> cd CMSSW_1_2_0/src> cd CMSSW_1_2_0/src> project CMSSW> project CMSSW> cvs co -r CMSSW_1_2_0 JetMETCorrections/JetVertexAssociation> cvs co -r CMSSW_1_2_0 JetMETCorrections/JetVertexAssociation> cd JetVertexAssociation> cd JetVertexAssociation

Edit *.h, *.ccEdit *.h, *.cc

> scramv1 b> scramv1 b> eval `scramv1 runtime -csh`> eval `scramv1 runtime -csh`

> scramv1 project CMSSW CMSSW_1_2_0> scramv1 project CMSSW CMSSW_1_2_0> cd CMSSW_1_2_0/src> cd CMSSW_1_2_0/src> eval `scramv1 runtime -csh`> eval `scramv1 runtime -csh`

7

STEP II

Configure Job

8

Configure Job: STANDARD JOB

Requires 2 files:

1. CMSSW .cfg file

2. Configuration file for ASAP (task.conf)

9

Example of task.conf

# specify directory to store tasksjobdir = /afs/cern.ch/user/n/nilina/scratch0/ASAP_tracking

# Store output at SEstore_output = 1

# store output directly to castor (the dir should already exist and have # the correct permitions: rfchmod 777 /path/output)output_se = srm.cern.choutput_se_path = /castor/cern.ch/user/n/nilina/asap

# bad_se=fnal.gov

# specify grid to submit to (lcg, my)grid = lcg

# or specify dataset pathprimary_dataset = mc-onsel-120_qqH135_2tautier = FEVTprocessed_dataset = CMSSW_1_2_0-FEVT-1166726234

10

Example of task.conf (continue)

# pset file (CMSSW configuration file)pset_file = JetVertex.cfg

# specify output sandbox, could be filename, # a comma sep list of filename or *output_files = jetvertex.root

# total number of events you wish to process # (if unspecified assume all)events_required = 20

# events per jobevents_per_job = 5

# specify minimum time requirements for the jobmin_wall_clock_time = 100min_cpu_time = 100

11

Configure Job: PRIVATE JOB

Requires 4 files:

CMSSW .cfg file

Configuration file for ASAP (task.conf)

dataset.conf - dataset file which tells ASAP which files to use for which jobs and with which parameters

template.sh – script that has to be run at working node

12

template.sh

#!/bin/sh

lcg-cp --vo cms lfn:/my_lfn file:///`pwd`/my_lfn

cmsRun --parameter-set `pwd`/test_analysis.cfg

Instead of cmsRun one can use any other executable

13

dataset.conf

input_1.root 0 100 3.5 32 input_2.root 100 200 4.5 20input_3.root 0 100 3.5 11

FORMAT: INPUT_FILE SKIP_EVENTS MAX_EVENTS ATTRIBUTE_1 … ATTRIBUTE_N

Example N1:

Example N2 (if the file is stored in a SE):

Example N3 (no input data):

srm://srm.cern.ch/castor/cern.ch/user/u/user/input.root 0 100 srm://srm.cern.ch/castor/cern.ch/user/u/user/input.root 100 100

- 0 100 345 9329- 100 100 1345 93290- 200 100 2345 13290- 300 100 1111 345

14

Changes in CMSSW .cfg file

# Now it should be:

source = PoolSource { untracked vstring fileNames = {${FILENAMES}} }

service = RandomNumberGeneratorService {

untracked uint32 sourceSeed = ATTRBUTE_1 PSet moduleSeeds = { untracked uint32 VtxSmeared = ATTRBUTE_2 }}

15

task.conf for private job

# specify directory to store tasksjobdir = /afs/cern.ch/user/n/nilina/scratch0/ASAP_tracking

# Store output at SEstore_output = 1

# specify grid to submit to (lcg, my)grid = lcg

# specify dataset filedataset_file = dataset.conf

# pset file (CMSSW configuration file)pset_file = test_analysis.cfg

16

task.conf for private job (continue)

# template file (shell script to run at WN)application_wrapper = template.sh

# specify output sandbox, could be filename, a comma sep list of filename or *output_files = *.root

# specify minimum time requirements for the jobmin_wall_clock_time = 100min_cpu_time = 100

17

STEP III

Job submition

18

JOB SUBMITION

Create jobs:

ASAP will package your executable for transfer to the Grid,

check DBS/DLS if it is required,

provide you with a TASK ID which you should use for all future operations

You’ll see on the screen:

> > asap --create asap --create task.conftask.conf

* starting asap3 (0_7_13), fasten your seatbelts* split 4 jobs with 20 events* using SCRAM environment* application: CMSSW_1_2_0* use taskid 95312 for further tracking

19

JOB SUBMITION

Match jobs:

Checking if there are resources available to satisfy your jobs requirements

Submit manually:

> > asap --taskid TASKID --submitasap --taskid TASKID --submit

> > asap --taskid TASKID --matchasap --taskid TASKID --match

20

JOB SUBMITION

Monitor status:

You’ll see on the screen:

> asap –taskid TASKID –update --list> asap –taskid TASKID –update --list

Job ASAP Status GRID Status GRID Reason----------------------------------------------------------------------------------------------- 1 SUBMITTED Scheduled Job successfully submitted to Globus 2 SUBMITTED Scheduled Job successfully submitted to Globus 3 SUBMITTED Scheduled Job successfully submitted to Globus 4 SUBMITTED Scheduled Job successfully submitted to Globus-----------------------------------------------------------------------------------------------

21

JOB SUBMITION

Fetch output from GRID:

If you have not specified an output directory the output will be stored in jobdir/TASKID/output (jobdir was specified in your original config file)

> > asap --taskid TASKID --fetchasap --taskid TASKID --fetch

22

Using monitor Delegate your proxy:

Register Jobs with the Monitor:

ASAP will submit and resubmit jobs to GRID upon completion the output. The address of a webpage will be displayed. Before looking at the task one need to insert the GRID certificate

in the Browser. You can track the progress of your jobs by webpage.

> > asap-user-register

> > asap --taskid TASKID --register

23

Using monitor

Monitor status with cmd line:

Unregister Jobs:

Before retrieving the output one need to unregister job

> > asap --taskid TASKID --unregister

> > asap --taskid TASKID --list

24

Other usefull commands

Kill jobs:

Fetch output:

Resubmit jobs:

> > asap --taskid TASKID --killasap --taskid TASKID --kill

> > asap --taskid TASKID --fetch --output_dir OUTPUT_DIR

> > asap --taskid TASKID –fetch-logs

> > asap --taskid TASKID –fetch-data

> > asap --taskid TASKID --submitasap --taskid TASKID --submit

25

Usefull links

ASAP’s User Guide:

http://lxarda13.cern.ch/docs/rn01.html

Contact:

[email protected],,

[email protected],,

[email protected]

26

Practical part of tutorial

2 examples of standard job:

~nilina/scratch0/ASAP_TUTORIAL/ASAP/EXAMPLE1~nilina/scratch0/ASAP_TUTORIAL/ASAP/EXAMPLE1


1 example of private job:


1 cms user jobs submission with the usage of asap natalia ilina 16/04/2007, itep, moscow

Documents

asap slide

job slide

conf slide

job events

file configuration file

asap task

scramv1 project cmssw

conf dataset file