BES III Computing at The University of

Minnesota

Dr. Alexander Scott

6/13/2008 BES III software at MN 2

Acknowledgements Cheng Ping Shen (Hawaii)

– Traveled to Minnesota to help set up BESIII environment

Alexey Zhemchugov (Dubna)– Created pacman installation of BOSS,

advised on setup Pete Zweber, Nick Howell (Minnesota)

– Extensive testing of local installation

6/13/2008 BES III software at MN 3

Talk Outline Release installation and testing Monte Carlo farm implementation Comparison of IHEP and MN Monte

Carlo Analysis support Future issues

6/13/2008 BES III software at MN 4

Installing Releases We use Alexey’s pacman packaging of the

BES environment– Instructions for install located at Alexey’s site– We installed both 6.3.1 and 6.3.3 successfully

release 6.3.1 had some version drift had to manually update packages

– Download and installation takes ~5 hours– Directory structure fully built by package

Execute shell scripts to set environment– Release ready for use

6/13/2008 BES III software at MN 5

Testing Releases

We verified that Alexey install alone sufficient to generate MC – Started over with install from scratch on a

workstation isolated from network– From workstation, successfully:

installed release generated Monte Carlo ran sample analysis package

We are developing documentation on doing remote installs

6/13/2008 BES III software at MN 6

Monte CarloExample (6.3.1)

Generated 1M generic J/ψ decays

Used getacAlg analysis package to reconstruct J/ψ→(ηc)

Selected candidates based on good tracks, showers

Compared variable distributions between IHEP and MN

Thanks to Cheng Ping

6/13/2008 BES III software at MN 7

Example 6.3.3 Generated 1M J/ψ

decays Used ρ analysis

package to reconstruct J/ψ→ ρ

Superimposed MN variable distributions on IHEP

Great agreement

Thanks to Nick Howell

6/13/2008 BES III software at MN 8

Monte Carlo Farm Used the

CLEO-c farm implementation for BES III generation

Only cosmetic changes required so far– Both have a “physics” and

“reconstruction” step– Easily modifiable job structure– Generic distributed job infrastructure

6/13/2008 BES III software at MN 9

Physics Generation File

#include "$ENV{KKMCROOT}/share/jobOptions_KKMC.txt"

KKMC.CMSEnergy = 3.770;

KKMC.CMSEnergySpread=0.0014;

KKMC.InitializedSeed={400081,1,0};

KKMC.NumberOfEventPrinted=5333;

KKMC.GeneratePsi3770=true;

EvtDecay.userDecayTableName = “psipp.dec";

#include "$ENV{BESEVTGENROOT}/share/BesEvtGen.txt"

BesRndmGenSvc.RndmSeed = 297181;

#include "$ENV{BESSIMROOT}/share/G4Svc_BesSim.txt"

G4Svc.RunID = 217804;

#include "$ENV{ROOTIOROOT}/share/jobOptions_Digi2Root.txt"

RootCnvSvc.digiRootOutputFile = Test_633_BES3_PsiPP_1_217804_0.rtraw";

MessageSvc.OutputLevel = 5;

ApplicationMgr.EvtMax = 5333;

6/13/2008 BES III software at MN 10

FarmInfrastructure

3 servers + 40 worker nodes– hyper-threaded dual Xeon 2.66 GHz processors– handle 4 jobs apiece

2 dual quad-core machines– 2.33 GHz processors – 3x as powerful as other nodes– handle 8 jobs apiece

Gigabit connections to all machines 10 TB of storage space Condor handles job distribution

6/13/2008 BES III software at MN 11

Monte Carlo Samples

MN farm samples generated using releases 6.3.1 and 6.3.3 – Statistics

ψ’: 3.5 s, 50 kB (5M/day) ψ’’: 3.5 s, 50 kB (5M/day) J/ψ: 2.7 s, 30 kB (6.5M/day)

Monte Carlo generated – Generic samples:

2M J/ψ, 4.5M ψ’, 1.5M ψ’’

– IHEP comparison samples 1M each of J/ψ, ψ’, and ψ’’

6/13/2008 BES III software at MN 12

MN vs. IHEP Comparison

Attempt to generate identical Monte Carlo to IHEP sample– Use the same scripts, random seeds, and

parameter files as IHEP– Generate 1M each of J/ψ, ψ’, ψ’’– Comparisons made at the track/shower level– Distributions match extremely well

IHEP believes discrepancies due to multiple random number engines

6/13/2008 BES III software at MN 13

Sample Distributions

6/13/2008 BES III software at MN 14

Analysis Support Documentation on analysis support for

users Users can create analysis areas and

compile packages– Set up environment and own workarea– Use cmt to create packages within workarea– Check out TestRelease package to run your own

packages Developing our own MC testing package

6/13/2008 BES III software at MN 15

Issues for the Future

6/13/2008 BES III software at MN 16

Network Transfers Transfer speeds differ between IHEP→MN

and MN→IHEP– 30 vs. 300 Mbps using iperf w/ 5 MB window

Require TCP/IP windows to be opened or transfer limited by latency– Investigating dynamic window sizing for scp

operations– Would like better transfer method

6/13/2008 BES III software at MN 17

Actual Run Simulations

Some questions remain before simulations of data– Based on CLEO-c, some components missing

Random triggers? Constants? Run by run info?

– Triggers and runinfo can be handled by existing infrastructure

– Manual installation requires constants database setup

Not a part of pacman installation Already present or not yet integrated?