Institut Laue – LangevinIT Priorities

Dec 2013

10th Dec 2013 Jean-François Perrin (ILL) - CERN Openlab IT challenges

Jean-François Perrin (ILL) - CERN Openlab IT challenges

Who are we ?

ILL is an analytical facilityThe most intense continuous neutron flux

38 world class instruments2000 invited scientists /year

480 StaffLocation: EPN-Campus, Grenoble (France)

10th Dec 2013

An international scientificcollaborationFounded in 1971 by France, Germany and

United Kingdom.Scientific partners that have joined in since

then:

Spain, Switzerland, Austria, Denmark, Italy, Czech Republic,Sweden, Hungary, Belgium, Slovakia and India.

Jean-François Perrin (ILL) - CERN Openlab IT challenges10th Dec 2013

Scientists submit

proposals

Proposals are

reviewed by external

experts

<50% get accepted

beam time

ExperimentData archiving

Data analysis

Publication

ScientificWorkflow

Jean-François Perrin (ILL) - CERN Openlab IT challenges10th Dec 2013

Experimental raw data 1973-2012

731-

995

002

004

011

013

021

023

031

033

042

051

053

062

071

073

081

083

091

093

101

103

112

114

122

131

133

0

300

600

900

1200

1500

1800

2100

80

0.4094.85.35.74.6147.316 149.216 14 22 46 66 56

287

144104

32

276283

62

142

405435

494

268

145

732

143

341

487

64

791

307

434

226

346282

120

480

765

2071

606

1200

Gb / Cycle

Gb / cycle Exponential (Gb / cycle) Linear (Gb / cycle)Cycles 1973/1 à 2012/2

Gb

Experimental raw data 1973-2013

731-

995

002

004

011

013

021

023

031

033

042

051

053

062

071

073

081

083

091

093

101

103

112

114

122

131

133

0

5000

10000

15000

20000

25000

30000

35000

40000

800.4094.85.35.74.6147.316 149.216 14 22 46 66 562871441043227628362142405435494268145732

14334148764791307434226346282120480765

2071

6061200

18100

36100

Gb / Cycle

Gb / cycleCycles 1973/1 à 2013/1

Gb

Impacts of the “data deluge”Storage

ILL archive capacity & performanceUsers’ storage becoming almost impossible

Moving dataToday how to carry 40TB?Why carrying them?

AnalysisAlmost impossible in most

users’ home labs.

10th Dec 2013 Jean-François Perrin (ILL) - CERN Openlab IT challenges

Our visionLarge raw data sets should stay and be

archived at the source (ILL in our case)Provide remote analysis infrastructurePreserve data and the scientific workflow

10th Dec 2013 Jean-François Perrin (ILL) - CERN Openlab IT challenges

IT Priority #1: be prepared for PBCost is a major issueChange from scale-up solution to scale outObject storage ? What about legacy

applications ? POSIX semantic?Aim for manageable solution (avoid

multiplicity of low cost solution)

10th Dec 2013 Jean-François Perrin (ILL) - CERN Openlab IT challenges

IT Priority #2: remote analysis infrastructureThe aim is to proposed to users to access workstation or

analysis application remotely using standard web browser (Cloud for data analysis).

Typical workflow:1) The user connects remotely using his web browser and its credentials (preferably FIM).2) Then select one of the experiment he has performed in the list.3) he gets access to a computer where the necessary analysis applications have been installed and configured for direct access to experimental data.4) If necessary he could receive help and support from facility expert, during the analysis.

10th Dec 2013 Jean-François Perrin (ILL) - CERN Openlab IT challenges

BenefitsProvide a user friendly environment (most of or

users are not expert neither in data treatment, neither in IT and have no home IT support).

Accelerate the analysis process, ease collaboration during analysis.

Solve the difficult security problem of letting external users access internal networks.

Solve the problem of transport of experimental raw data.

Move the work from 'software installation' to 'scientific analysis‘.

Authorize the preservation of the full workflow.

10th Dec 2013 Jean-François Perrin (ILL) - CERN Openlab IT challenges

Thanks for your attention

10th Dec 2013 Jean-François Perrin (ILL) - CERN Openlab IT challenges