Tier-1 Evolution and Futures

GridPP 29, OxfordIan Collier

September 27th 2012

Where are we going?

Where are we going?

• Good question

Where are we going?

• Good question

• We know things are changing

Where are we going?

• Good question

• We know things are changing

• We don’t know how things will turn out

Where are we going?

• Good question

• We know things are changing

• We don’t know how things will turn out

• We can make choices

Where are we going?

• Good question

• We know things are changing

• We don’t know how things will turn out

• We can make choices

• We must be ready to try things and to adapt

Where we are

• Services running.– Lots of cores– Lost of disk– Lots of tape

• Castor running well & stably– Shaun will talk about storage future

• EMI/UMD Middleware rolling out– Continues to be a moving target

• Virtualisation– Usage growing fast

• Configuration management– Batch & disk farms much more tractable– Deploying new instances of services easier

2012

• Virtualisation– Heading to most services running on resilient

virtualisation platform with shared storage• Unless there is a definite contra-indication (performance)

– Currently MS Hyper-V• It works. But it is Windows (unfamiliar territory still)• Likely to migrate to open source solution (ovirt/RHEV)

• Cloud– Stratuslab prototype

• Continuing EMI/UMD rollout– Focus on EMI-2

• CVMFS Stratum 0 for non-LHC VOs– Ready for testing by VOs

Fabric - Tier1 Review, June 2012

Virtualised hosts

20/06/2012

2012

• Combination of Hyper-V and SCD cloud – Deploying new services easier– Testing things much much easier

• Combined with Quattor (fabric & configuration management) it is pretty easy to roll out, for example, a test cluster

Getting nebulous

• Tier 1 developing SCD Cloud platform– Using StratusLab – integrating cloud & grid– Very much a prototype so far– Several dozen VMs there as well

• Very good for ‘just trying things’

• Several use cases:– Internal (Tier1 & SCT) development & testbeds

• At present trust users as regular sysadmins– Scientific computing resource for other departments in

STFC– Virtualised layer in batch farm– Experiments are investigating clouds

• We need to be building up experience– EGI Federated cloud

Looking to Future

• Integrating Aquilon– ‘next generation’ configuration database for Quattor– SQL database plus git workflow– Faster more flexible development & deployment– Better relational integrity for the ‘database like’

parts• STFC deploying new DNS management

– We will have direct access to manage our networks including API

– More dynamic provisioning possible

• Entirely more ‘dynamic’ infrastructure?– See cloud work…

Looking to Future

• There are emerging themes

• Virtualisation making hardware provisioning easier– Can abstract hardware delivery/maintenance from

service provision– As site admins we like this

• Cloud frameworks should allow – containerisation

• Users may like this• Within some higher level requirements set by sites users

can have much more choice/control of their software environments

– More elastic & dynamic access to resources• Cloud interfaces may make access to resources

easier for users

Where are we going?

• We know things are changing

• We don’t know how it will turn out

• We can make choices

• Have to be ready to try things and to adapt