experiences & directions of shift/core at cern october 13, 1994
Post on 06-Feb-2016
36 Views
Preview:
DESCRIPTION
TRANSCRIPT
F. Hemmer
CERN
HepIx Fall 94
Experiences & directions
of SHIFT/COREat CERN
October 13, 1994
F. Hemmer
CERN
HepIx Fall 94
SHIFT
Designed in 1990
• fast access to large amounts of data
• good tape support
• cheap & easy to expand
• vendor independent
First implementation in operation in Jan 1991
F. Hemmer
CERN
HepIx Fall 94
SHIFT Model
CPUServerCPU
ServerCPUServerCPU
ServerCPUServerCPU
ServerDisk
ServerDiskServer
TapeServerTape
ServerTapeServer
High Speed network
F. Hemmer
CERN
HepIx Fall 94
CORE
• Centrally Operated RISC environment
• Grouped CSF, SHIFT, HOPE,PIAF, TAPES, Network, Racks ...
• CERNVM vs. CORE
F. Hemmer
CERN
HepIx Fall 94
1993-1994 Strategy
• Decision to run down the CRAY X/MP
• Decision to run down the IBM 9000
Migrate batch by end of 94
Migrate interactive by end of 95
• Decision to acquire a 64 node SP2
F. Hemmer
CERN
HepIx Fall 94
SGI 340 SGI 340SGI 340 SGI 340
SUN
4/330
SUN
4/330
STK 4280
STK 4280
EXABYTE
UltraNet
EXABYTE
JUKEBOX
cisco AGS+IP router
CHEOPS
~250 GBytesSCSI disk
10 GBytesSCSI disk
25 H-P 9000-720s
H-P Apollo DN10040
HOPE
CSF
shift
Simulation
Facilities
Data AnalysisFacility
CrayX-MP/48
cassette tapesand robot
ethernet
CERN site LAN
FDDI
SUN
4/630
STK 4280
STK 4280
STK 4280
STK 4280
CrimsonCrimson
1992
F. Hemmer
CERN
HepIx Fall 94
CORE in 1994
Put les core picture
F. Hemmer
CERN
HepIx Fall 94
CORE in 1994• CPU server
HP 9000-735/755
SGI Challenge
SGI Power Series
DEC 3000/400,500
IBM RS/6000-370
• Disk servers
SGI Power series
SGI Crimson
DEC 3000-600
IBM RS/6000 970
SUN Sparc 10
• Tape servers
IBM RS/6000-370
Sun 4/330
• Monitors,consoles,YP,...
Sun SPARCstations
TOTAL
# Systems
43
4
1
11
3
1
2
3
1
1
8
3
8
90
# CPUs
43
44
4
11
10
4
2
3
1
1
8
3
8
143
F. Hemmer
CERN
HepIx Fall 94
CORE operation
issues• Reliability
• Manpower/machine decreases
• Automation
• Monitoring
• "Standard" tools
• Centralize management
F. Hemmer
CERN
HepIx Fall 94
CORE 1994
Here MTBI's per manuf
F. Hemmer
CERN
HepIx Fall 94
CORE 1995
• CORE is the infrastructure
operation, network, racks, cables, ...
• Public services
CSF,PIAF,SP2,CS/2
• Public staging pool
• Tape servers
F. Hemmer
CERN
HepIx Fall 94
SHIFT CPU 1992
IBM
9000
CSF Cray SHIFT VAX
9000
0
200000
400000
600000
800000
1000000
F. Hemmer
CERN
HepIx Fall 94
CORE CPU 1992-94
CERNVM VXCERN CSF SHIFT0
1000000
2000000
3000000
4000000
5000000
F. Hemmer
CERN
HepIx Fall 94
SHIFT 1995
here LEP Z0 + Req for 95 + NA 48/49 + LHC
Plus data flows LEP LHC
F. Hemmer
CERN
HepIx Fall 94
DISKS 07/94Vendor Model Formatted capacity Number Total size (MB)
IBM OEM 0664CSH IBM OEM 0664M1H 1920 12 23040IBM OEM 0664N1D 1920 228 437760SGI 0664N1D 1920 1 1920MICROP 2112 1001 1 1001HP C2247 1001 10 10010DEC DSP5350 3406 31 105586DEC RZ26-VA 1075 10 10750DEC RZ28-VA 2150 6 12900DEC RZ74 3406 63 214578SEAGATE ST12400 2048 52 106496SEAGATE ST2383N 317 2 634SEAGATE ST41200 990 44 43560SEAGATE ST41600 1331 1 1331SEAGATE ST41650 1351 162 218862SEAGATE ST41651 1350 6 8100SEAGATE ST42100 1812 12 21744SEAGATE ST43400 2778 61 169458SEAGATE ST4766N 669 1 669 Total 704 1392239
F. Hemmer
CERN
HepIx Fall 94
Tape mounts week 1-43
1992
4% 4%
75%
17% SHIFT/CRAY
SHIFT/SUN
VM
Others
F. Hemmer
CERN
HepIx Fall 94
TAPES in 1994
46%54%
SHIFT
CERNVM
F. Hemmer
CERN
HepIx Fall 94
Tapes in 1994
1 7
13
19
25
31
370
50001000015000200002500030000
SHIFT
CERNVM
F. Hemmer
CERN
HepIx Fall 94
TAPES 1995
• NTP robots
• 3494 & central data recording
• DLT robot
• Any "reasonable" media support
F. Hemmer
CERN
HepIx Fall 94
Network
• UltraNet still the most used
• FDDI used more and more
• 1995 : HIPPi, FCS ...
• Spectrum
F. Hemmer
CERN
HepIx Fall 94
UltraNet Bandwith
Oct 92
MB/S
293 294 295 296 297 298 299 300 301 DAY No
0
0.5
1
1.5
2
2.5
3
3.5
F. Hemmer
CERN
HepIx Fall 94
Network
Jacek Foils
F. Hemmer
CERN
HepIx Fall 94
UltraNet SoftwareUser application Software
Socket Compatibility Library
NFS Sockets
UDP TCPIP
Data LinkDriver
EtherNet UltraNetFDDI
UltraNet Driver
TransportNetwork
Data LinkPhysical Link
Hardware AssistedProtocol Engine
F. Hemmer
CERN
HepIx Fall 94
SHIFT Software
• Unix Tape subsystem
multi-user, labels, multi-file, operation
• Fast Remote File Access system
• Remote Tape Copy System
• Disk Pool manager
• Tape Stager
• Clustered NQS Batch System
• Integration with standard I/O packages
Fatmen, Zebra, EPIO
• Network Operation
• Monitoring
F. Hemmer
CERN
HepIx Fall 94
Remote Tape Copy System
• select a suitable tape server
• initiates the tape-disk copy
tpread -v I29127 -g SMCF -q 4,6 pathname
tpread -v CUT322 ‘sfget -p opaldst filename‘
F. Hemmer
CERN
HepIx Fall 94
RFIO
High performance, reliability (improve on NFS)
• C I/O compatibility
Fortran subroutine interface
• rfio daemon started by open on remote machine
• optimized for specific networks
• asynchronous operation (read ahead)
• optional vector preseek
F. Hemmer
CERN
HepIx Fall 94
SW
show what has changed since 1992AFS
F. Hemmer
CERN
HepIx Fall 94
Stager
• up to now : dpm + tpread
• lack of robustness faced to system errors
• concurrency not handled
• staging space not handled
F. Hemmer
CERN
HepIx Fall 94
Stager architecture
CPUServer
TapeServerDisk
Server
CPUServer
CPUServer
DiskServer
stgcat
TapeServer
TapeServer
F. Hemmer
CERN
HepIx Fall 94
Stager functions
• process stage requests
• manage space
- By default purge oldest file when space needed
- if (size < 1024) w=max(atime, mtime)
else w=max(atime,mtime)-(86400*log(size/1024))
• a different algorithm may be provided
F. Hemmer
CERN
HepIx Fall 94
Stager commands
• stagein
• stageout/stageput
• stagewrt
• stageclr
• stageqry
F. Hemmer
CERN
HepIx Fall 94
Stager future extensions
• Staging of disk files
• Transfer of files already staged on CERNVM
• Access control lists
• Prioritization of requests
• Tape to Tape copy
• Automatic file migration to robots
F. Hemmer
CERN
HepIx Fall 94
SHIFT SW Futures
• SHIFT library in CERNLIB
• RFIO review for high speed protocols
• RFIO wide area
• RFIO "gateway" for SP/2
• RFIO per platform tuning
• VMS final port
• RFIO Checkpoint/Restart
• Ports to Irix 6, AIX 4, HP/UX 10
top related