system management: node configuration & software package management
DESCRIPTION
System Management: Node Configuration & Software Package Management. [email protected]. System management Architecture. The pillars: Central Configuration Database Node Configuration Management Base system installation Software Package Management Monitoring. - PowerPoint PPT PresentationTRANSCRIPT
German Cancio – WP4 developments
PartnerLogo
System Management:Node Configuration &
Software Package Management
German Cancio – WP4 developments - n° 2
System management Architecture
The pillars: Central Configuration Database
Node Configuration Management
Base system installation
Software Package Management
Monitoring
German Cancio – WP4 developments - n° 3
Node Configuration Management
German Cancio – WP4 developments - n° 4
Client Server
XML
HLDL
PAN
DBMNotification+ Transfer
Low Level API
High Level API
ComponentAccess API
GUINode Configuration Management
German Cancio – WP4 developments - n° 5
Node Configuration Management (NCM)
Client software running on the node which takes care of “implementing” what is in the configuration profile
Sits on top of the low-level Config access library (NVA-API)
Modules: “Components”
Component support libraries
Framework
German Cancio – WP4 developments - n° 6
NCM: Components
“Components” (like SUE “features” or LCFG ‘objects’) are responsible for updating local config files, and notifying services if needed
Components register their interest in configuration entries or subtrees, and get invoked in case of changes
Components do only configure the system Usually, this implies regenerating and/or updating local config files (eg.
/etc/sshd_config)
Use standard system facilities (SysV scripts) for managing services Reuse the standard facilities: most services come with a SysV script. Components can notify services using SysV scripts when their configuration
changes. Components are kept small&simple
German Cancio – WP4 developments - n° 7
Component examplesub Configure {
my ($self) = @_;
# access configuration information
my $config=NVA::Config->new();
my $arch=$config->getValue('/system/architecture’); # low-level API
$self->Fail (“not supported") unless ($arch eq ‘i386’);
# (re)generate and/or update local config file(s)
open (myconfig,’/etc/myconfig’); …
# notify affected (SysV) services if required
if ($changed) {
system(‘/sbin/service myservice reload’); …
}
}
German Cancio – WP4 developments - n° 8
Existing component taxonomy Components can be classified into basic, service-specific and Grid components
Basic components: Manage basic system configurations eg. network,NFS, printers, security, time…
Service specific components: For batch nodes, servers (provided by service managers)
Eg. LSF, PBS, Castor, accounting, root mail, …
Grid components (provided by Grid middleware WP’s) Eg. Globus, GDMP, LCAS, Resource Broker…
Existing components are available both from SUE and LCFG SUE features: Basic and Service specific LCFG components: Basic and Grid components
Need complete classification (which components to port, which ones to rewrite) Functionality -> component
German Cancio – WP4 developments - n° 9
NCM: Component support libraries
Component support libraries for recurring tasks:
Logging and error reporting
Template processor (for fast config file generation)
SUE sysmgt libraries Eg. check/edit files, system information, regexps, crontab,
(x)inetd…
Monitoring integration
German Cancio – WP4 developments - n° 10
NCM: framework
A light weight framework (cdisp) glues the components to the Configuration Client.
Overall functionality:
1. Register which components are interested in which configuration entries/subtrees
2. Upon a config update, look up the components which need to be invoked
3. Ordering of component invocations according to dependencies
4. Invocation of components
German Cancio – WP4 developments - n° 11
NCM Architecture
cdisp
server
client
NVA API
DBM Cache
Invocation
registration ¬ification
CCConfig
ComponentsComponents
Components
XML
Component libsSUE sysmgtLoggingTemplate processorMonitoring interface
Configure()
Node Config Client
Node Config Mgmt
“Low-level” API
“High-level” API
German Cancio – WP4 developments - n° 12
Software Package Management
German Cancio – WP4 developments - n° 13
Software Distribution Architecture
A packaging tool takes care of installing/deinstalling/upgrading packages on a computer node and keeps an inventory of currently installed packages.
The packages themselves are stored on a managed Software Repository accessible via multiple protocols (eg. HTTP, FTP, shared file system,..)
Information on which packages are to be deployed on which nodes, and which packages are available on which repositories, is kept in the fabric-wide Configuration Database.
A 'glue' application (running on the target nodes) computes the necessary package upgrade operations and invokes the packaging tool.
(The SW generation and packaging process is outside the scope.)
German Cancio – WP4 developments - n° 14
Software Package Manager (SPM)
The SPM is the glue application. Functionality:
1. Compares the packages currently installed on the local node with the packages listed in the configuration
2. Computes the necessary install/deinstall/upgrade operations
3. Invokes the packager with the right operation transaction set
The SPM is driven via a local configuration file For batch/servers: A component generates/maintains this cf file out of CDB
information For desktops: Possible to write a GUI for locally editing the cf file
The SPM core is independent of which packaging format is used.
German Cancio – WP4 developments - n° 15
Software Package Manager (II)
Packager: the standard system packaging format is used (rpm for Linux, pkg for Solaris) rpmt (for rpm ‘transactional’) used for transactions handling
• Packager (rpmt) functionality:
1. Read operations (transaction)
2. downloads new packages from Repository
3. orders the transaction operations taking into account dependency information
4. Executes the operations (installs/removes/upgrades)
German Cancio – WP4 developments - n° 16
SPM Architecture
Component
rpmt
Repository
packages
GUI
SPM
Packages (RPM, pkg)
RPM db
Local Configfile
Transaction set
filesystem
Package files
HTTP(S), NFS, FTP
Central Config DB
Installed pkgs
“desired”configuration
Desktops
external
SPM
German Cancio – WP4 developments - n° 17
Manpower
German Cancio – WP4 developments - n° 18
Module Work for LCG1 Estim. Full work Estim. cdisp Full functionality 4 PW - 4 PW components Import+adapt from SUE, LCFG
(no Grid comp.) 10 PW Rewrite +
Grid comp. 16 PW
Component libs Import from SUE, LCFG whenever possible
9 PW Rewrite 17 PW
Total: 23 PW 37 PW Module Work for LCG1 Estim. Full work Estim. SW repository Basic functionality 3 PW Full funct. 6 PW rpmt Simple packages - All pkgs 5 PW SPM Basic functionality 5 PW Full funct. 10 PW SPM component Full functionality 1 PW - 1 PW Total: 9 PW 21 PW
• Grand total: 32 PW (=7.4 PM) for LCG1, 58 PW (13.5 PM) for full solution
• Required manpower type: System+perl programming/development expert
• Available WP4-install development manpower (from R2.0 on): 0.3 (Enrico) + 0.15 (German)• Edinburgh?
Manpower requirements
SPM
NCM