installing and maintaining clusters of freebsd servers using pxe and rsync cor bosman xs4all...
TRANSCRIPT
![Page 1: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/1.jpg)
Installing and maintaining clusters of FreeBSD servers
using PXE and Rsync
Cor Bosman
XS4ALL
![Page 2: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/2.jpg)
![Page 3: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/3.jpg)
![Page 4: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/4.jpg)
The problem
Operating dozens of servers individually
• Installation
• Maintenance
• Security
• Upgrades
• Number of servers increased rapidly. A solution is necessary
![Page 5: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/5.jpg)
Fixing the problem
• Standardize hardware
• Create fast and easy installation
• Centralize maintenance to keep groups of servers identical
![Page 6: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/6.jpg)
Standardize hardware
• Choose a few different server layouts. Low, medium, high performance. We prefer on-board devices so we can use 1U rack mounted servers
• Spare policy is simple. Keep one or more spare boxes. Swap complete box when hardware fails
• You always know what to expect. Hardware will be supported by your OS
![Page 7: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/7.jpg)
installation
Different possibilities• Copy an image using tools like dd. Slow,
can’t easily be done in parallel. Problem with drive sizes
• Scripted install using floppy/CD• PXE
• Preboot Execution Environment• Standard created by Intel• Built on TCP/IP, DHCP, TFTP
![Page 8: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/8.jpg)
PXE‘normal’ DHCP protocol
Discover boot server
DHCP
Fetch bootstrap codeTFTP
NFS
Fetch additional bootloadersKernel, config files
CLIENT SERVER
![Page 9: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/9.jpg)
Installing FreeBSD using PXE• PC with PXE capable network card• Ethernet connection• DHCP/Boot server• TFTP server
– Boot image
• NFS server– Boot loader files– Loader config file– Kernel– Memory file system
• Install.cfg
• FreeBSD distribution
![Page 10: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/10.jpg)
server-name "DHCPserver";default-lease-time 86400;option subnet-mask 255.255.255.0;option broadcast-address 192.168.1.255;option domain-name "xs4all.nl";option domain-nameservers 194.109.6.66,194.109.9.99;option routers 192.168.1.2;subnet 192.168.1.0 netmask 255.255.255.0 { range dynamic-bootp 192.168.1.10 192.168.1.254; filename "pxeboot"; next-server 192.168.1.3; option root-path "/usr/local/export/pxe";}
![Page 11: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/11.jpg)
Installing FreeBSD using PXE• PC with PXE capable network card• Ethernet connection• DHCP/Boot server• TFTP server
– Boot image
• NFS server– Boot loader files– Loader config file– Kernel– Memory file system
• Install.cfg
• FreeBSD distribution
![Page 12: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/12.jpg)
echo Loading Kernel...load /kernelset choice=defaultechoecho Please select one of the following installs within 15 secondsechoecho defaultecho scsiecho dhechoread -t 15 -p "Type in the exact word of your selection: " choiceechoinclude /boot/loader.rc.$choiceecho booting...set vfs.root.mountfrom="ufs:/dev/md0c"boot
load -t mfs_root /mfsroot-default /boot/loader.rc.default
/boot/loader.rc
![Page 13: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/13.jpg)
Tips
• Use an install server. Insert empty HD
• Keep a stack of installed harddisks ready
• You can install multiple servers at the same time
![Page 14: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/14.jpg)
Centralized maintenance• Scaling choices have created clusters of
servers• Every cluster has a parent (“golden master”)
– Doesn’t do any production work. Merely a repository
– Low end hardware– Parent is called the ‘zero’ server. Production
servers are named after their task. smtp0, smtp1, smtp2….
• OS has a parent
![Page 15: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/15.jpg)
![Page 16: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/16.jpg)
Testing• Testing on production server is a bad idea• Testing on parent is also a bad idea• Use specific test and accept servers
parent0
test1
test0
child3child2child1
![Page 17: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/17.jpg)
Synchronizing servers
• Mirror parent over the network– Exclude files
• Machine specific config files• Temporary files/directories• Customer data
– Preview changes
Possible tools: rdist, rsync, …
• Manual syncing prone to human error
![Page 18: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/18.jpg)
$ sync Usage: sync -t -s <host> -f <conf file> -f alternate file. Default is sync.conf-t test, don't actually do it. Recommended before ALL syncs-s hostname to sync. Use "all" for all sites.-c allow recursive updates
Config file
$ cat sync.conf# hostname exclude_file recursive?#smtp0.xs4all.nl exclude.smtp yeswebmail0.xs4all.nl exclude.webmail yesdh0.xs4all.nl exclude.dh yesbackup.xs4all.nl exclude.backup no
![Page 19: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/19.jpg)
Security• Security and ease of use often go hand
in hand– Define what is important to you– We try to avoid tasks that need interaction
per individual server.
• Protect your parent servers– Harden servers– Firewall– IDS
![Page 20: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/20.jpg)
Security• Connecting to children
– Use ssh key agent for authentication
• Use sync to check for changes
• Quick reaction possible
![Page 21: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/21.jpg)
Enhanced PXE installation
• Installation is adding a server to a group• Choices possible through loader.rc• DHCP server drives the PXE installation
VLANS• VLAN allows subnetting of physical network• Divide switch into separate networks • DHCP server can reply differently for each
network
![Page 22: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/22.jpg)
server-name "DHCPserver";default-lease-time 86400;option subnet-mask 255.255.255.0;option broadcast-address 192.168.1.255;option domain-name "xs4all.nl";shared-network "VLAN0" {
subnet 192.168.1.0 netmask 255.255.255.240 { range dynamic-bootp 192.168.1.4 192.168.1.14; filename "pxeboot"; next-server 192.168.1.1; option root-path "/usr/local/export/pxe-freebsd-45-scsi"; option domain-name-servers 192.168.1.1; option routers 192.168.1.1;
} } shared-network "VLAN1" {
subnet 192.168.1.16 netmask 255.255.255.240 { range dynamic-bootp 192.168.1.20 192.168.1.30; filename "pxeboot"; next-server 192.168.1.17; option root-path "/usr/local/export/pxe-freebsd-45-ide"; option domain-name-servers 192.168.1.17; option routers 192.168.1.17;
} }
![Page 23: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/23.jpg)
Immediate Synchronization
• Create FreeBSD package for each type of server
• Install package from install.cfg• “post” script runs sync with correct
parent• End result is fully installed and
configured server, ready for production. • Total install time less than 5 minutes
![Page 24: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/24.jpg)
PXE ServerDHCPTFTPNFSNAT
Cisco Catalyst 2900XL switch
192.168.1
Parent
![Page 25: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/25.jpg)
![Page 26: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/26.jpg)
![Page 27: Installing and maintaining clusters of FreeBSD servers using PXE and Rsync Cor Bosman XS4ALL cor@xs4all.net](https://reader034.vdocument.in/reader034/viewer/2022051401/56649de55503460f94add909/html5/thumbnails/27.jpg)
Questions ?
Slides, paper, scripts and example files: http://www.xs4all.nl/~scorpio/sane2002
Email me at [email protected]