progress s o f t w a r e linux and the progress rdbms gus björklund ([email protected]) wizard...
TRANSCRIPT
PROGRESSS O F T W A R E
Linux And The Progress RDBMS
Gus Björklund ([email protected])WizardProgress Software
PUG Challenge 2002, Veldhoven, the Netherlands
PROGRESSS O F T W A R E
Engine Crew
© 2002, Progress Software Corporation3
PROGRESSS O F T W A R EAbstract
The Linux operating system can be the right choice for many Progress RDBMS installations. This talk describes how an Intel-based system can be a highly cost-effective and powerful database server running the Progress RDBMS on RedHat Linux 7.2 on a powerful but inexpensive computer. We will show how to configure the system, what to do, and some things to avoid. Informal benchmark results from the Spring of 2002 are used to illustrate the performance of various configurations.
Time: 90 minutes
© 2002, Progress Software Corporation4
PROGRESSS O F T W A R E
Please ask questionsif I do not explain something clearly
© 2002, Progress Software Corporation5
PROGRESSS O F T W A R E
Spring 2002Linux Benchmark Team
Bravepoint Dan Foreman John Harlow
Progress Gus Björklund
© 2002, Progress Software Corporation6
PROGRESSS O F T W A R EGoals
Prove that Linux is stable and viable in the real world
Prove that price/performance is excellentProduce configuration recommendationsAnd finally…...
© 2002, Progress Software Corporation7
PROGRESSS O F T W A R E
© 2002, Progress Software Corporation8
PROGRESSS O F T W A R EWhy Linux ?
Commodity Priced Hardware Inexpensive Operating System Linux is no longer a toy Currently being used in production (details to
follow) in mission critical applications Excellent alternative to Microsoft Believers in Open Source We don’t drink Microsoft Kool-Aid
© 2002, Progress Software Corporation9
PROGRESSS O F T W A R EWhy Not Linux ?
The constant temptation to apply the “patch of the day”
Credibility (the gap between perception of reality and reality)
Track record is relatively short
© 2002, Progress Software Corporation10
PROGRESSS O F T W A R EWhy Not Linux
It’s not like an OS where you call up your friendly 24 hr/day support line and they stick with fixing your problem; we posted a note to one of the Linux lists during our testing and never got a response; There are two ways to go: the slow, plodding, conservative method the complex, inter-dependent, patch-of-
the-day method
© 2002, Progress Software Corporation11
PROGRESSS O F T W A R EHistory
August 25, 1991: Linus Torvalds posts to comp.os.minix that he has his experimental kernel running gcc and bash. “just a hobby, won’t be big and professional” Version 0.01 sources posted September 1991
1.0 stable kernel release was March 1994 SCO OpenServer binaries of Progress were being
run on Linux as early as 1995 Not supported by PSC
First Progress release for Linux (RedHat 6.2) was Version 8.3C in Spring 2000
© 2002, Progress Software Corporation12
PROGRESSS O F T W A R EExample Production Linux Site
Transplatinum Financial services provider for large trucking
companies Progress V9.1B Linux 2.2.19-6.2.1 DB Size 12GB 137 Users Only problem encountered: “getting the
correct Linux patch combinations installed”
© 2002, Progress Software Corporation13
PROGRESSS O F T W A R E
Benchmark HardwareIn Zero Gravity
© 2002, Progress Software Corporation14
PROGRESSS O F T W A R EBenchmark Server Hardware
Disks - 10 IBM Deskstar IDE (not SCSI) disk drives, 60 gigabytes each
3Ware 7810 Disk Controller with H/W RAID
Memory - 2 gigabytes, PC133, SDRAMCPUs - 2, 1 gigahertz Pentium IIIASUS Motherboard
© 2002, Progress Software Corporation15
PROGRESSS O F T W A R EHardware Cost (August 2001)
Item Price
Server Case $528
Hot Swap Power Supply $338
10 IBM Deskstar 60 GB IDE disks $1890
8 Hot swap cases for disks $200
3Ware 7810 RAID controller $385
Motherboard, RAM, CD, Floppy, Video $891
Shipping $161
Total $4413
© 2002, Progress Software Corporation16
PROGRESSS O F T W A R E
Hardware Cost (1991)Sequent SMP Unix System
Item Price
16 x 20 megahertz 80386 cpu, 64 MB RAM, cabinet
$318,500
176 megabytes additional RAM $195,200
21 Fujitsu Eagle 200 MB ? disks $246,000
Dynix/PTX Operating System $46,000
Installation $8,090
Miscellaneous items $23,500
Total $837,290
© 2002, Progress Software Corporation17
PROGRESSS O F T W A R EThen Versus Now
Result Then: 170 tps $4,925 per tps
Result Now: 580 tps $7.60 per tps
3.4 x more performance1 / 648 the price per tps
© 2002, Progress Software Corporation18
PROGRESSS O F T W A R ESoftware Details
Linux 2.4.18 KernelStock except for the RAID array driversProgress V9.1C09
© 2002, Progress Software Corporation19
PROGRESSS O F T W A R E
Multi-million dollar benchmarking lab in a secret location
© 2002, Progress Software Corporation20
PROGRESSS O F T W A R E
Benchmark LaboratoryUnderground Bunker
© 2002, Progress Software Corporation21
PROGRESSS O F T W A R EBenchmark Team (Partial)
© 2002, Progress Software Corporation22
PROGRESSS O F T W A R EBenchmarks
Lies, True Lies, and BenchmarksATM benchmarkDidn’t compare performance to NT,
W2K, or any other Intel Operating System
© 2002, Progress Software Corporation23
PROGRESSS O F T W A R EATM Benchmark
Simulates ATM withdrawal transaction read, update account read, update branch read, update teller create history
Execute as many times as possible in given time Produces heavy update workload
© 2002, Progress Software Corporation24
PROGRESSS O F T W A R EATM Database
4 tables account: 20,000,000 records branch: 20,000 records teller: 2,000 records history: create 1 per transaction
3.2 gigabyte total size 2 extents, 1,600 megabytes each 2,300 megabytes initial data 175 megabytes indexes
© 2002, Progress Software Corporation25
PROGRESSS O F T W A R ECreating The 3.2 GB Database
prostrct create to allocate space about 2 minutes
Creating the initial records via 4GL 35 min
procopy to make backup 10 minutes, including creating extents
© 2002, Progress Software Corporation26
PROGRESSS O F T W A R EOther Times
Index rebuild (eventually) 9 minutes, 10 seconds proutil atm -C idxbuild all -TB 31 -TM 32 -B 250 -T /bi -t
sort file size: 381,086,720 bytes
dbanalysis 6 minutes
© 2002, Progress Software Corporation27
PROGRESSS O F T W A R EDocumentation
© 2002, Progress Software Corporation28
PROGRESSS O F T W A R EBaseline Run
Database size 3 gigabytes
Everything on one disk8k database block size but otherwise
untuned “out-of-the box installation”
© 2002, Progress Software Corporation29
PROGRESSS O F T W A R EBaseline Run
Database size 3 gigabytes 4 tables
account, branch, teller, historyEverything on one disk8k database block size but otherwise
untuned “out-of-the box installation”Result: 30 tps (4 users)
© 2002, Progress Software Corporation30
PROGRESSS O F T W A R ETop 8 Progress Tuning Items
1. BI on a separate disk2. -spin 50,0003. -B 32000
Later we tried 640004. BI Cluster Size (-bi) 16 mb
© 2002, Progress Software Corporation31
PROGRESSS O F T W A R ETop 8 Progress Tuning Items
5. Page Writers (4)6. BI Writer7. BI Buffers (25+)8. 8K Database Block Size
© 2002, Progress Software Corporation32
PROGRESSS O F T W A R ETop 8 Progress Tuning Items
5. Page Writers (4)6. BI Writer7. BI Buffers (25+)8. 8K Database Block Size
Results before: 30 tps (4 users)Results after: 61 tps (4 users)
© 2002, Progress Software Corporation33
PROGRESSS O F T W A R ELinux Changes
Reiser FS some of the best and worst numbers too erratic so we didn’t continue
EXT2 versus EXT3 no difference observed
noatime no difference observed
Larger shared memory segments (32mb default raised to128 mb) 2% gain
© 2002, Progress Software Corporation34
PROGRESSS O F T W A R EAdditional Progress Changes
-semsets no statistically significant difference
Index rebuild of the test database 3% gain
© 2002, Progress Software Corporation35
PROGRESSS O F T W A R EAdditional Progress Changes
Doubled -B from 32000 to 64000 (8k blocks)
Before: 557 tps (54 users)After: 581 tps (58 users)
© 2002, Progress Software Corporation36
PROGRESSS O F T W A R EAdditional Progress Changes
File System Block Size/DB Block Size rated from best to worst: 4k/8k 4k/4k 1k/8k - not tested
prostrct create took too long 1k/1k- not tested
prostrct create took too long
© 2002, Progress Software Corporation37
PROGRESSS O F T W A R EDisk Array Changes
JBOD 8 disks 479 tps (24 users)
RAID 0 Striped 8 disks, 64k stripe size, bi on stripe set 339 tps (26 users)
Moved BI file to internal (non-RAID) drives 481 tps (32 users)
© 2002, Progress Software Corporation38
PROGRESSS O F T W A R EDisk Array Changes
RAID 0 8 disks; just striping -B 64000, index rebuild, raise shmmax 581 tps (58 users)
Stripe & Mirror 3 x 2 for data disks 1 pair for BI 304 tps (60 users)
© 2002, Progress Software Corporation39
PROGRESSS O F T W A R EDisk Setups Compared
Configuration tps % of base
1 60 GB internal IDE disk (base) 61 100
JBOD, 8 disks 479 785
RAID 0, 8 disks, data + bi 339 555
RAID 0, 8 data + separate bi disk 481 788
RAID 0, tune + separate bi disk 581 952
RAID 10, 3 pair data, 1 pair bi 304 498
© 2002, Progress Software Corporation40
PROGRESSS O F T W A R EConventional Wisdom
There’s no such thing! Direct I/O parameter reduced performance
(551 118) but chopped off the spikes in the longest transaction duration; resorted to manual syncing similar to pre-directio days
Enabling On-board disk cache DECREASED Performance (506 474)
RAID 0 Stripe Size 1mb was much better than 64k
© 2002, Progress Software Corporation41
PROGRESSS O F T W A R ECost Of After Imaging
Before: 289 tps (92 users)After: 278 tps (82 users)3.8% reduction
© 2002, Progress Software Corporation42
PROGRESSS O F T W A R EConstraints
Time - infinite number of benchmarking possibilities
The unexpected: 3 hours to configure the RAID array from JBOD to RAID 10
Ran out of beer on the last night Occasional Disagreement between the
Members of the Benchmarking Team BI Buffers What do we do next? Where do we eat tonight?
© 2002, Progress Software Corporation43
PROGRESSS O F T W A R ELinux Virtual Memory
bdflushIt’s in fluxHurriedly overhauled in 2.4.10Will be overhauled again in 2.4.19, soonDirect I/O does not produce results
similar to other Unixen
© 2002, Progress Software Corporation44
PROGRESSS O F T W A R EMiscellaneous Stuff
Can’t control the amount of OS Cache at least, we could not
© 2002, Progress Software Corporation45
PROGRESSS O F T W A R EWhat we Didn’t Do
RAID 5 - because as you know it’s Evil Software striping & mirroring Raw partitions Non-standard file systems (JFS, etc) 2.5 kernel Timeslice parameters NAS Storage Areas Records per Block
© 2002, Progress Software Corporation46
PROGRESSS O F T W A R E
Yes, but what about Windows
© 2002, Progress Software Corporation47
PROGRESSS O F T W A R E
Digression:Linux Vs. Windows Performance
Progress 9.1A05 Windows 2000 Advanced Server @$3,500 USD RedHat Linux 6.2, 2.2.14 kernel Same exact hardware for both
IBM Netfinity 8500, 8 x 550 mhz Pentium III 8 gigabytes RAM 40 x 9 Gigabyte SCSI disks 32 megabytes of disk cache
In a previous benchmark, we compared Linuxagainst Windows
© 2002, Progress Software Corporation48
PROGRESSS O F T W A R E
Linux Vs Windows PerformanceResults (9.1A05)
Windows: 3,655 tpsLinux 4,508 tps
Linux was 23 % fasterLinux had more idle cpu ( ~ 15 %)
© 2002, Progress Software Corporation49
PROGRESSS O F T W A R EConclusions
Linux provides excellent performance Could probably run 100 MFG/PRO users
System was easy to set up, easy to useSystem cost was very lowWe had no way to backup the system !!
493 GB of disk in the array Tape robot could cost $12,000 or more
Better than Windows on Intel
© 2002, Progress Software Corporation50
PROGRESSS O F T W A R EWhat Should We Test Next Time ?
Next Linux Benchmark Oct 28, 2002Should we
compare Linux vs Windows with 9.1D? compare AMD vs Intel ? compare filesystems in detail? try BIOS “tweaking”?
Ideas?
© 2002, Progress Software Corporation51
PROGRESSS O F T W A R E
Vragen