![Page 1: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/1.jpg)
Firebird Firebird ScalabilityScalability
Dmitry [email protected]
The Firebird Projecthttp://www.firebirdsql.org
![Page 2: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/2.jpg)
A36Dmitry Yemanov
Slide 2
AgendaAgenda Classic Server vs Super Servercomparison, limits, solutions
Memory usagepage and sorting buffers
CPU and concurrencymulti-CPU and multi-core, TPC-C figures
I/O specificspossible improvements
Scaling up vs scaling outclustering possibilities
![Page 3: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/3.jpg)
A36Dmitry Yemanov
Slide 3
Architectural DifferencesArchitectural Differences Classic Server Process per connection, single-threaded Native scheduling Private caches Shared database file usage Shared lock table
Super Server Single process, multi-threaded Cooperative scheduling Shared caches Exclusive database file usage In-process lock table
![Page 4: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/4.jpg)
A36Dmitry Yemanov
Slide 4
Classic Server: Limiting FactorsClassic Server: Limiting Factors Page cache Balance between available RAM and load Frequent modifications = extra page writes
Lock manager bottlenecks Single mutex, LockAcquireSpins on SMP Lock access time, LockHashSlots LockMemSize: 100 * <connections> * <cache pages>
Other limits Signal delivery (POSIX) Desktop heap size (Windows)
![Page 5: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/5.jpg)
A36Dmitry Yemanov
Slide 5
Super Server: Limiting FactorsSuper Server: Limiting Factors Page cache Total amount, 32-bit is not enough Many pages (old FB versions)
Cooperative thread scheduling One request active at a time Rescheduling points
Other limits Socket pool limits (1024 on Windows) Stack size (~1000 WNET/XNET connections)
![Page 6: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/6.jpg)
A36Dmitry Yemanov
Slide 6
A Mixture of BothA Mixture of Both Key ideas behind
Single process, multi-threaded Thread pooling Private caches per connection Communication via the lock manager In-process lock table
Implementations Vulcan “No-Shared-Cache” mode RedSoft “SuperClassic” architecture Firebird “Transition” branch
![Page 7: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/7.jpg)
A36Dmitry Yemanov
Slide 7
Memory UsageMemory Usage Page cache Few pages = many I/O operations Many pages = possible problems in CS Remember about the metadata in CS Efficient implementation in SS
Temporary space The bigger the better Should be used with care in CS
Other Lock and event tables (shared memory)
![Page 8: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/8.jpg)
A36Dmitry Yemanov
Slide 8
Page Cache Performance for Page Cache Performance for WritersWriters
2000 10000 50000 200000
v1.5v2.0v2.1
![Page 9: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/9.jpg)
A36Dmitry Yemanov
Slide 9
CPU ScalabilityCPU Scalability
1 2 4 8 16
SuperServerClassic
![Page 10: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/10.jpg)
A36Dmitry Yemanov
Slide 10
Concurrency: TPC-C benchmarkConcurrency: TPC-C benchmark
1 10 100 1000
SuperServerClassic
![Page 11: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/11.jpg)
A36Dmitry Yemanov
Slide 11
I/O SpecificsI/O Specifics Tablespaces Data separated from indexes and blobs Separated user data (per schema)
Partitioning Active vs backup, by warehouse, etc
Underlying improvements Asynchronous I/O Multi-page reads Cache prefetch
![Page 12: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/12.jpg)
A36Dmitry Yemanov
Slide 12
Scaling Up: Searching for the Best Scaling Up: Searching for the Best SetupSetup
Firebird 2.0 Classic on Linux Use a 8K or 16K page size Find (experimentally) an optimal page cache size Calculate and set up LockMemSize accordingly Play with LockAcquireSpins and test the performance Consider FW=OFF with MaxUnflushedWrites = 1 Set up SortMemUpperLimit, if necessary Monitor the lock table and increase LockHashSlots
![Page 13: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/13.jpg)
A36Dmitry Yemanov
Slide 13
Scaling Out: Firebird ClusterScaling Out: Firebird Cluster
fb_inet_server
fb_inet_server
fb_inet_server
fb_inet_server
LockTable Database
![Page 14: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/14.jpg)
A36Dmitry Yemanov
Slide 14
Scaling Out: Firebird ClusterScaling Out: Firebird Cluster
fb_inet_server
fb_inet_server
fb_inet_server
fb_inet_server
DistributedLock
Manager
e.g.
RedHatDLM
Database
LT
LT
LT
LT
![Page 15: Firebird Scalability, by Dmitry Yemanov (in English)](https://reader033.vdocument.in/reader033/viewer/2022061210/54906bbbb4795971178b4b72/html5/thumbnails/15.jpg)
A36Dmitry Yemanov
Slide 15
The Firebird ProjectThe Firebird Projectwww.firebirdsql.org