![Page 1: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/1.jpg)
PostgreSQL in Mission-Critical
Financial SystemsMay 20th 2010
Flavio Henrique Araque Gurgelhttp://creativecommons.org/licenses/by-sa/3.0/deed.pt
![Page 2: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/2.jpg)
4Linux in Caixa 2nd level support, 24/7 Configuration, tuning Tests and documentation Knowledge transfer (training) Commitment: employees should attend
communities Send code to the upstream in the same
license
![Page 3: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/3.jpg)
Caixa Econômica Federal
All cities of Brazil Social benefits “Every Brazilian has used or will use
Caixa services” Caixa is a bank too!
![Page 4: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/4.jpg)
Multicanal EnvironmentWhat is it?
Multicanal is the Brazilian word for “Multichannel”
1) Banking and Social transactions− Today: ATM self-service
2) Monitoring
3) SMS
![Page 5: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/5.jpg)
Multicanal Environmentin Numbers
More than 22.000 ATM Peak: more than 7 million transactions in
a day Peak: more than 18 million database
transactions in a day More than R$ 1 billion per month
(US$ 1 ~ R$ 1.8 in May 14th, 2010)
![Page 6: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/6.jpg)
Multicanal EnvironmentCharacteristics
High availability High performance High reliability
MISSION-CRITICAL!!!
![Page 7: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/7.jpg)
CSM
JBoss AS
JBoss AS
JBoss AS
JBoss AS
PostgreSQL
PostgreSQL
ATM
Site replicationHitachi USP-VM
Contingency Site
Java EEApplication
Host IBM
Multicanal EnvironmentTransactions (self-service)
![Page 8: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/8.jpg)
Multicanal EnvironmentTests made with the system
750 TPS MinimumFinancial Transactions per Second
zOS + WebSphere + DB2
Debian + JBoss + PostgreSQL
Solaris + Sun Java System + Oracle
![Page 9: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/9.jpg)
Multicanal EnvironmentThe choice
zOS + WebSphere +DB2Best TPS.
Debian + JBoss + PostgreSQLTPS was ok.1% of costs compared to zOSCertified
Solaris + JBoss + PostgreSQLLeast total costSolaris is widely used in CaixaDefinitive Platform
![Page 10: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/10.jpg)
JBoss AS
JBoss AS
JBoss AS
JBoss AS
PostgreSQL
PostgreSQLATMs
Site replicationHitachi USP-VM
Contingency Site
Java EEApplication
Multicanal EnvironmentMonitoring
TransactionalApplication
Managers and Agencies
![Page 11: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/11.jpg)
JBoss ASJBoss ASPostgreSQL
PostgreSQL
Site replicationHitachi USP-VM
Contingency Site
Java EEApplication
Host IBM
Multicanal EnvironmentSMS
SMSBroker
Managers and Agencies
![Page 12: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/12.jpg)
Multicanal EnvironmentDatabase Infrastructure
Infrastructure until final scale− Debian Etch Linux kernel 2.6.24/64bit
− PostgreSQL 8.3.6 – community code compiled in-house
Final infrastructure− Migrated to Solaris 10u6/Sparc 64bit
− PostgreSQL 8.3.7 – Sun Package - 64bit
− Transactions System – OLTP - 250 GB database
− Monitoring System – DW - 1 TB database
− SMS System – OLTP – 20 GB database
![Page 13: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/13.jpg)
Multicanal EnvironmentHardware
Dell Servers:− 2x Quad Xeon HT (2x4x2HT cores) @1.8GHz
− 16 GB RAM
HP Servers:− 2x Quad Xeon (2x4 cores) @2GHz
− 32 GB RAM
Sun Fire E25K (per system board):− 2x Ultra Sparc IV+ (4x2 cores) @1.5GHz
− 32 GB RAM
![Page 14: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/14.jpg)
Multicanal EnvironmentStorage Sub-system
Características:− Storage Area Network – Fiber Channel
− Virtualizador Hitachi – cache 16 GB
− Switch McData
Produção:− Hitachi USP-VM – cache 8 GB
− SATA 15000 rpm – RAID5 – 6 e 7 discos
Backup:− On-line: Storage EMC CLARiiON CX Series
− Fita: IBM Tivoli
Characteristics:− Storage Area Network – Fiber Channel
− Hitachi Virtualizer – 16 GB cache
− McData Switches
Production:− Hitachi USP-VM – 8 GB cache
− FC disks 15000 rpm – RAID5 – 6 and 7 disks
Backup:− Online: EMC CLARiiON CX Series Storage
− Tape: IBM Tivoli
![Page 15: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/15.jpg)
Multicanal EnvironmentReplication
Hitachi Virtualizer Synchronous Replication
Fiber Optics Site-to-site distance:
approximately 4 km (~ 2.5 miles)
![Page 16: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/16.jpg)
Daily Challenges
Availability OLTP Stability DW Performance Maintenance Other DBMS knowledge
![Page 17: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/17.jpg)
Daily ChallengesAvailability
Online Backup and PITR strategy One initial copy each day Two days in disk Initial copy and archived WAL are
replicated
![Page 18: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/18.jpg)
Daily ChallengesStability
Keep the Throughput!!! Tuning PostgreSQL
− shared buffers− work_mem− Connection count− checkpoints − bgwriter − sync method− autovacuum
![Page 19: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/19.jpg)
Daily ChallengesStability
Manter Throughput!!! Tuning Debian
− escalonador de I/O− patch kernel, kswapd
Tuning Solaris− max_queue_msg− force_direct_io (UFS)− ARC cache (ZFS)
Keep the Throughput!!! Debian Tuning
− I/O scheduler− kswapd kernel patch
Solaris Tuning− max_queue_msg− forcedirectio (UFS)− ARC cache (ZFS)
![Page 20: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/20.jpg)
Daily ChallengesStability
Keep the Throughput!!! Disks Tuning
− tablespaces for mostly accessed tables− tablespaces for mostly accessed indexes− Exclusive disk array for pg_xlog
Synchronous replication tuning− High concurrency in remote storage− Fiber parameters and cache
![Page 21: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/21.jpg)
Daily ChallengesDW performance
DW query tuning When more than 10 million live tuples:
− Table partitioning− Automatic partitioning function− Development collaboration
XML data in tables− Development collaboration− Indexing tables
![Page 22: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/22.jpg)
Daily ChallengesDatabase maintenance
All systems are 24h!!! autovacuum How to schedule REINDEX How to schedule dump/restore
− Or VACUUM FULL + REINDEX??? “state machine” → partitioned
− Development collaboration Purge (database and application)
![Page 23: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/23.jpg)
Daily ChallengesParadigms
Reality in data-centers is not PostgreSQL (yet)
Knowledge transfer Relative limitations Relative advantages No DBMS is the same as another How to “think PostgreSQL”
![Page 24: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/24.jpg)
After challenges, praises
Performance, high TPS Data was never lost Extensive documentation Open source software model Issues are solved quickly Cost relatively low
![Page 25: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/25.jpg)
NeedsPerformance and availability
Scale out− Multi-master cluster (Postgres-XC)?
− Transparent memcached
− Use more resources in maintenance routines
SQL administration (configuration) System catalogs for performance
monitoring Corporate audit flexibility
![Page 26: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/26.jpg)
pgpool-II and Sequoia OLTP – performance was not enough PostgreSQL stand-alone – 750 TPS*
pgpool-II + 2 PostgreSQL – 170 TPS*
Sequoia + 2 PostgreSQL – 50 TPS*
* Financial Transactions per Second, Caixa minimum is 700 TPS
Multi-master for availabilityTests 4Linux and Caixa
![Page 27: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/27.jpg)
Final needs(and more important)
Tools Integrated and graphical monitoring and
administration tool All the database infrastructure Incident prevention alerts, automatic
reports
![Page 28: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/28.jpg)
Conclusion
PostgreSQL is ready for mission-critical but we need to evolve we know how to evolve we can evolve
![Page 29: PostgreSQL in Mission-Critical Financial Systems€¦ · · 2016-01-14PostgreSQL in Mission-Critical Financial Systems ... zOS + WebSphere + DB2 Debian + JBoss + PostgreSQL Solaris](https://reader031.vdocument.in/reader031/viewer/2022022514/5af52a687f8b9a9e598dad3d/html5/thumbnails/29.jpg)
Special Thanks! Euler Taveira de Oliveira Caixa Econômica Federal PostgreSQL Community 4Linux