informix cac: competitive benchmarks · iwa: enabler for the effective parallel access of the...
Post on 22-May-2020
2 Views
Preview:
TRANSCRIPT
Utility-Scale Smart Meter Deployments, Plans &Proposals*September 2010
© 2010 The Institute for Electric Efficiency
*This map represents smart meter deployments,planned deployments, and proposals by investor-owned utilities and large public power utilities.
http://www.edisonfoundation.net/IEE/
Deployment for>50% of end-users
Deployment for<50% of end-users
Example of Changing Storage Requirements
Monthly MeterReads
Daily MeterReads
15 Minute MeterReads
350.4B
3.65B
120M
Changing Workloads For 10 Million Smart Meters:Today – Each meter read once per monthVery soon – Each meter read once every 15 minutesRegulations – Need to keep data on line for 3 years (PUC) and, perhaps, save for 7 years
# of RecordsPer Year for10M meters
Data for a Utility In California
Frequencyof reads
Keeping up with Smart Meter Data
Large amounts of data causes problems in 2 areas:
• Storage management
• Large storage = Expensive
• Cumbersome to maintain
– Require sophisticated partitioning schemes
– Reorganization often required, which leads to downtime
• Query performance
• Compliance Reports must be completed before the end of eachday
• Customer portal queries must be handled in a timely manner
• Customer billing must be completed each day
100 Million Smart Meter Benchmark: Goals
1. Measure processing times for data collected over a 31 day period for:
100 million meters at 30-minute intervals in an 8 hour day.
2. Demonstrate consistent processing times over the 31 day period.
3. Demonstrate linear storage growth of data stored over the 31 day period.
4. Complete one day’s billing cycle while simultaneously processing and loading meter datafor 100M within an 8 hour time period.
5. Demonstrate all processing can be done using a low-cost combination of commerciallyavailable hardware, storage, and software.
•Operations performed each day
– Load interval and register data for 100M meters at 30 minute intervals (49records/day/meter)
• 49 records/day/meter * 100M = 4.9 Billion records/day
– Perform VEE on the data (validation/estimation/editing)
– Run a daily billing cycle on 6% of the meters
– Gather results for 31 days
100 Million Smart Meter Benchmark: Operations
Perform Validation, Estimation & Editing
Run daily billing cycle
Gather Results
Load & Register Data (100 million meters)
Processed over 30 minute intervals over 31 Days
Proc
essi
ng
100 Million Smart Meter Benchmark: Components
Stor
age
Are
a N
etw
ork
(SA
N)
IBM Power P75032 cores (3.5 GHz) – 16 active500 GB RAM1 GB LAN Fiber dual port adapter - 1 active2 X 8GB FC dual port adapter - 4 active portsof storage
XIV Storage System15 X 2TB storage6 X 6 FC connections @ 4GB
4 X 8GB6 X 4GB
Software Stack
Informix v11.70.xC3 with TimeSeries version 5.0
AMT-Sybex Affinity Meterflow Meter
AIX v7.1
Hardware StackIBM System P Series &Storage
AIX v7.1
Informix 11.70
Monitor & Admin
AMT-Sybex AffinityMeterflow
An “end to end” run of 100 Million Meters at 30 minute intervals was performed for 31 days ofdata
The result: all data was prepared, loaded, validated as well as a billing cycle run in less than 8hours
The average time to do these operations remained consistent over the 31 days– Performance remained constant even as storage increase
The billing cycle completed in less than 5 hours and ran concurrently
100 Million Smart Meter Benchmark: Load Results
Process Avg. ElapsedTime
Avg. ThroughputRatePreparation and Technical
Verification2 hrs 10 min 628,205
records/secData Load 3 hrs 14 min 420,962
records/secValidation, Estimation, andEditing (VEE)
2 hrs 11 min 623,409records/sec
Total Time: 7 hours and 35 minutes!
Tota
l Tim
e-M
inut
es
No. of Days
Total Process Time over 31 Days100 Million Meters @ 30 minute intervals
Load Time over 31 Days100 Million Meters @ 30 minute intervals
Tota
l Tim
e-M
inut
es
No. of Days
Load Performance: Storage Comparison over TimeD
isk
Spac
e in
TB
No. of Days
Storage in TB over 31 Days100 Million Meters @ 30 minute intervals
Why did we choose MDM benchmark?
Informix database has a very strong extensibility story unmatched by anyone inthe industry
Informix’s native handling of time stamped data through TimeSeries extensionsand APIs is already proven to perform orders of magnitude faster takes muchless storage
Helps recruit new partners in the E&U space
Opportunity/potential to grow business by winning new customers in a industrythat is getting transformed
Meet the MDM challenge and solve a key customer problem. Many Energy andUtilities companies are facing scalability issues with competing databasesolutions.
Opportunity leverage on IBM's larger ‘Smarter Planet’ initiative. Gain visibilityand re-vitalize Informix within IBM GBS and S&D teams who are focusing onSmarter Planet solutions
Tick-Tock Development ModelSustained Xeon® Microprocessor Leadership
Tick Tock Tick Tock Tick Tock Tick Tock
Intel® Core™Microarchitecture
NehalemMicroarchitecture
Sandy BridgeMicroarchitecture
65nm65nm 45nm45nm 32nm32nm
Xeon® 5300
Xeon® 5100 Xeon® 5400
Xeon® 5200Xeon® 5500 Sandy
Bridge-EP/ENXeon® 5600 Ivy Bridge
EP/EN
22nm22nm
Dedicated high-speed bus per CPU
HW-assisted virtualization (VT-x)
Integrated memory controller with DDR3support
Turbo Boost, Intel HT, AES-NI1
End-to-end HW-assisted virtualization (VT-x, -d, -c)
Integrated PCI Express
Turbo Boost 2.0
Intel Advanced Vector Extensions (AVX)
First high-volume server Quad-CoreCPUs
Up to 6 coresand 12MB Cache
Up to 8 coresand 20MB Cache
13
Product Line Descriptions
Intel® Xeon® processor 3000 sequence platforms » Intel® Xeon® processor E3 platformsEconomical (1-way) dependable general purpose 64-bit servers well-suited for small businesses andeducation with features that optimize performance, uptime, and security
Intel® Xeon® processor 5000 sequence platforms » Intel® Xeon® processor E5 platformsVersatile (up to 4-way) servers for all your infrastructure, high-density, workstation and HPCapplications with features that enable optimal performance and power efficiency for the datacenter.
Intel® Itanium® processor 9000 sequence platformsArchitected for mission critical UNIX and mainframe-class reliability & scalability. Targeted at large-scale databases, data warehouses, ERP, business intelligence, and data analytics. Deliversuncompromising scalable performance for today’s most demanding workloads and world-classreliability for uninterrupted real-time business processing and decision support
Intel® Xeon® processor E7 platformsScalable (up to 256-way), reliable, powerful 64-bit multi-core servers offering industry-leadingperformance, expanded memory & I/O capacity, and advanced reliability ideal for the most demandingenterprise and mission critical workloads, large scale virtualization and large-node HPC applications.
17
INTEL/IWA: Breakthrough technologies for performance
1
2
34
5
6
7 1
2
34
5
6
7
1. Large memory support64-bit computing; System X with MAX5 supports upto 6TB on a single SMP box; Up to 640GB on eachnode of blade center. IWA: Compress large datasetand keep it in memory; totally avoid IO.
7. Multi-core, multi-node environmentNehalem has 8 cores and Westmere 10 cores. This trend isexpected to continue. IWA: Parallelize the scan, join, groupoperations. Keep copies of dimensions to avoid cross-nodesynchronization.
4. Virtualization PerformanceLower overhead: Core micro-architectureenhancements, EPT, VPID, and End-to-EndHW assist IWA: Helps informix and IWA toseemlessly run and perform in virtualizedenvironment.
5. Hyperthreading2x logical processors; increases processorthroughput and overall performance of threadedsoftware. IWA: Does not exploit this since thesoftware is written to avoid pipeline flushing.
3. Frequency PartitioningIWA: Enabler for the effective parallel accessof the compressed data for scanning.Horizontal and Vertical Partition Elimination.
2. Large on-chip CacheL1 cache 64KB per core, L2 cache is 256KB percore and L3 cache is about 4-12 MB.Additional Translation lookaside buffer (TLB).IWA: New algorithms to avoid pipelineflushing and cache hash tables in L2/L3 cache
6. Single Instruction Multiple DataSpecialized instructions for manipulating128-bit data simultaneously. IWA:Compresses the data into deep columnarfashion optimized to exploit SIMD. Used inparallel predicate evaluation in scans.
Why did we choose Intel benchmark?
IWA takes full advantage of Intel Xeon processor architecture
IWA working with Intel to provide commodity hardware with the bestTCO
Customers can take full advantage of future “chip spin” in processortechnology of Intel with minimal cost
Accommodates terabytes of main memory at little cost
In line with IBM System X product family but compatible with othervendor offerings
top related