think huddle and neustar performance gains with 12c -...
TRANSCRIPT
![Page 1: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/1.jpg)
OVERVIEW • “Big Data” and its impact on Spatial performance • Testing configurations • Use cases / techniques for improving performance • Performance improvement data
CHALLENGES / OPPORTUNITIES • With 11g, performance sometimes limited interactions with
Big Data • Making improvements was time-consuming
SOLUTIONS • Oracle Database 12c Enterprise Edition
• Spatial Option with GeoRaster, Network Data Model • Partitioning
• Oracle Fusion Middleware 11.1.1.7 • MapViewer
• Oracle Business Intelligence Enterprise Edition
RESULTS • Achieving at least 2x performance improvement – in
some cases up to 300x improvement • Only small configuration changes are needed • Time to develop and maintain these solutions is reduced
Think Huddle and Neustar Performance Gains with 12c /
0 500
1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 5,000
11g 12c 12c SVA
6.5x faster performance 11m 53s vs 1hr 17m 26s
![Page 2: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/2.jpg)
May 2014 Oracle Spatial Summit
![Page 3: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/3.jpg)
May 21, 2014 Walter E. Washington Convention Center
Washington, DC USA
![Page 4: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/4.jpg)
Steve Pierce CEO
Nick Salem Distinguished Engineer
![Page 5: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/5.jpg)
Performance Gains with Spatial & Graph Using Oracle Database 12c
![Page 6: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/6.jpg)
• Big Data: What’s the big deal? • Testing configurations • Use cases and performance data on:
– Geometry aggregation – Point in polygon – Joins
• Q&A
Program Agenda
![Page 7: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/7.jpg)
Think Huddle Corporate Overview
• Our company – Oracle Spatial and MapViewer – Big Data and BI for Spatial – Fortune 500 clients
• Steve Pierce – Chair – Oracle Spatial SIG – 15 years Oracle
![Page 8: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/8.jpg)
What’s the Big Deal?
• Data growing – Big Data systems for spatial processing
• Fundamental changes in 12c – Faster vector algorithms – Better caching – Improved memory management
• 12c works for Big Data – (and maintains data integrity – … and keeps data secure – … and supports relational integrity – … and your users can access it)
BIG DATA
![Page 9: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/9.jpg)
Test Environment
Environment Oracle 11g Oracle 12c
OS Oracle Linux 6.5 Oracle Linux 6.5
Processor E5 Xeon 2.9Ghz x 2
E5 Xeon 2.9Ghz x 2
Disk Intel SSD s Intel SSD s
Memory 32GB 32GB
Huge Pages Yes Yes
SGA/PGA 8GB/2GB 8GB/2GB
Source data Census / Generated
Census / Generated
• 1 Server – 2 Oracle Databases
![Page 10: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/10.jpg)
• Trusted company serving the internet, telco, information services, retail, media and advertising industries
• Publicly held - $902M+ Company • 1,600+ employees worldwide – HQ’d in Sterling, VA • Extensive data assets (4 billion global phone numbers,
2.8 billion global IP addresses, 6.4 million global domain names and 13.1 US business listings)
• 20 billion requests a day (DNS, text messages, phone)
Neustar Corporate Overview Real Intelligence. Better Decisions.
![Page 11: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/11.jpg)
Neustar’s ElementOne Platform Geospatial Analytics in the Cloud
• Highly flexible and high performance mapping
• Rich market and trade area capability • Road network / drive time analysis • Heat themes • GeoRaster images • Aerials / terrain map overlays • Scalable and high performance data
retrieval • Centralized secure data repository
![Page 12: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/12.jpg)
• Oracle VM Server 3.0 (Xen version 4.1 PVM) • Created 2 identical VMs with Oracle Linux 6.5
– Installed Oracle 11g on one and 12c on the other
• Database configuration – 8G of SGA / 2G of PGA – 4 x 8 Core CPUs (Intel Xeon 2.70GHz)
• Loaded core ElementOne database on each • Compare 11g to 12c w/ & w/o spatial vector acceleration
Neustar’s ElementOne Platform 11g vs 12c Test Benchmark Environment
![Page 13: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/13.jpg)
Use Case – Geometry Aggregation
Aggregation of census tracts into larger geometries based on neighbor relationships
INTERACT UNION
![Page 14: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/14.jpg)
• Each Census tract in Virginia – Aggregate Union of Neighbors
• SDO_ANY_INTERACT + SDO_AGGR_UNION
– 2,000 Aggregations – Tracts have 7 neighbors = 14,000 tracts – 280 tracts processed per second 0
50
100
150
200
250
300
350
11g 12c 12c SVA
Elapsed Time (s)
Use Case – Geometry Aggregation
![Page 15: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/15.jpg)
Neustar’s ElementOne on 12c SDO_AGGR_UNION – CA ZIPs
• Use case consisted of aggregating all ZIP Codes in California using plain SDO_AGGR_UNION and with mod/group by functions
• Achieved up to 300x faster performance w\ SDO_AGGR_UNION and 40x faster with mod/group by method in12c with SVA enabled vs 11g
• Reduced elapsed time from 1 hour 54 seconds to just 12 seconds
0 500
1,000 1,500 2,000 2,500 3,000 3,500 4,000
11g 12c 12c SVA
Elapsed Time In Seconds
Using SDO AGGR UNION
SDO AGGR UNION + Mod/Group
![Page 16: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/16.jpg)
Neustar’s ElementOne on 12c SDO_AGGR_SET_UNION – CA ZIPs
0 20 40 60 80
100 120 140 160 180 200
11g 12c 12c SVA
14x faster performance 12s vs 2m 51s
SDO AGGR SET UNION
![Page 17: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/17.jpg)
Neustar’s ElementOne on 12c SDO_TOUCH – CA ZIPs touch BGs
• For all California ZIP codes, retrieve the block groups that they touch
0
50
100
150
200
250
11g 12c 12c SVA
8x faster performance 25s vs 3m 23s
SDO_TOUCH
![Page 18: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/18.jpg)
Neustar’s ElementOne on 12c Spatial Relationships
• Determine spatial relationships and interactions – Inside, touch, anyinteract, overlap, disjoint
• SDO_UTIL and SDO_GEOM for various geography creation and manipulation
• Shape intersection, XOR, union, difference, point at bearing, radian, distance, convex hull, polygon smoothing and more
• ElementOne Trade Areas (customer threshold, non-overlapping radii + component geos, donuts, square + hex grids, drive time, heat regions)
• Thematic overlays and extents
![Page 19: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/19.jpg)
Neustar’s ElementOne on 12c SDO_WITHIN_DISTANCE Benchmarks
• For 544 super regional shopping centers, retrieve all businesses with a 10 mile radius
0 100 200 300 400 500 600 700 800 900
1,000
11g 12c 12c SVA
6x faster 2m 17s vs 14m 32s
SDO WITHIN DISTANCE
![Page 20: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/20.jpg)
Neustar’s ElementOne on 12c SDO_GEOM.RELATE – CA ZIPs & BGs
• Determine spatial topological relationship for California ZIP codes and surrounding block groups
0 50
100 150 200 250 300 350 400 450
11g 12c 12c SVA
3x faster performance 2m 16s vs 7m 2s
SDO_GEOM RELATE (mask = determine)
![Page 21: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/21.jpg)
Neustar’s ElementOne on 12c SDO_GEOM.DISTANCE Benchmarks
• Calculate distances between a single site and 112,610 surrounding sites
0
50
100
150
200
250
11g 12c 12c SVA
2.6x faster performance 1m 13s vs 3m 11s
SDO_GEOM DISTANCE
![Page 22: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/22.jpg)
Neustar’s ElementOne on 12c SDO_INSIDE – Businesses inside NY BGs
• Retrieve all businesses inside New York state block groups
0 20 40 60 80
100 120 140 160 180 200
11g 12c 12c SVA
2.1x faster performance 1m 22s vs 2m 53s
SDO_INSIDE
![Page 23: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/23.jpg)
Use Case – Point in Polygon
Point in polygon calculations for finding houses within an area of interest
What’s in there?
![Page 24: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/24.jpg)
Use Case – Point in Polygon 11g
• In 11g there were ways to do this SELECT lon, lat FROM (SELECT SDO_GEOMETRY(2001, 8307,
SDO_POINT_TYPE(lon, lat, NULL), NULL, NULL) geom FROM large) c,
states s WHERE SDO_ANYINTERACT(s.geom,c.geom) = 'TRUE' AND s.state_abrv='CA'
![Page 25: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/25.jpg)
Use Case – Point In Polygon 12c
0
50
100
150
200
250
300
11g Relate
12c SVA PiP
12c SVA PiP (8)
Elapsed Time (s)
SELECT lon, lat FROM TABLE(mdsys.sdo_PointInPolygon( CURSOR(select lon, lat from large),
(select geom from states where state_abrv='CA‘ ) , .05))
• /*+ PARALLEL(#) */ – Near linear performance boost
• Sdo_PointInPolygon
![Page 26: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/26.jpg)
Use Case – Joins
Which objects are in which region?
And there?
What’s in there? And here?
![Page 27: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/27.jpg)
Use Case – Joins
• SDO_JOIN – For bulk joins – sdo_join('LARGE', 'GEOM', 'STATES',
'GEOM', 'mask=anyinteract'
• Works in 11g
• Faster in 12c 0
100
200
300
400
500
600
700
11g 12c 12c SVA
Elapsed Time (s)
![Page 28: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/28.jpg)
Neustar’s ElementOne on 12c SDO_JOIN – ZIP & Block Group Tables
• Join using mask=anyinteract
0 500
1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 5,000
11g 12c 12c SVA
6.5x faster performance 11m 53s vs 1hr 17m 26s
SDO_JOIN
![Page 29: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/29.jpg)
Neustar’s ElementOne on 12c SDO_CS.TRANSFORM – US block groups
0 200 400 600 800
1,000 1,200 1,400 1,600
11g 12c 12c SVA
10x faster performance 2m 10s vs 22m 26s
SDO CS Coordinate Transform
![Page 30: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/30.jpg)
Neustar’s ElementOne on 12c SDO_GEOM.VALIDATE_GEOM Benchmarks
• Validate all US county geometries
0
5
10
15
20
25
30
35
11g 12c 12c SVA
1.9x faster performance 15s vs 29s
SDO_GEOM Validate Geometry with Context
![Page 31: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/31.jpg)
• Reduction in redo/undo generation using global temporary tables resulting in 1.7x performance boost
• 2x performance gains with inline PL/SQL functions • Up to 10x faster processing time with constructing large
CLOBs / BLOBs • Faster response times with cross session result set
cache
Neustar’s ElementOne on 12c Miscellaneous Features
![Page 32: Think Huddle and Neustar Performance Gains with 12c - Oracledownload.oracle.com/.../pdf/oss14/oss14_performance_piercesalem.pdf · • “Big Data” and its impact on Spatial performance](https://reader030.vdocument.in/reader030/viewer/2022020723/5a8531107f8b9a87368c9a45/html5/thumbnails/32.jpg)
Q&A