22059 slides
DESCRIPTION
TRANSCRIPT
Building a low cost virtualized computing grid for scale
By Bradley D. Brown, TUSCOracle Certified Advantage Partner
The Oracle Experts
3
Bradley D. Brownhttp://bradleydbrown.blogspot.com
TUSC Founder, Author, Chief Technology Officer
University of Denver Adjunct Professor Graduate Class – New Venture Creation
Clients IntelliReal, OAB, EventConnex, Jepp, Sun, …
Groups YPO, OOW, IOUG, ODTUG, LAOUG, RMOUG, etc. Oracle Fusion Middleware Director/Ace IOUC Fusion Council
4
Agenda
Startup 101 N-tier grid
architecture Why build a grid? Scaling
considerations Then and Now Lessons Learned
Moving to Production
Competitive Benefit Cloud computing
and other reference material
5
Startup 101 for Startup #10
Failure 101 Purchased software
of failed entity Denver MSA only,
MySQL and Delphi, Russia
Focused on small market (appraisers)
Goal / Vision Best valuation and
real estate intelligence in the USA
MUST be nationwide
Prototype / POC $20k All Oracle and ApEx Denver MSA only
People loved what we were about to deliver to the market
B2B is key, $ while sleep
Primarily “fixed” costs, low variable costs Conserve cash, retain
equity, build a scalable platform
6
Beyond the Prototype into Beta
Next stages - $1M+ Procure data Differentiator - MLS Full development
Oracle SE One Numerous MySQL
databases for scaling services, etc.
Purchased 25 servers, 30TB of storage for about $50K
Light weight SOA Web Services
Business to Business Used iPerspective to
expose data and business logic as Web Services
Write it once Use it everywhere
Customers UI Internally
7
The Concept and Reference Arch.Building an N-Tier Grid Architecture Approach and
architecture for UI layer and for B2B
Robust, scalable computing environment
Limitless scaling and redundancy in this architecture
8
Grid Architecture
9
Why Build a Grid?
Virtualization Call one service Loose coupling
Scale Redundancy Failover Backup, slower,
lower priority data access
Cost Open Source
10
Grid and Scaling Considerations Oracle SE One
$5000/socket 2 sockets max No DB size limits
4TB database 100M rows “Wide” rows, many
tables 400Gb de-normalized
No partitioning No clustering (RAC) Expensive to move
to EE
Oracle Express Free Very Limiting Limited to 1
processor 4 GB of DB space 1 GB of memory
MySQL Free No limits Oracle version 5
Index optimization? Physical partitioning Partitioning better
now
11
The Grid – then and now
Physical Partitioning 800 partitions initially 10,000+ partitions today Could be different drives or different boxes Partitioner moves the data from Oracle to MySQL MySQL replication moves it to all 10 boxes
10 small Dell 850s - $2k, 4Gb, 1Tb Initially partitioned 5 ways by County (800 counties) with 2
deep for failover and performance Later 1 way, 10 deep and by zip codes
Queries get directed based on partition and prioritized round robin technique through boxes Tiers (best data - Oracle, older - MySQL, etc) Prioritized per box
Upward scalability
12
Lessons Learned – “The Ugly”
Initially boxes were crashing regularly Took time to rebuild Virtualization (VMs)
solves this to a large degree
Can be a lot to support on your own if you don’t purchase software to manage
Extra processing was required Constantly reading
the master database with 10 Partitioner threads Extra “master”
processing Network bandwidth
Replication chews up network bandwidth too
Recommend Open Source ETL approaches such as Pentaho
13
From Concept Into Production
Took $1M more… Gave up some
equity Focused on core
Better values, data More services Benchmarks MLS data
Quick changes based on customer demands
Extensible API for customers
14
Blowing Away The Competition
Big companies Slower to move to
new technology Slower to integrate
new data sources They have a flywheel
$20k today is valued at $30M – 2 more years, $100M
Why? Quick response to
market needs Focused on core, not
context Displaced the market iPerspective
Write in language you know, takes years to master a language
15
Cloud Computing and Virtualization Amazon EC2
Powerful Grid
Oracle Cloud Uses EC2 and S3
SaaS
Deploy and scale your app in hours without changing code
Provision, monitor and manage operations with just a browser
Scale from a fraction of a server to hundreds of CPUs in days
Get your life back -- no more late night rushes to replace failed equipment
16
Periscope Virtualizes Google’s Web ServiceSELECT *
FROM google
WHERE searchstring = ‘RMOUG Brad Brown’
RMOUG Brad Brown – 8 records
RMOUG – 374 records
17
Good Papers, Presentations, …
Step-by-step Web Service Creation Web 2.0 & Apex Presentations Watch my Blog
http://bradleydbrown.blogspot.com
IntelliReal Case Studies
Java-based Oracle Web Development Java Server Pages JavaMail Java for the PL/SQL Developer Web Cache – achieving 150 the performance 9iAS Installation, Configuration, and Tuning Wireless
Practical Portal Practices Implementing JSP in Portal UltraSearch Search Engines Utl_smtp and Utl_http iFS JavaScript Top DBA scripts for Web
Developers Security
18
Summary
Startup 101 N-tier grid
architecture Why build a grid? Scaling
considerations Then and Now Lessons Learned
Moving to Production
Competitive Benefit Cloud computing
and other reference material
19
Questions?
20
Copyright Information
Neither TUSC nor the author guarantee this document to be error-free. Please provide comments/questions to [email protected].
TUSC © 2008. This document cannot be reproduced without expressed written consent from an officer of TUSC.