Download - Michael newberry
Extracting Value from Big Data in the Cloud -
Michael Newberry
Big data in a Hybrid-Cloud worldDr Michael Newberry
Windows Azure Lead, Microsoft [email protected]
Doggerland: Simon Fitch, Vince Gaffney and Ken ThomsonImage Source: drowned-landscapes.tumblr.comRoyal Society's Summer Science Blog (http://summer-science.tumblr.com/)
Big Data.
Big Data.
VOLUME (Size)
VARIETY (Structure)
VELOCITY (Speed)
Getting useful insightsfrom awkward data setsusing the most appropriate computing platform at each stage.
Dr Michael NewberryWindows Azure LeadMicrosoft UK
Big data in a Hybrid-Cloud worldDr Michael Newberry
Windows Azure Lead, Microsoft [email protected]
Machine Learning & Bayes theorem
π ( hπππ πππππ’π¦ππππ ππππ ππhπhππ ππ’π π‘ hπππ’π π‘ hπ πππ )πππππππ ππ
π ( hπππ πππππ’π¦ππππ ππππ )π ( hπππ πππππ’π¦πππ hπ πππ )
π ( π΄β¨π΅ )=π (π΅β¨π΄ ) π ( π΄ )π (π΅ )
β¦.Amazon (AMZN) calls this homegrown math "item-to-item collaborative filtering," and it's used this algorithm to heavily customize the browsing experience for returning customersβ¦. Judging by Amazon's success, the recommendation system works. The company reported a 29% sales increase to $12.83 billion during its second fiscal quarter, up from $9.9 billion during the same time last year. A lot of that growth arguably has to do with the way Amazon has integrated recommendations into nearly every part of the purchasing process from product discovery to checkout.
http://tech.fortune.cnn.com/2012/07/30/amazon-5/
βIn theory there is no difference between theory and practice; in practice, there isβ.
Yogi Berra, cited in Nassim Taleb, Antifragile.
Big data techniques
NoSQL (ala MongoDB) Map-Reduce (e.g. Hadoop)
Embedded devices
Connected Devices
On Premise
Off Premise
Business Intelligence
Customers Employees, Partners
The Power of an Intelligent System
Modern Platform for the Worldβs Apps
Cloud OS
transforms the datacenterenables modern appsunlocks insights on any dataempowers people-centric IT
Cloud OS
flexible developmentunified dev-ops & managementcomplete data platformcommon identityintegrated virtualization
MICROSOFT
SERVICE PROVIDERON-PREMISES
1CONSISTENTPLATFORM
What Makes the Cloud OS Unique
RelationalNon-Relational Streaming
MANAGE ANY DATA, ANY SIZE, ANYWHERE
010101010101010101101010101010101001010101010101101010101010
Unified Monitoring, Management & Security
Data Movement
POLYBASE: COMBINING RELATIONAL AND NON-RELATIONAL DATAThe future of query processing
select... results set
Hadoop Data Warehouse
PolyBase
Single query for relational & Hadoop data
Process data in place
Future expansion to other data sources
Seamless: regular T-SQL command
19
20
Avoiding Lock-InWindows Virtual machines can move freely between all 3 clouds.
Windows Azure
Customer Data Center
Other Service ProvidersWindows
Virtual Machine
LocationOn-Premises On-Premises or
Service ProviderMicrosoft Cloud orService Provider
Rationale for Usage
Compliance
Scalability
Economies of Scale
Rapid Development
Complex, Legacy Applications
Compliance
Economics
TraditionalNON-VIRTUALIZED
AppliancePRIVATE
CloudPUBLIC
(Outside Firewall)
DATA PLATFORM DELIVERY MODELS
(Inside Firewall)
BALANCING ON PREMISE & CLOUDSnowline graph
A
Takeaways
1. βbig dataβ can do some amazing stuff.2. Donβt think βbig dataβ as much as βdata needing non-
relational approachesβ3. If your big data insights are probabilistic, which they often are,
have a plan to deal with variance. 4. Pick the most appropriate platform: Think βandβ not βorβ:
- Balance public cloud AND on-premise,- Combine βbig dataβ with RDBMS.
Q+A