the new alchemy: turning data into gold by brian johnson engineering director, ebay
DESCRIPTION
Developers are leading the charge to turn consumer behavior into profitable solutions. By accessing and analyzing the explosion of data from consumer activities, any developer can create the personalized, relevant products and services that customers demand and merchants urgently need. Brian will discuss how to acquire, store, and mine information, and how to design analytics-focused software and build data-driven software engines.TRANSCRIPT
![Page 1: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/1.jpg)
!!
!
CloudCon, Tuesday, October 2nd, 11am
![Page 2: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/2.jpg)
Every Second – in over thousands of Categories
![Page 3: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/3.jpg)
www.wallpapertimes.com
$’s per year in incremental revenue
Value > Cost
![Page 4: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/4.jpg)
Big
![Page 5: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/5.jpg)
Detail
![Page 6: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/6.jpg)
structured
incremental storage
processing
change
Volume
Variety Velocity DATA
semi-structured un-structured
![Page 7: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/7.jpg)
Data Warehouse + Behavioral
Data Warehouse
Semi-Structured SQL++
Structured SQL
Low End Enterprise-class System
Contextual-Complex Analytics Deep, Seasonal, Consumable Data Sets
Production Data Warehousing Large Concurrent User-base
Enterprise-class System
Unstructured Java/C++/Pig/Hive
Structure the Unstructured Detect Patterns
Commodity Hardware System
8+PB 60+PB 40+PB
Hadoop
Analyze & Report
Discover & Explore
![Page 8: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/8.jpg)
! Data Growing Faster
![Page 9: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/9.jpg)
![Page 10: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/10.jpg)
10
questions later
structure later
(<$0.04/GB, <$80/2TB)
single HDFS instances >50PB
Value > Cost
Data
![Page 11: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/11.jpg)
Designing for the Unknown >85% of analytical workload is NEW & Unknown The metrics you know are cheap The metrics you don’t know are expensive – but high in potential ROI Exploration & Testing are core pillars of an analytics-driven
organization
![Page 12: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/12.jpg)
• Impact
![Page 13: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/13.jpg)
![Page 14: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/14.jpg)
!"#$ %$& '()*+,"-+ .-)/01$2& 3-#$4! 5"*2"$, 5"*2&4! 6*77"$, 6*77&4! 82*+6$22"$, 82*+6$22&4! 9-77"+7 9-74! :",;"+7/,#"8<$2 :",;/,#"8<$2,4% )*+$=, )*+$=="+74% )2-#$8#"-+ )2-#$8#$24% ="+"+7 ="+$54% *+">*#"-+ *+">*#$54% #218<, #218<"+74% $57"+7 $57$,4% +$#, +$##"+7
![Page 15: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/15.jpg)
!"#$ %$& '()*+,"-+ .-)/01$2& 3-#$4! 5"*2"$, 5"*2& 6*7)"2$/5"*2"$,4! 8*99"$, 8*99& )*#*9-+"*/8*99"$, 9--5/:-2/)*#*9-+"*/8*99&;/+-#/9--5/*<-+$4! =2*+8$22"$, =2*+8$22& #>$/=2*+8$22"$,4! ?-99"+9 ?-9 ?-99"+9/,#2-<<$24! :",>"+9/,#"=@$2 :",>/,#"=@$2, :",>"+9/,#"=@$2 ,)-2#,/6,A/@"5,/2--7,4% )*+$<, )*+$<<"+9 :$+=$/)*+$<,4% )2-#$=#"-+ )2-#$=#$2 7=*:$$/#-#*</)2-#$=#"-+/BCDB ,=2$$+/)2-#$=#$2/",/#-)/4!/E1$2&4% <"+"+9 <"+$5 )"+@/<"+"+9/=>*+9"+9/8*94% *+"7*#"-+ *+"7*#$5 *+"7*#"-+/=$<4% #21=@, #21=@"+9 =-29"/#21=@,4% $59"+9 $59$, 9*25$+/$59"+94% +$#, +$##"+9 )12,$/+$#,
![Page 16: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/16.jpg)
![Page 17: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/17.jpg)
![Page 18: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/18.jpg)
www.wallpapertimes.com
$’s per year in incremental revenue
Value > Cost
![Page 19: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/19.jpg)
![Page 20: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/20.jpg)
![Page 21: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/21.jpg)
Toys and Hobbies ATC > Artist trading card in ART ATC > Automatic Tool Change in Business and Industrial
![Page 22: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/22.jpg)
German Compound Words • German compound words can be arbitrarily created and extremely long
Adidastrainingsanzug (Adidas track suit) Rindfleischetikettierungsüberwachungsaufgabenübertragungsgesetz
(beef labeling regulation & delegation of supervision law) • Syntactically, words can be combined and split in many ways. • Some words shouldn’t be de-compounded.
beiden (both) – bei(at) den(the) • Too many candidates for
Granitpflastersteine (granite paving stones) Granit(granite) pflastersteine(cobblestones) Granit(granite) pflaster(paving/band-aid) steine(stones)
• Binding characters Hochzeitsschuhe (grammatically correct, 593 hits on ebay.de) Hochzeitschuhe (129 hits on ebay.de).
![Page 23: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/23.jpg)
Synonyms derived from top queries in item query clusters texas instruments ba ii plus / ba ii plus brighton handbag brighton purse lenovo x200 thinkpad x200 king bedspread king coverlet rockabilly dress swing dress 1963 ford falcon 63 falcon jessica simpson hair extensions jessica simpson hairdo
Abbrevia7ons/acronym derived from query transi7ons
stanford ky stanford kentucky dc sub dc subwoofer snowboard helmet l snowboard helmet large motorcycle cam motorcycle camera diamond amp diamond amplifier
![Page 24: The New Alchemy: Turning Data into Gold By Brian Johnson Engineering Director, eBay](https://reader033.vdocument.in/reader033/viewer/2022052622/5593c2151a28abaf4a8b468a/html5/thumbnails/24.jpg)