big data i arkitektura big data aplikacije meetup
TRANSCRIPT
![Page 1: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/1.jpg)
Big Data i Arhitektura Big Data aplikacije
Darko Marjanovic CEO & Co-Founder @ Things Solver [email protected]
![Page 2: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/2.jpg)
• Implementacija Big Data rešenja
• Big Data consulting
• Big Data treninzi
Partneri
![Page 3: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/3.jpg)
Agenda
• Big Data
• Big Data primeri
• Arhitektura Big Data aplikacije
• Primeri
![Page 4: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/4.jpg)
Šta je to Big Data?
• Big Data predstavlja podatke koji pristižu velikom brzinom i one su količine koja prevazilazu mogućnosti tradicionalnog softvera za skladištenje, obradu i upravljanje podacima.
• Big Data je sve ono što ne može da stane u Excel.
![Page 5: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/5.jpg)
Big Data - Dimenzije
![Page 6: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/6.jpg)
Big Data - Izvori podataka
• Društvene mreže (Twitter, Facebook…)
• Email, HTML, Click Stream
• Slike, Video, Logovi, Senzorski podaci
• Relacione Baze podataka
![Page 7: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/7.jpg)
![Page 8: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/8.jpg)
Big Data - Primeri
• Šta se najviše kupuje uz pivo?
• U kojoj vezi su kiša i integralna peciva?
• House of Cards i Big Data?
![Page 9: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/9.jpg)
![Page 10: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/10.jpg)
![Page 11: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/11.jpg)
Big Data - Korisnici
![Page 12: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/12.jpg)
Male i srednje kompanije?
• Cena
• Benefiti
• Budućnost
![Page 13: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/13.jpg)
Aplikacije - Šta treba uzeti u obzir
• Sadašnje i potencijalne izvore
• Brzinu rasta podataka
• Biznis potrebe
• Resursi? Cloud je jeftin, Hadoop je besplatan
• Cena
![Page 14: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/14.jpg)
Arhitektura Big Data Aplikacije
![Page 15: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/15.jpg)
Big Data Aplikacija - Cloud VS Bare Metal
• Cena
• Fleksibilnost
• Administracija
• Performanse
• Kontrola
![Page 16: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/16.jpg)
Big Data alati - Ingestion
• Unos podataka u storage sloj
• Unošenje samo validnih podataka
• Unos podataka u realnom vremenu ili ne
• Korišćenje gotovih alata
![Page 17: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/17.jpg)
Big Data alati - Ingestion • Flume
• Sqoop
• Kafka
• Storm
• Kinesis
• …
![Page 18: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/18.jpg)
Big Data alati - Storage
• Hadoop HDFS - velika brzina upisa, write once read many…
• NoSQL Baza - brzo čitanje, operativna baza…
![Page 19: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/19.jpg)
Hadoop
• Hadoop Common
• HDFS
• MapReduce
• YARN
![Page 20: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/20.jpg)
Big Data alati - Analytics • MapReduce (Java, Python, C#…)
• Pig (ETL…)
• Hive, Tajo (Hadoop DWH alat)
• Spark (10x brži MapReduce)
• Storm (Streaming)
• Python, R
• Flink
![Page 21: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/21.jpg)
![Page 22: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/22.jpg)
![Page 23: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/23.jpg)
Big Data greške!• Prebacivanje relacionih baza u Hadoop za potrebe
obrade Hive-om (Hadoop nije zamena za RDBMS).
• Parčanje podataka u više “Data Lake”-ova.
• SQL orijentacija (Big Data analiza je moćnija ako se kombinuje više alata, Hive(SQL), R, Pig, MapReduce…)
• …
![Page 24: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/24.jpg)
Korisni linkovi
• Hadoop Srbija
• Meetup grupa
• Hortonworks
• Big Data Application Architecture Q&A
![Page 25: Big data i arkitektura big data aplikacije meetup](https://reader031.vdocument.in/reader031/viewer/2022021815/5870023f1a28ab427f8b532b/html5/thumbnails/25.jpg)
Big Data i Arhitektura Big Data aplikacije
Darko Marjanovic CEO & Co-Founder @ Things Solver [email protected]
25