bigbig-data challenges in civil engineering · pdf filetoo big or not too bigbig-data...
TRANSCRIPT
![Page 1: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/1.jpg)
Too big or not too
big…Big-data challenges
in Civil Engineering
applications
José Barateiro
Lisboa, 23rd November, 2015
![Page 2: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/2.jpg)
Who am I?
LNEC | 2
![Page 3: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/3.jpg)
Outline
• What is Big Data?
• Science paradigms
• Big data in Civil Engineering
LNEC | 3
![Page 4: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/4.jpg)
What is Big Data?
LNEC | 4
Source: Gartner, 2015
![Page 5: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/5.jpg)
The V’s
LNEC | 5
Source: Wired, 2013
![Page 6: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/6.jpg)
Three Vs
LNEC | 6
4th Vs: Veracity,
Value…?
![Page 7: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/7.jpg)
What is Big Data?
“Big data is high-volume, high-velocity and high-variety
information assets that demand cost-effective, innovative
forms of information processing for enhanced insight and
decision making.”
LNEC | 7
Source: Gartner, 2015
![Page 8: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/8.jpg)
1st Paradigm
• Thousand years ago:
science was empirical
describing natural
phenomena
LNEC | 8
Source: Jim Gray, 2007
![Page 9: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/9.jpg)
2nd Paradigm
• Last few hundred years:
theoretical branch using models, generalizations and theories
(E.g., Newton, Kepler, Maxwell’s laws and equations)
LNEC | 9
Source: Jim Gray, 2007
![Page 10: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/10.jpg)
• Last few decades:
a computational branch simulating complex phenomena
LNEC | 10
Source: Jim Gray, 2007
3rd Paradigm
Finite element, finite difference,
spectral, boundary, Turbulence
models, Monte-Carlo
simulations, etc.
![Page 11: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/11.jpg)
• Today: data exploration (eScience) unify theory,
experiment, and simulation
Data captured by instruments or generated by simulators
Processed by software
Information/knowledge stored in computer
Scientist analyzes database/files using data management and
statistics
LNEC | 11
Source: Jim Gray, 2007
4th Paradigm
![Page 12: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/12.jpg)
Data-driven science
• Managing and extracting useful information from massive
data sets
Modeling
Data management
Analytics
Visualization
• Data Warehouse and Business Intelligence systems are
data-driven Decision Support Systems
LNEC | 12
![Page 13: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/13.jpg)
Raw data to information
LNEC | 13
![Page 14: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/14.jpg)
Big data challenges and opportunities
• Data quality and data validation?
• Processing
• Organizational culture Making decisions?
Is this our science?
• Actors Data scientists
Business scientists
IT specialists
LNEC | 14
![Page 15: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/15.jpg)
Big data in Civil Engineering• Scenario 1: Static monitoring
LNEC | 15
000 E+0
5 E+6
10 E+6
15 E+6
20 E+6
25 E+6
1930 1940 1950 1960 1970 1980 1990 2000 2010 2020
# S
enso
r m
easu
rem
ents
Year
Total RMD RAD Total RMD RAD
![Page 16: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/16.jpg)
Big data in Civil Engineering
• Scenario 2: Dynamic monitoring
(example: Ponte 25 de Abril)
87 (Sensors) x 500Hz (frequency) * 60 *
60 * 24 = 3.758.400.000 measurements
per day.
Number of sensors will be extended to
approx. 200
Computing correlations: 1.4 x 1019 space
(per day)
LNEC | 16
![Page 17: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/17.jpg)
Raw data to information
LNEC | 17
![Page 18: bigBig-data challenges in Civil Engineering · PDF fileToo big or not too bigBig-data challenges in Civil Engineering applications José Barateiro Lisboa, 23rd November, 2015](https://reader034.vdocument.in/reader034/viewer/2022051723/5ab5607a7f8b9a1a048cc3a8/html5/thumbnails/18.jpg)
LNEC | 18