the future marriage of big data and railroad engineering€¦ · the future marriage of big data...
TRANSCRIPT
![Page 1: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/1.jpg)
The Future Marriage of Big Dataand Railroad Engineering
Nii Attoh-Okine, PhD, P.E.Professor
Department of Civil and Environmental Engineering
1 / 61
![Page 2: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/2.jpg)
”Big Data” in Railroading
”The challenge is utilizing Big Data to improve
efficiency, reliability, velocity, productivity, and
safety.” - Railway Age. August, 2014.
2 / 61
![Page 3: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/3.jpg)
Big Data Revolution is in Motion in the Railway Industry
• Given recent evolution in the railway industry,remaining competitive requires an ability toprocess a growing flow of data.
• We strongly believe that the keys to success
are twofold when it comes to designing a
successful Big Data strategy:
• A structured and robust framework.• A continuous upgrade of hardware and
infrastructure to stick to volume of dataand complexity of analyses.
• Big Data represents business opportunities formajor players in the railway industry...
• . . . from rethinking client relationship tocrafting tomorrow’s operations and riskmanagement.
3 / 61
![Page 4: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/4.jpg)
Big Data Requires Investing in New Forms of DataProcessing
4 / 61
![Page 5: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/5.jpg)
Big Data and its 5V Properties
5 / 61
![Page 6: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/6.jpg)
Learning for Big Data
14 / 61
![Page 7: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/7.jpg)
A Brief History of Big Data with Major Milestones
16 / 61
![Page 8: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/8.jpg)
Traditional Analysis
• As data sets gets larger, complexity of false findings growexponentially.
• Serious statistical skill is required to avoid being misled.
• Large N, Small P.
• Small N, Large P.• Use of statistical significance is inappropriate.
20 / 61
![Page 9: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/9.jpg)
Big N
21 / 61
![Page 10: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/10.jpg)
Bias and Variance in Statistics
• Bias is error that cannot be corrected by repeatedexperiments.
• Bias-variance decomposition states that expected squarederror is equal to the bias plus the random error.
• You can reduce the variance but not the bias.
• True value of the parameter is a constant.
• Experimental estimate is a probabilistic variable.
• Bias is the systematic or average difference between these twovariables and variance is the probabilistic component.
22 / 61
![Page 11: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/11.jpg)
Tensors
Multiway Data Analysis
31 / 61
![Page 12: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/12.jpg)
Beyond Correlation: Causation
58 / 61
![Page 13: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/13.jpg)
Massive Amount of Data (Tank Safety)
59 / 61
![Page 14: The Future Marriage of Big Data and Railroad Engineering€¦ · The Future Marriage of Big Data and Railroad Engineering Nii Attoh-Okine, PhD, P.E. Professor Department of Civil](https://reader033.vdocument.in/reader033/viewer/2022060401/5f0e18247e708231d43d95af/html5/thumbnails/14.jpg)
Traditional analytical techniques are inadequate inanalyzing and drawing conclusions
60 / 61