weather data analytics using hadoop
TRANSCRIPT
![Page 1: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/1.jpg)
WEATHER DATA ANALYTICS USING
HADOOP
By,
M.S. Najima Begum,
II B.Sc(Computer Science)
Department of Computer Science,
ANJA College, Sivakasi.
![Page 2: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/2.jpg)
About Hadoop• The Apache Hadoop software library is a framework that
allows for the distributed processing of large data sets across
clusters of computers using simple programming models.
• It is designed to scale up from single servers to thousands of
machines, each offering local computation and storage.
![Page 3: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/3.jpg)
AbstractIn this paper, we have used Hadoop platform to analyse
the weather data of various years using MapReduce programs
written in JAVA and we can find the maximum or minimum
temperature every year that can be used for other purposes.
![Page 4: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/4.jpg)
The Paper includes these modules:
• Hadoop Common: The common utilities that support the
other Hadoop modules.
• Hadoop Distributed File System (HDFS™): A distributed
file system that provides high-throughput access to application
data.
• Hadoop MapReduce: A YARN-based system for parallel
processing of large data sets.
![Page 5: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/5.jpg)
Hadoop ArchitectureHadoop Architecture = HDFS Architecture + MAPREDUCE
![Page 6: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/6.jpg)
MapReduce
![Page 7: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/7.jpg)
WEATHER DATA ANALYTICS
Weather prediction is the application of technology to predict
the action of the atmosphere for a given location.
![Page 8: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/8.jpg)
DATA • The data we will use is from the National Climatic Data
Center (ncdc, http://www.nc.noah.gov/).
• The data is stored using a line-oriented ASCII format, in
which each line is a record.
• The format supports a rich set of meteorological elements,
many of which are optional or with variable data lengths.
![Page 9: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/9.jpg)
FINDING MAXIMUM TEMPERATURE IN VARIOUS YEARS
• INPUT0043011990999991950051512004...9999999N9+00221+99999999999...
0043011990999991950051518004...9999999N9-00111+99999999999...
0043012650999991949032412004...0500001N9+01111+99999999999...
0043012650999991949032418004...0500001N9+00781+99999999999...
• INPUT TO MAPPER:(106,0043011990999992010051512004...9999999N9+00221+99999999999...)
(212,0043011990999991950051518004...9999999N9-00111+99999999999...)
(318,00430126509999920119032412004...0500001N9+01111+99999999999..)
(424,0043012650999991949032418004...0500001N9+00781+99999999999...)
![Page 10: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/10.jpg)
INPUT TO REDUCERThe input to the reducer is the output of the mapper
• OUTPUT FROM MAPPER
(2010, 22)
(1950, −11)
(2011, 111)
(1949, 78)
![Page 11: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/11.jpg)
WEATHER DATA
PROCESSING
![Page 12: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/12.jpg)
![Page 13: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/13.jpg)
![Page 14: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/14.jpg)
![Page 15: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/15.jpg)
![Page 16: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/16.jpg)
![Page 17: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/17.jpg)
![Page 18: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/18.jpg)
OUTPUT2010 280
The maximum temperature in the year 2010 is
280.
![Page 19: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/19.jpg)
CONCLUSIONwe have analysed the weather data using big data
environment. The method used in our paper is Hadoop with
mapreducer which processes the sensor data which is stored in
the National Climatic Data Cementum (NCDC)
![Page 20: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/20.jpg)
![Page 21: Weather Data Analytics Using Hadoop](https://reader034.vdocument.in/reader034/viewer/2022042906/58ac1fd41a28abf03a8b4e6d/html5/thumbnails/21.jpg)
Any Queries…