what is data?
DESCRIPTION
What is data?. Wietse Dol, LEI-WUR ([email protected]) 13 November 2012, 9.40 – 10.25, C435 Forumgebouw. LEI: Agricultural Economic Research Institute. Part of Wageningen University & Research center (WU R ) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/1.jpg)
What is data?
Wietse Dol, LEI-WUR ([email protected])
13 November 2012, 9.40 – 10.25, C435 Forumgebouw
![Page 2: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/2.jpg)
LEI: Agricultural Economic Research Institute
Part of Wageningen University & Research center (WUR)
Part of the Social Science Group within the WUR
We are the research part of WUR/SSG (advice ministry of Agriculture) in The Hague
Consultancy (applied research): ministries, EU, local government, industry,…
Collecting data (Farm data: FADN), building models and agricultural content specialists
![Page 3: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/3.jpg)
University vs. Research center
University: teaching, publications, new theory and technology
Research center:
●applied work/consultancy
●reusing things from the past (e.g. yearly publications)
●sharing knowledge (how to become a content specialist)/teaching for small groups
●working in groups (different disciplines)
●Working in (inter)national groups with many different disciplines
![Page 4: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/4.jpg)
Wietse Dol
PhD Econometrics
10 years University of Groningen (Econometrics, sampling theory)
18 years LEI (many different departments)
Data and models, i.e. use/reuse and quality, trouble shooter + statistical methods + ICT + user interfacing
Not and IT guy but a researcher (I build software because I use it myself)
Many model projects and user interfaces for models (not only LEI)
Currently: data, data quality, …
![Page 5: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/5.jpg)
![Page 6: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/6.jpg)
Data, lifecycle and data management
http://datalib.edina.ac.uk/mantra/researchdataexplained.html
http://www.dcc.ac.uk/resources/curation-lifecycle-model
http://www.data-archive.ac.uk/create-manage/life-cycle
![Page 7: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/7.jpg)
Data is anything and everything
Research data: collected, observed, or created, for the purpose of analysis to produce and validate original research results.
Anything can become the interest of research …
Data Research
![Page 8: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/8.jpg)
Primary v.s. Secondary data
Primary data: you collect, targeted to answer/validate your questions.
Secondary data: not yours.
• Quality of data
• Meta-information is crucial
• More and more need of secondary data (primary is expensive and takes a lot of time to collect).
![Page 9: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/9.jpg)
Production data
Meta-information: Source, Version, Dimension, Definitions etc. without proper information you use the wrong data
is FR with or without DOM?
Is the production in tons or in Euros.
Does the year start 1-1 and ends 31-12?
What’s the definition of Tomato
Product Country Year Production
Tomato NL 2005 325
Wheat BE 1999 100
Sugar FR 2003 450
![Page 10: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/10.jpg)
DCC Curation Lifecycle Model
![Page 11: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/11.jpg)
CREATE & MANAGE DATA: RESEARCH DATA LIFECYCLE
![Page 12: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/12.jpg)
Data
How to get the data, filter it and store it
Quality checks on the data
How to make it available for others
What scientific actions are done on the data
Curate, preserve, versions, ..
![Page 13: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/13.jpg)
Types of databases according MetaBase
Statistical database
Scientific database
Meta-database
![Page 14: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/14.jpg)
Statistical database
Databases provided by international organizations like EU, FAO and OECD are in general statistical databases:
●Data are stored as they are received
●Data are consistent in their own domain
●No aggregations are made when underlying data are missing
●Not much attention for data checking
![Page 15: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/15.jpg)
Scientific versus Statistical database
Problems with statistical database:
●Different definitions of territories and commodities
●Typing errors
●Missing data
●Break in series
Scientific database:
●Problems solved
●Transparency (original data sources and underlying assumptions are kept)
●Essential for modeling and research
![Page 16: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/16.jpg)
Structural design of a scientific database
Key words for structural design HarDFACTS project IPTS 2007 done by vTI/LEI
●Transparent
●Harmonised
●Complete
●Consistent
Harmonised Database for Agricultural Commodity Time Series
![Page 17: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/17.jpg)
Transparent
Original data from statistical database are stored
Complete and consistent data are stored
Original and completed data can be compared
Calculation procedures are stored and can be repeated
![Page 18: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/18.jpg)
Harmonised
Definition used here is to bring together the different international databases in one framework and to link the data through a unique coding system (keywords are classifications and tree structures)
![Page 19: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/19.jpg)
Complete
Definition used in MetaBase is that an econometric procedures will be proposed to complete the new (time) series in the database.
●Trend estimates
●Interpolation
●Correlation and regression with other variables (e.g. TRAMO: Time series Regression with Arima noise, Missing observations and Outliers)
![Page 20: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/20.jpg)
Consistent
Definition used here is that the inter relationship of the data in the database holds over classifications (time, territories and variables).
![Page 21: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/21.jpg)
MetaBase
![Page 22: What is data?](https://reader035.vdocument.in/reader035/viewer/2022062408/5681394e550346895da0e9ee/html5/thumbnails/22.jpg)
MetaBase
1. many different data sources (e.g. FAO, Eurostat) all in same user-interface (SDMX, NetCDF)
2. find data alternatives using Meta-Information
3. search data content (e.g. oilseed)
4. all content easily available in research software (R/GAMS)
5. recodings, aggregations and concordances are all implemented in GAMS
6. Statistical methods in GAMS and R