wp7 multi domains...2016-11-02 2016-11-30 face to face meeting brussels 2016-11-17 2016-11-18 1.3....
Post on 22-Jun-2020
0 Views
Preview:
TRANSCRIPT
WP7 MULTI DOMAINS
WP7 Multi domains
WP7 Multi domains
1. Population
2. Tourism/border crossing
3. Agriculture
• Regional statistical office in Poznań
• Regional statistical office in
Bydgoszcz
Population
• Regional statistical office in Rzeszów
• Department of Social
Research
Tourism/
border crossing
• Department of Agriculture
• Regional statistical office in Olsztyn
AGRICULTURE
Country leaders of methodology
WP7 TEAM
Anna Nowicka Leader cooperation
Weronika Jaźwińska
PARTNERS
Janusz Dygaszewicz Project Manager of Polish work
John Sheridan Piet Daas Nigel Swier
may enrich statistical output in domains:
Aim of WP7 is to find out how
a combination of:
Big Data sources
administrative data
statistical data
WP7 - Future perspectives
Suggest pilots and domains with successful implementation potential for further elaboration
in the second wave of pilots in 2018
WP 7 Time frame
February 2016
February 2017 Maj 2018
FPA (Framework Partnership Agreement) 2 YEARS + 5 MONTHS (JAN. 2016 – MAY. 2018)
SGA-1 SGA-2
Milestones and deliverables(SGA-1)
Milestone 1.
Progress and technical report of
internal WP-meeting;
by M4
Milestone 2.
List of availability
big data sources in
the domain(s);
by M8
Milestone 3. Recommendation for using two or three
big data sources in the domain(s); by
M13
DELIVERABLE
THE PARTIAL REPORT FOR EACH DOMAIN CONTAINING BASIC INFORMATION ON:
The data access (with legal and privacy aspects)
The data quality issues
The methodology (focus also on combining data)
The technical aspects
by M14
Milestones and deliverables(SGA-2)
Milestone 3. Combining data
analyze;
Milestone 4. List of pilots and domains with successful
implementation potential for further elaboration in the
second wave of pilots in 2018;
Deliverable
THE GENERAL REPORT FOR DOMAINS including:
The data access (with legal and privacy aspects)
The data quality issues
The methodology (focus also on combining data)
The technical aspects
WP 7 Work plan SGA-1
Data access
Data feasibility
Data combination(SGA-2)
Summary plus future perspectives (SGA-2)
WP 7 Work plan SGA-1 TASK FROM TO
Planning and organization 2016-02-01 2016-03-31
Task 1. Data availability/Data inventory 2016-03-01 2016-09-30
1. Identify Big Data sources taking into account sustainability and availability in
several countries. 2016-03-01 2016-07-15
1.1. Establishing an inventory of these sources by: 2016-03-01 2016-06-08
1.1.1. Brainstorming - a review of potential sources, including the
2015 UNSD Survey on the Use of Big Data for Official Statistics 2016-03-01 2016-05-19
1.1.2. Preparation of a questionnaire with questions about the
sources used by the project participants 2016-04-29 2016-05-30
DONE
1.1.3. Sending the questionnaire to participants 2016-05-30 2016-05-31
1.1.4. Gathering answers and preparation for analysis 2016-05-31 2016-06-08
MILESTONE 1: Progress and technical report of internal WP-meeting 2016-05-24 2016-05-31
1.2. Assessment of the possibility of using sources for Big Data analysis in
the domains of population, tourism/border crossings, agriculture 2016-06-01 2016-06-24 STARTED
Face to face meeting Tallinn 2016-06-13 2016-06-15
WP6&WP7 internal meeting in Warsaw 2016-06-28 2016-06-30
1.3. Build the list of potential sources 2016-06-24 2016-07-15
2. Identify which results or new products from the source-oriented pilots may
contribute to these domains 2016-07-15 2016-08-26
2.1. Match the sources from the list of potential sources to each domain 2016-07-15 2016-07-22
2.2. Preliminary analysis of possibility for using sources to each domain
(legal aspects, availability, methodology, IT, quality) 2016-07-22 2016-08-12
2.3. Build the list of exploitable sources for each domain 2016-08-12 2016-08-26
3. Describe the added value of delivered linkage between these sources to
current statistics 2016-08-26 2016-09-30
3.1. Analyze the list of exploitable sources for each domain 2016-08-26 2016-09-09
3.2. Prepare the map of linkages between Big Data sources (e.g which
aspect of one data source can be used in several domains) 2016-09-09 2016-09-20
3.3. Describe the added value for each domain 2016-09-20 2016-09-30
MILESTONE 2: List of available Big Data sources in the domain(s) 2016-09-30 2016-09-30
WP 7 Work plan SGA-1 TASK FROM TO
Task 2. Data feasibility 2016-10-01 2017-02-28
1. Carry out explorative analyses in order to apply two or three Big Data sources in the
domain of population, tourism / border crossings or agriculture 2016-10-01 2016-12-30
1.1. Selection the most value big data sources for each domain (evaluation of the
legal aspects, availability, methodology, IT, quality) 2016-10-01 2016-10-31
1.2. Analying results. 2016-11-02 2016-11-30
Face to face meeting Brussels 2016-11-17 2016-11-18
1.3. Preliminary assessment of the usefulness - developing the assessment
factors. 2016-12-01 2016-12-30
2. Selection and recommendation of two or three big data sources for using in the domain
of population, tourism / border crossings, agriculture. 2017-01-02 2017-02-28
2.1. Preparing the SWOT analysis (positive and negative factors of using several
sources) 2017-01-02 2017-01-16
2.2. Recommendation of the most important and useful sources. 2017-01-16 2017-01-31
MILESTONE 3: Recommendation for using two or three Big Data sources in the
domain(s) 2017-01-31 2017-01-31
SGA-1 deliverables - report for each domain 2017-02-01 2017-02-28
Submission to the Review Board 2017-02-01 2017-02-01
Document analysis by the Review Board 2017-02-01 2017-02-15
Overview of comments and document analysis 2017-02-15 2017-02-28
Send the document to the Coordinator 2017-02-28 2017-02-28
Face to face meeting Sofia + workshop 2017-02-16 2017-02-16
Multi domains internal meeting
• Work package 7 has only one face to face internal meeting during all SGA-1 duration and therefore the meeting should be planed in the most suitable time. Under the Agreement, it is required that it will take place in the second quarter of 2016.
• For this reason, it was established that it is not necessary to organize a kick-off meeting now. At the end of June, it will be more efficient to build first list of big data sources for each domain. Sufficient means of cooperation between the participants of the package will be mails exchange and a meeting on Webex.
• Common WP6 & WP7 face to face meeting will take place on 28-30 of June in Warsaw. • FIRST DAY – common
• SECOND DAY – each WP separetly
• THIRD DAY – common
Stage: End of agreeing detailed agenda
Multi domains internal meeting
• All WP7 participants was represented by the following experts:
• The meeting was carried out via Webex. It was lead by Anna Nowicka in cooperation with Weronika Jaźwińska from Central Statistical Office in Poland.
John Sheridan IE √
Nigel Swier UK √
Piet Daas NL √
Anna Nowicka (Report)
Weronika Jaźwińska (Report)
PL
PL √
√
Multi domains internal meeting took place on 24th of May via Webex
Multi domains internal meeting
The meeting was carried out according to agenda: Nr Issues
1 Opening and agenda (Anna) 5 min
2 Information on Wiki access (WP7) (All) 5 min
3 Information on state of the art WP7 (Anna&Weronika) part 1 15 min
- Brainstorming – why we do it?
- The results of brainstorming on country level
- The results of brainstorming on international level
- Conclusion
4 Discussion and proposal on brainstorming (All) approx. 5-10 min
5 Information on state of the art WP7 (Anna&Weronika) part 2 15 min
- Questionnaire:
- the context
- the construction of the questionnaire
- the questions
- the terms of realization
6 Discussion and proposal on the questionnaire (All) approx. 5-10 min
7 Preparation to the Tallinn meeting (Anna) 5 min
8 Action plan – next steps (Anna) 5 min
9 Milestone 1: Progress & technical report of internal WP-meeting 5 min
Multi domains internal Webex
meeting – results for Wiki page
Mapping between sources and domains could be developed as a section within the ESSNet Wiki.
The idea is that we could have a list of sources, we could then click on a source,
which would then send us to page which listed relevant use cases. We could then click a use case, which would take us to a webpage containing more detailed information, including links to published research and contact information for any projects investigating these areas.
Similarly, we could have a list of domain areas, which would direct the user to
relevant use cases and the sources needed to support them. So basically, the idea would be to develop a corpus of what we known about big data sources and their application within official statistics which could be added to over time.
The Wiki would provide a platform for making this open for others to contribute to.
Multi domains internal Webex
meeting – results for Wiki page
-for a particular source, e.g. mobile phone data, a page listing the existing use cases, in which each item in this list links to a page further explaining this particular use case;
- and for a particular area, e.g. tourism statistics, a similar list of use cases linking to use cases (some of which are linking to a mobile phone use case also in the list of sources above).
All participants agreed on this way of organizing big data sources and decided to use
“The classification of types of big data” created by UNECE as a structure. As a structure for the domain participants agreed to “Classification of Statistical Activities”.
WP7 MILESTONE 1
MILESTONE 1
Progress and technical report of internal WP-meeting based on:
BRAINSTORMING RESULTS QUESTIONNAIRE PREPARATION
Multi domains internal meeting took place on 24th of May
• The aim of the task 1 in the Work Package 7 is to identify the sources of big data (including their durability and availability in different countries), assessment of the possibility of using selected sources for data analysis in the areas of population, agriculture and tourism and to identify which of the results or new products from pilot studies may be useful in these areas.
• One of the methods to build a preliminary extensive list of potential sources is conducting a brainstorming session. This method is characterized by the use of intuition to problem solving and teamwork, the advantages are: higher efficiency of a group than individuals, better detecting errors in a group, greater objectification of results in a group, greater creativity, greater degree of humanization of work in a group, learning cooperation and collaboration of the group members.
Brainstorming
Why did we do the brainstorm?
to create the widest possible range of big data sources (a cafeteria
possible sources of data that public statistics could use for new
developments or supplement existing ones, so that in the later
stages these sources can be verified from different points of view and
gradually part of them will be eliminated as the least useful.
to analyze as many as possible use cases of using big data sources
to take into account the most popular source
Big Data is a new phenomenon we should take into account that the potential of each source may still change.
Brainstorming - organization
Poland organized several brainstorming sessions:
The first on a country level, in the classic form, took
place during a meeting on 26 of April 2016
1 Electronic
brainstorming at country level 2
Electronic brainstorming
at international level (by WP7`
participants)
3
Brainstorming Created list is sorted by different levels of typology and then aggregated by the convergence of records: a. The original number of records: 122 b. Number of records on the Aggregation: 86 (including 4 without categorization). Eventually, the list was narrowed to those that initially can be considered useful in the context
of their further use (38 items) in official statistics.
Polish brainstorm
• Sources indicated in the brainstorm sessions in Poland were considered from the perspective of data. Records suggest that the overall potential of the source and the majority does not imply a clear use cases.
International brainstorm
• The material obtained from the United Kingdom, the Netherlands and Ireland was presented, but in a more compact manner, it means that the teams were limited to sources that have been identified and based on them were carried out some pilot projects or such sources are in the course of diagnosis (most have additional description and a link to the report).
• The results are presented in terms of source - use case.
Additionally, a further step in task 1 in the work package, that is, mapping sources to the areas of statistics (including, in most cases, the three set out in the WP7).
Brainstorming - results The most common source belong to the category Machine-generated data (Automated Systems): 38, of which the vast majority is a source based on various sensors / sensors: 30.
Often, the source pointed out partially processed (2. Process-mediated date): 28, including those which are administrators of tourist offices or institutions 14.
No sources which can be classified into types:
1300 - Personal documents, 1800 - User-generated maps,
But: Information on the movement of population on the basis of data from an application like Google Fit, etc. Endomondo suggests the use of log files, although such applications also provide the ability to create and spread on social networking personalized maps
Type 3131-3135 - any of the types of satellite data were specified.
from BRAINSTORMING
to the QUESTIONNAIRE
Why did WP7 carry out the questionnaire?
to find out more about the possibilities of technical, methodological quality,
access in different countries
recommending the source to the pilots after 2018 to know the plans for big data of different
countries
questionnaire was sent to countries outside the
FPA (but EU country), because we recommend beyond the period of its
duration
recognize the obstacles in the using of big data sources
QUESTIONNAIRE - preparation
1. Prepare draft questions for the questionnaire by Poland
2. Consultation and comments WP6 (in particular questions about the combining)
3. Testing of the questionnaire by the UK
4. Sharing a link to the questionnaire
QUESTIONNAIRE LINK: http://interankiety.pl/i/ma6JA9a3
Questionnaire - results Greece
BULGARIA
Belgium
Netherlands
Denmark
Hungary
Slovakia
Spain
UK
Italy
Germany
Ireland
italy
Estonia
Lithuania
Sweden
Austria
Finland
Poland
Each questionnaire is important to us. Thank you very much for completing it! WP7 has received 19 completed questionnaire.
Questionnaire - results
Now we are at the stage of analyzing the results.
Questionnaire - results
Questionnaire - results
WP7 plan for SGA-2
TASK 3. Data combination The experimental work (if practical work would be possible/if not it would be theoretical considerations including consultation with practice ex. data hub)
Data collection Data preparation Data analysis
Describe practical, technical and methodological aspects when combining big data outputs in the statistical system. For example, differences in definition, populations and volatility etc. Provide first answers on quality issues when combining big data with traditional outputs. Provide answers on the question whether micro-data have to be used when combining big data estimates with traditional outputs or data at aggregated level can be considered.
Analysis advantage and disadvantages of combining data Preparing the list of criteria for combining data Milestones 3. Combining data analyze; by M19
WP7 plan for SGA-2 TASK 4. Summary plus future perspectives
Suggest pilots and domains with successful implementation potential for further elaboration in the second wave of pilots in 2018.
Recommendation on legal aspects; Recommendation on availability; Recommendation on methodology; Recommendation on quality; Recommendation on technical requirements.
Conclusion
Milestones 4. List of potential pilots and domains with successful implementation potential for further elaboration in the second wave of pilots in 2018. by M23
M24 Deliverable - THE GENERAL REPORT FOR EACH/SEVERAL DOMAIN including:
The data access (with legal and privacy aspects) The data quality issues The methodology (focus also on combining data) The technical aspects
Sandbox
WP7 access to the Sandbox is not necessary at this stage of work
Anna Nowicka Central Statistical Office of Poland
and thank all partners for their contribution to the work!
top related