® st-09-02 open meeing. salzburg september, 2011 coordinating earth and environmental cross-...

®

QUAlity aware VIsualisation for the Global Earth QUAlity aware VIsualisation for the Global Earth Observation system of systemsObservation system of systems

ST-09-02 Open Meeing. Salzburg September, 2011

Coordinating Earth and Environmental cross-Coordinating Earth and Environmental cross-disciplinary projects to promote GEOSS disciplinary projects to promote GEOSS

GEO Label Questionnaire

Choosing possible labels at work

Joan Masó

CREAF

ST-09-02 Salzburg open meeting. September 15th, 2011

GEO Label development phases

• Phases in the development of the GEOLabel discussed in GeoViQua

• Phase I – will be looking at what information people want without giving them actual

examples (trying not to bias them);

• Phase II – will present some examples and, under controlled conditions and in a well-

managed and structured way, elicit their feedback on some examples;

• Phase III – will create physical prototypes which will be used in a human subject

study.


GEO Label phase II

• Phase I – will be looking at what information people want without giving them actual

examples (trying not to bias them).– We have been moving back and forth for a while. We have some

background that we could apply to the next phase

• Phase II – will present some examples and, under controlled conditions and in a well-

managed and structured way, elicit their feedback on some examples.– This presentation and the GEO Exhibition questionnaire is a contribution

to this phase.

• Phase III – will create physical prototypes which will be used in a human subject

study.


About the questionnaire

• Objectives of the questionnaire– To help understanding if people do really care about having a label/seal on

the information– Which GEO Label approach do they want/need– Which elements are expected in those labels– Implicitly: How important quality information about data is for them

• really important aspect for GeoViQua

• Number of questions – ~10 direct questions. – ~3 extra control questions to test people understanding

• This meeting will serve to gather ideas on how to improve the questions and to detect important missing questions.


Common for all GEO Labels

• For all labels shown:– Share the same look. An invented "GEOSS" logo.

• It is a neutral look and is used repetitively because we do not want to condition answers to how the label looks like but to the additional elements it has or the real meaning of them.

– Remember that we are on Phase II so we are NOT looking for the final content and design of the label but to the user preferences

– All labels could be written on paper or used on the web. • On the web, elements could be linked to extra information

• This will be Phase III


The questionnaire

• QUESTION 1: Introduction• QUESTION 2: Product integrate in GEOSS label• QUESTION 3: Quality, relevance and rate label• QUESTION 4: Spatial, thematic, date, provider and rate label• QUESTION 5: Choose one dataset only considering the label• QUESTION 6: Quantitative bar label• QUESTION 7: Descriptive label• QUESTION 8: Certification label• QUESTION 9: View and get the data label• QUESTION 10: Inter-comparison of labels• QUESTION 11: Full label• QUESTION 12: Final remarks


Question 1:Introduction

• Identify yourself as one of the following user typea) GEO committee memberb) Data provider with some dataset in the GCIc) Data provider not part of GEOSSd) Private sector data usere) Governmental data userf) Researcher data userg) Other data userh) Neither of those

• Do you read the meta-information of the dataset you use?a) Yes, only use data with enough meta-information.b) If it is available but I will use the data anywayc) No.


Question 2: Product integrate in GEOSS label

The following label can be used for a GEOSS contributor organization next to a product documentation or website data access.

• When you see a label like this, what do you think?

a) I do not know about GEOSS. I want to know more.b) I know about GEOSS so their data has to be relevant to me.c) This is a GEOSS product. So what?d) Another useless icon that confuses mee) I do not understand the label.

• When the previous label is found on the GEOSS contributor website it links to the GEOPortal or CSR record for this component or services integrated in GEOSS.a) It is useful because I can easily access other equivalent resource from other provider and

compare themb) It is interesting to know but it does not help me.c) The information in GEOSS could be out of date or incomplete. I prefer the original source.d) I prefer to stay in the contributor website and not get lost on links.


Question 3: Quality, relevance and rate label

The following label can be found in the GEOPortal or in a GEOSS contributor organization, in their documentation or in their website.

• When you see a label like this, what do you think? (check all that apply)a) It is an easy way to compare datasetsb) It seems relevant to some application areasc) People like this resource. I want to know more about itd) I do not understand the small blue symbolse) I do not care about user rates or what the people likef) Quality can not be summarized in a single numberg) Too complicatedh) There are important aspects missing like scale, extent, time.


Question 4: Spatial, thematic, date, provider and rate label

We have added more information: It is more explicit about quality.

• What is your opinion about this (compared with the last one)?

a) Additions are very important

b) Additions are more informative

c) Additions are irrelevant

d) Additions are confusing

e) Too much information

f) I do not understand the label


Question 5: Choose one dataset only considering the label

• Which one do you choose?a) "A" because I work with global data

b) "A" because it has more categories

c) "B" because it has more thematic quality

d) "B" because it has more spatial quality

e) I need more data to decide

f) I'm completely overwhelmed by the amount of information.

g) I do not understand the labels


Question 6: Quantitative bar label

We have simplified the information in the GEO Label to a single indicator you do not know how it is generated.

• When you see a label like this, what do you think?a) That is what I like. Easy to compare an simple

b) I do not know what the value represents. I need more information

c) This is a GEOSS product

d) Another useless icon that confuses me


Question 7: Descriptive label

This GEO Label shows that the product has data quality information (does not gives us a value), provenance information and possible use cases documented. A click on the text will get more information.

• When you see a label like this, what do you think?a) That is what I like. I know that the data has been carefully documentedb) I would have preferred to see real values in the labelc) Another useles icon that confuses med) I do not understand the label


Question 8: Certification label

This GEO Label certifies that the product has followed strict procedures and protocols that GEOSS requires to get this label

• When you see a label like this, what do you think?a) That is what I like. I know that the data has been carefully processed and

documented

b) I do not care about certification. This is only a bureaucratic step

c) Another useless icon that confuses me

d) I do not understand the label


Question 9:View and get the data label

This GEO Label guides the user to viewing the data, the quality information and also to download the data.

• When you see a label like this, what do you think?a) Now I know that I can download the data and test it myself

b) Now I know that data will not cost me money so do not care about quality.

c) Knowing the formats is important. Some formats do not contain important metadata or the data is degraded

d) All data in GEOSS can be downloaded for free. Isn't it?

e) Another useless icon that confuses me

f) I do not understand the need for this label


Question 10: Inter-comparison of labels

• You have the possibility to choose a particular GEO Label design

• Please choose one of these labelsa) Ab) Bc) Cd) De) None of them are usefulf) I do not careg) I do not understand some of these labels

• Why do you choose a particular one?a) More informativeb) Easy to comparec) A gateway to more informationd) Simplee) I do not understand the others

A B

C D


Question 11:Full label

This GEO Label combines many of the aspects previously shown in a single label.

• What do you think? (check all that apply)a) Some important aspects are still missingb) All information is important to mec) The box with quality rates, scale and dates is not

important to med) The box with provenance info and use cases is not

important to mee) The box with "view and download" is not

important to mef) Certification is not important to meg) Too much information.h) Some aspects are redundanti) I do not understand the label


Question 12: Final remarks

• Final remarks. (check all that apply)a) The idea of the GEO Label is important to compare products.b) The label has to have the GEO logo and not the GEOSS logo.c) I prefer numerical labelsd) I prefer simple labels with only a couple of parameterse) I prefer simple labels with as much information as possiblef) I do not understand the need for the GEO Labelg) Any label should link to a comprehensive quality reporth) Any label should link users comments list and a representation translated

by system into “rate”)i) Any label should include any available certificate attached to the selected

information source and link to a explanation.j) The existence of usage information ( i.e. a system derived parameter

based exclusively on users comments) should be linked to the GEO Label.


The role of GeoViQua, EGIDA and STC

• GeoViQua may conduct the questionnaire and test future prototypes

• GEO STC is formally endorsing this approach (thanks!) and will help to promote it

• GEO STC and EGIDA may contribute – Moving the questions to final – Announcing/emailing the questionnaire – Inferring conclusions from the results analysis


The process

• October 2011– Have the questions ready to distribute in

the GEO exhibition– Contact EC and request space and time

to do this

• November 2011– Do the questionnaire in the EC booth

• December 2011– Leave open the questionnaire for 3 more

weeks

• January 2012– Analyze the results

• Aston University in GeoViQua could do this

• February 2012– Report back to the relevant committee,

board, task…

• March 2012– Start Phase III during 2012:

• development and test of selected alternatives

• October? 2012– Deliver final test results to the GEO

Plenary

• What if replies result in users do not wanting any ‘label’?


Discussion

• Thanks• [email protected]


Possible extra design

• This are the 4 checks that are new on the CSR. Producer has to mark the relevant concepts:

[x] GEOSS Data Core [x] GEOSS No Monetary Charge [x] GEOSS User Registration [x] GEOSS Attribution

• This gives use 4 flags that can be included in the GEOLabel


First approach: Encourage to contribute

• Aim: Encourage scientists, researchers, and others to contribute their data and systems to GEOSS by offering an accepted voluntary label that provides recognition that their contribution is valued by the GEO community.

• From: STCGEOlabel_June2010.doc


First Approach: Encourage to contribute

Contributor documentation,website, etc

Contributor documentation,website, etc

GEOSS

Contributes to the CSR and the GCI

Contributor to GEOSS (Scientist, data provider,

or other) Gets a GEOLabel

CSR moderator

Contribution aproved

Publicites the GEO Label along with the

data

EO user

Reads the GEO Label and knows

about GEOSS

Uses GEOSS

GEO Label shall contain references to GCI and CSR records

The registration confirmation shall contain a GEO Label as HTML code and text and suggestions on how to embed it in websites, metadata, etc

«precedes»

«precedes»

«invokes»


Second approach: Differentiate components

• Aim: Differentiate components, data and products delivered through GEOSS and provide a “trusted brand” to GEOSS users; member governments may base their decisions on data/products of such contributions.

• From: STCGEOlabel_June2010.doc


Second approach: Differentiate components

GEOSS

Contributes to the CSR and the GCI

Contributor to GEOSS (Scientist, data provider, or

other)

Gets a GEOLabel

CSR moderator

EO user

Provides well documented quality

information

Searchs and picks a dataset

Decides if the dataset fits for the

purpose

Uses the dataset

Returns feedback

Explores the GEO Label data

Measures quality (e.g.

cal/val)

Documents processes and

traceability

Certification

Feedback is incorporated to the

GEO Label

CSR registration shall include quality information, traceability and certification

GEO Label shall be incorporated in the GEOPortal results.

GEO Label shall be build directly from information from the registration process

GEO Label content shall be easily comparable

GEO Portal shall collect feedback

Feedback shall be connected back to the GCI/CSR for integration into the GEO Label

Certification Body

«include»

«include»

«include»

«extend»

«extend»

«invokes»

«extend»

«extend»

«extend»«precedes»

® st-09-02 open meeing. salzburg september, 2011 coordinating earth and environmental cross-...

Documents

data labelquestion

rate labelquestion

geoss labelquestion

salzburg open meeting

descriptive labelquestion

certification labelquestion

information people

geo label phase iiphase