29-31-May-2012 1

HBS 2010 Transmissions: Current Situation

Meeting of the Working Party on Living Conditions 31st May 2012

29-31-May-2012 Data Collection HBS 2010 Wave 2

HBS 2010 Data Transmissions: State-of-Play

Six countries have sent their data. All except one, as SAS datasets. An initial data validation report has been produced for 3 countries and these are being checked prior to being sent to the respective countries for comments.

29-31-May-2012 Data Collection HBS 2010 Wave 3

Problems Encountered in 2005 and in this round

Some files do not have a HA04 identifier

Identifiers containing underscores ( _ )

The Household Consumption Expenditure identifiers HExxA/HExxB/HExxC are being sent in as AHExx/BHExx/CHExx ( i.e. a Prefix is used instead of a Suffix)

Some 'data loss' due to the formats used for codes. For example, codes with a leading zero are being transmitted as a standard numeric format in a SAS dataset, rather than using the format ZN. which conserves the leading zero

Characters have to be converted to Numeric

Conversion of Income & Expenditure data to annual

Missing values need to be replaced with a ‘9’ – “Not Specified”

Weights need to be multiplied by 1000

29-31-May-2012 Data Collection HBS 2010 Wave 4

Problems encountered in 2005 and this round (Continued)

Certain variables have to be interchanged / swapped around

Drop any variables not listed in the HBS/153E/2012/EN document

Assign to certain variables, a default value

Conversion to euro and PPS needed to be done

Some files arrived as RAR-zipped archives

Up to 16 separate text files needed to be read (CSV) in to a SAS dataset

29-31-May-2012 Data Collection HBS 2010 Wave 5

How to Bring some Order to the Chaos? Perhaps, in the future Estat may specify in the existing Word table in the

document HBS/153E/2012/EN an additional column showing the SAS Format to use for the identifier: e.g. $4. / Z2. / 14. / 5.2

In cases where Eurostat brought specific formatting issues to the attention of countries during the 2005 wave, Estat asks these countries to please avoid sending 2010 data in the previously incorrect format

Estat asks countries to send in data files in a similar format & breakdown ( grouping into several data files ) as in 2005. This way we could make use of the 2005 SAS programs for reading data from each country

29-31-May-2012 Data Collection HBS 2010 Wave 6

How to Bring some Order to the Chaos? ( Continued)

Instead of sending out a Microsoft Word table in a document

like HBS/153E/2012/EN showing the list of variables, codes & labels etc. one might consider instead a sort of electronic-codebook, built around a collection of SAS Formats/Informats as well as SAS Macros,

to read in country data;

validate this data against the Formats in the Format catalogue & generate a report indicating if non-valid identifiers or codes are in the files, prior to sending them via eDamis to Eurostat

29-31-May-2012 Data Collection HBS 2010 Wave 7

Timetable for Transmitting HBS 2010 Data?

Finally, those Countries who have yet to send in their data, please

inform Eurostat via E-mail, of the expected delivery dates

Thank you for your attention!