editing process and its quality regarding design and ... · pdf file － the editing process...

Editing process and its quality regarding design and production phases using process metadata and calculation modules

Pauli Ollila 15 September 2015 Work Session on Statistical Data Editing Topic (iv): Evaluation and feedback

Contents － Different phases of creating and using the editing strategy － GSBPM and the phases of creating and using the editing strategy － Generic E&I-process and process step definitions from GSDEM － Designing editing strategy before production － One process step with functions, modules and production actions － Metadata in editing process －  Planning, production and development tasks －  Problematic situations and unexpected tasks in production － Development tasks at different levels － Conclusions (in a bit straightforward manner)

15 September 2015 Pauli Ollila 2

DIFFERENT PHASES OF CREATING AND USING THE EDITING STRATEGY

3 15 September 2015 Pauli Ollila

1) DESIGNING THE EDITING STRATEGY The phase includes a variety of different planning and decision actions for editing, ranging from the construction of the process flow to selecting some principles to carry out edit rules. 2) CREATING THE REALIZATION SYSTEM OF THE STRATEGY The phase includes the IT choices for carrying out the editing strategy, both containing solutions for different parts in the editing strategy and possibly decisions for interactions between different IT environments and data structures. In principle this phase could also include decisions about non-IT practices belonging to the strategy, e.g. paper questionnaire studies. There might be various ways to carry out the methods, but the corresponding IT solutions for the methods are denoted here as modules. 3) TESTING THE STRATEGY The phase includes operations in the realization (IT) system of the editing strategy with specific test data sets (unedited, edited), in practice mostly from earlier rounds of the production of statistics. The testing procedure may have a systematic structure, but known statistic-dependent problematic areas can also be studied. 4) APPLYING THE STRATEGY IN PRODUCTION The phase contains the implementation of the editing strategy with chosen methods and parameterization in the corresponding IT system with data set(s) to be edited in the production process of statistics. Note that editing in production can also happen in a collection phase, i.e. when we are acquiring the data gradually.

The editing process “contains a number of activities or tasks that aim to assess the plausibility of the data, identify potential problems and perform certain selected actions that intend to remedy the identified problems”. The editing strategy covers in addition to the editing process also many other things connected to the operations dealing with the editing process in different contexts.

1)  Designing  the   edi-ng  strategy    

2)  Crea-ng  the  realiza-on   system  of  the  strategy  

3)  Tes-ng   the  strategy    

    4)  Applying  the  strategy   in  produc-on  

(4.3  for  edi,ng  during   data  acquisi,on)        

GSBPM  AND  THE  PHASES  OF  CREATING  AND  USING  THE  EDITING   STRATEGY  

4  

Generic E&I process from GSDEM

5 15 September 2015 Pauli Ollila

Process step definitions in GSDEM

－  The editing process can be structured by splitting it up into sub-processes, called process steps, and a process flow that describes the navigation among the process steps during execution.

－  An operational data editing process usually contains a considerable number of functions with specified methods that are executed in an organised way. The function types are Review, Selection and Amendment. A function is an instance of one of the three function types that serves a specific purpose in the chain of activities that leads to the edited data.

－  Data editing functions specify what action is to be performed in terms of its purpose, but not how it is performed. The latter is specified by the process method. For the purposes of production some computerized methods are parameterized in order to define the process exactly. Methods can be interactive as well.

6 15 September 2015 Pauli Ollila

The connection between four phases of creating and using editing strategy and GSDEM is emphasized via terms process flow, process steps, functions, methods and parameterization.

PROCESS  FLOW  (here  without  order)            

PROCESS   STEP  

PROCESS  STEP  (F)   with  func-ons  defined  

FUNCTION   FUNCTION  

FUNCTION  

A  SET  OF   METHODS  

FOR   FUNCTIONS  

PROCESS   STEP  

PROCESS   STEP  

PROCESS   STEP  

PROCESS   STEP  

PROCESS  FLOW    

PROCESS  STEP  (MP)     with  methods  and  parameters  

PROCESS  STEP  (M)   with  methods  decided  

METHOD   METHOD  

METHOD  

PROCESS   STEP  

PROCESS   STEP  

PROCESS   STEP  

PROCESS   STEP  

PROCESS   STEP  

METHOD     (with  param.)  

PLANNING   DECIDING   PROCESS FLOW WITH IT SOLUTIONS

PROCESS STEP (IT SOLUTION) with method decision and parameter

definition possibility

PROCESS STEP IT

METHOD MODULE

DEVELOPING IT SYSTEM

METHOD     (with  param.)  

METHOD     (with  param.)  

PROCESS STEP IT

PROCESS STEP IT

PROCESS STEP IT

PROCESS STEP IT

METHOD MODULE

METHOD MODULE

NEEDED  IN  PLANNING:  Process   flow,  Process  steps,  Func-ons  

NEEDED  IN  IT  SOLUTIONS:  Modules  for   Methods,  Parameteriza-ons  for  Modules  

NEEDED  IN  DECIDING:  Methods  for   Func,ons,  Parameteriza,ons  for  Methods  

A SET OF MODULES

FOR METHODS

DESIGNING  EDITING  STRATEGY  BEFORE  PRODUCTION  (ideal  situa1on)  

ONE  PROCESS  STEP  WITH  FUNCTIONS,  MODULES  AND  PRODUCTION   ACTIONS    

15  September  2015   Pauli  Ollila   8  

According  to  the  edi-ng  process  it   tells  what  needs  to  be  done  and  to   be  obtained  as  results.  

Includes  user  ac-ons  within  the  IT   environment:  parameter  defini-ons,   interac-ve  opera-ons,  decisions,   possibly  prepara-ons  and  updates.    

Provides  modules  to  carry  out  the  tasks   given  in  planning  with  chosen  methods.   Supported  by  a  metadata  system  for  user   defini-ons  and  process-­‐based  parameter   produc-on  with  metadata  for  monitoring.  

15  September  2015   Pauli  Ollila   9  

INPUT  METADATA  (needed  for  the  ediAng  process,  i.e.  referenAal  metadata)   -­‐  User-­‐made  metadata  at  the  produc-on  stage  (e.g.  parameter  defini-ons,  rules  for  data  treatment)   -­‐  Imported  metadata  (predefined  metadata  with  e.g.  parameters,  auxiliary  data*,  other  non-­‐sta-s-cal   data*)  

-­‐  Process-­‐made  metadata  (mainly  func-on  indicators  and  derived  variables  for  further  use)   OUTPUT  METADATA  (produced  in  the  ediAng  process,  i.e.  paradata)   -­‐  Func-on  indicator  metadata  (may  be  used  as  subsequent  input  metadata  but  also  informa-on  about   edi-ng  history  of  a  unit)  

-­‐  Process  metrics  metadata  (metrics  describing  the  process  and  its  quality)   -­‐  Process  informa-on  metadata  (what  happens,  what  is  used,  excep-onal  situa-ons,  warnings  …)

METADATA  IN  EDITING  PROCESS   NOTE:  Subclassifica,on  not  from  the  task  team  work,  this  is  only  some  general  descrip,on  (not  ”official”)  

STEERING  THE  PROCESS  mainly  via  metadata  for  parameteriza-on  (e.g.  edit  rules,  limits,   method-­‐dependent  parameters)  with  some  decisions  and  defini-ons  during  the  process.     MONITORING  AND  EVALUATING  THE  PROCESS  mainly  via  output  metadata  of  process  indicators   and  process  informa-on  together  with  experiences  and  special  studies  some-mes.  

Planning,  producAon  and  development  tasks