a fieldwork management and monitoring system … fmms_technical...h a fieldwork management and...

h

A Fieldwork Management and Monitoring System (FMMS) for

the European Social Survey

Documentation to accompany release of prototype tool (v1.0)

28th February 2017

Revision History

Name Date Reason For Changes Version

28.02.17 - v1.0

www.seriss.eu GA No 654221 2

Contents

1. Introduction ......................................................................................................................................... 41.1 Purpose of the tool ................................................................................................................................. 4 1.2 Current status ........................................................................................................................................... 4 1.3 This document .......................................................................................................................................... 4 1.4 Further information ................................................................................................................................ 5 1.5 FMMS Definitions, Acronyms and Abbreviations ..................................................................... 5 1.6 Acknowledgements ................................................................................................................................ 6

2. Software Requirements Specification ..................................................................................... 62.1 General Requirements .......................................................................................................................... 6

2.1.1 Software Perspective ........................................................................................................................................ 6 2.1.2 Software Functions ............................................................................................................................................ 8 2.1.3 User Roles and Characteristics ................................................................................................................... 8 2.1.4 Operating Environment ................................................................................................................................. 10 2.1.5 Design and Implementation Constraints .............................................................................................. 10 2.1.6 User Documentation ....................................................................................................................................... 10 2.1.7 Assumptions and Dependencies ............................................................................................................. 10

2.2 External Interface Requirements................................................................................................... 11 2.2.1 User Interfaces .................................................................................................................................................. 11 2.2.2 Hardware Interfaces ....................................................................................................................................... 11 2.2.3 Software Interfaces ......................................................................................................................................... 11 2.2.4 Communication interfaces ........................................................................................................................... 11

2.3 Other Non-functional Requirements ........................................................................................... 12 2.3.1 Performance Requirements ....................................................................................................................... 12 2.3.2 Safety Requirements...................................................................................................................................... 12 2.3.3 Security Requirements .................................................................................................................................. 12 2.3.4 Software Quality Attributes ......................................................................................................................... 13

3. Usability Specification ................................................................................................................. 13

4. System Architecture and Design............................................................................................. 184.1 Introduction ............................................................................................................................................ 18

4.1.1 System Overview ............................................................................................................................................. 18 4.1.2 Design Constraints .......................................................................................................................................... 19

4.2 System Architecture ........................................................................................................................... 19 4.2.1 System Hardware Architecture ................................................................................................................. 19 4.2.2 System Software Architecture ................................................................................................................... 19

4.3 File and Database Design ................................................................................................................... 21 4.3.1 Database Management System Files.................................................................................................... 21 4.3.2 Non-Database Management System Files ......................................................................................... 21

5. Software Testing ............................................................................................................................ 235.1 Researcher-led beta testing ............................................................................................................ 235.2 Classroom based testing with survey interviewers ............................................................. 23

6. Version Description ...................................................................................................................... 246.1 Inventory of Materials Released .................................................................................................... 24 6.2 Installation Instructions .................................................................................................................... 24 6.3 Known omissions ................................................................................................................................ 25

http://www.seriss.eu/


Appendix 1: FMMS detailed specification of requirements.....................................................27

Appendix 2: FMMS use cases .............................................................................................................35



1. Introduction

1.1 Purpose of the tool

The Fieldwork Management and Monitoring System (FMMS) developed by CentERdata, University of Tilburg for the European Social Survey is designed to allow ESS stakeholders - interviewers, survey agencies, National Coordinators and the Core Scientific Team - to monitor and manage fieldwork more effectively. The FMMS has three main aims:

To improve the amount and timeliness of fieldwork monitoring information available to all

ESS stakeholders by facilitating the transfer of data to and from the field in near real time

throughout the fieldwork period;

To generate standardised contact records and fieldwork progress reports across countries

to allow for more efficient and better informed monitoring of fieldwork cross-nationally;

To facilitate easier and more accurate collection of contact records by allowing one-stop

data collection on the doorstep through an easy to use app.

The FMMS consists of two components: a mobile “app” used by interviewers in the field to manage their caseloads and complete contact records on the doorstep and a centralised case management system (CCMS) which manages the exchange of data between the survey agency and interviewers and maintains a central database which can be used for fieldwork progress monitoring.

1.2 Current status

The FMMS currently exists as a prototype tool. Features identified as essential in the initial specification of requirements have been implemented in the current version (see Section 2.1 and Appendix 1 for further details). However, further development and testing is required before the tool can be considered ready for implementation in the field. A scoping study conducted with ESS stakeholders cross-nationally identified a number of barriers to implementation, including issues around the tool’s requirements for the transfer and storage of personal data, which require further investigation.

Further development work and an in-field test of the next version of the tool will be conducted as part of the pilot for ESS Round 9 in autumn 2017.

1.3 This document

This document provides information on the FMMS software that has been developed for the ESS. It can be used by developers taking over the development or maintenance of the system to learn about its architecture and design principles. We recommend to first read the section about the functional and non-functional requirements and then go to the system software architecture to get an overview of the system.

The report details the FMMS software specification including the tool’s key functionalities, intended user groups and operating environment. It also provides a high level overview of the system architecture and its in and output. It summarises the testing the tool has undergone to date and



the further testing required. Finally, it provides more specific details about the prototype version of the tool currently available and how to explore it.

1.4 Further information

The prototype tool is available online at: www.seriss.centerdata.nl and can be accessed free of charge for the duration of the SERISS project.

Web-based versions of both the FMMS app and central database (CCMS) are available to explore using dummy data pre-loaded into the tool. Test login details are provided. Detailed user guides for the tool are also available.

The website also provides access to the following reports providing further documentation of the FMMS and its development:

Butt, S., Sommer, E., Kuijten, L. and van der Wielen, I. (2016) Fieldwork Management andMonitoring System (FMMS) for the European Social Survey: Report on interviewer testingof the final mobile application and central database. Deliverable 4.9 of the SERISS projectfunded under the European Union’s Horizon 2020 research and innovation programme GANo: 654221.

Butt, S., Sommer, E., Kuijten, L., van der Wielen, I. and van Nieuwburg, B. (2016)Fieldwork Management and Monitoring System (FMMS) for the European Social Survey:Report on test case scenarios. Deliverable 4.8 of the SERISS project funded under theEuropean Union’s Horizon 2020 research and innovation programme GA No: 654221.

Butt, S., Sommer, E. and Kuijten, L. (2016) Fieldwork Management and Monitoring System(FMMS) for the European Social Survey: Report on the feasibility of cross-nationalimplementation. Deliverable 4.7 of the SERISS project funded under the European Union’sHorizon 2020 research and innovation programme GA No: 654221.

The reports are also available at: www.seriss.eu/resources/deliverables

To learn more about the European Social Survey: www.europeansocialsurvey.org The ESS paper contact form which the FMMS app is intended to replace can be viewed here: http://www.europeansocialsurvey.org/docs/round7/fieldwork/source/ESS7_source_contact_forms.pdf

1.5 FMMS Definitions, Acronyms and Abbreviations

FMMS Fieldwork Management and Monitoring System

app Mobile Application for use by interviewers in the field

CCMS Centralised Case Management System i.e. the FMMS central workstation (consisting out of an online portal and connected database)

ESS European Social Survey


http://www.seriss.centerdata.nl/

http://www.seriss.eu/resources/deliverables

http://www.europeansocialsurvey.org/

http://www.europeansocialsurvey.org/docs/round7/fieldwork/source/ESS7_source_contact_forms.pdf

http://www.europeansocialsurvey.org/docs/round7/fieldwork/source/ESS7_source_contact_forms.pdf


CF ESS Contact Form, used by interviewers and completed for each sample unit to record details of contact attempts made

CST ESS Core Scientific Team (responsible for the overall implementation of the ESS)

NC National Coordinator (responsible for overseeing ESS fieldwork in each country)

SA Survey Agency (responsible for delivering fieldwork in each country)

IWER Survey interviewer

admin System administrator

1.6 Acknowledgements

The FMMS was conceived as part of the EU funded FP7 project Data Service Infrastructure for the Social Sciences and Humanities (DASISH - GA No: 283846) and developed under the Horizon 2020 project Synergies for Europe’s Research Infrastructures in the Social Sciences (SERISS - GA No: 654221).

The following people were involved in the development of the FMMS: At ESS ERIC Headquarters, City, University of London: Sarah Butt, Yvette Prestage, Elena Sommer, Sally Widdop At CentERdata: Lennard Kuijten, Iggy van de Wielen, Bart van Nieuwburg, Mathijs van der Paauw At ESS ERIC (GESIS): Verena Halbherr At SHARE ERIC (MPG): Johanna Bristle

2. Software Requirements Specification

2.1 General Requirements

2.1.1 Software Perspective

A key challenge facing all social surveys, especially those operating cross-nationally, is to monitor and manage fieldwork effectively so as to achieve the required effective sample size and balanced response rates. Cross-national surveys without centrally managed fieldwork face a particular challenge as they try to monitor fieldwork conducted by survey agencies in different countries using different methods and systems to manage and monitor data collection. Some survey agencies have invested in technologically sophisticated fieldwork monitoring systems which enable them to closely monitor interviewers’ progress and call outcomes as they happen and so target national fieldwork efforts effectively. However, in other agencies, fieldwork monitoring still involves collection of contact data on paper in the first instance. This can result in delays and inconsistencies in the flow of information and even loss of information between the fieldwork agencies, interviewers in the field and the central survey team, making it difficult to monitor fieldwork in an effective, consistent and timely manner across countries.



The European Social Survey (www.europeansocialsurvey.org), which conducts face to face fieldwork in around 20 to 25 countries every two years, is therefore looking to develop a new electronic Fieldwork Management and Monitoring System (FMMS) in order for all stakeholders in the data collection process to have access to consistent and timely data on countries’ fieldwork progress throughout the fieldwork period. The FMMS consists of two components: a mobile “app” used by interviewers in the field to manage their caseloads and complete contact records on the doorstep and a centralised case management system (CCMS) which manages the exchange of data between the survey agency and interviewers and maintains a central database which can be used for fieldwork progress monitoring (see Figure 1 below).

Figure 1: Overview of Fieldwork Management and Monitoring System

Figure 1: Overview of Fieldwork Management and Monitoring System

The FMMS draws on CentERdata’s previous experience of developing a Sample Distributor and Sample Management System (SD-SMS) for the Survey of Health, Ageing and Retirement in Europe (SHARE ERIC).1 However, the FMMS differs in a number of important respects from the SD-SMS. The FMMS is designed around a mobile application rather than software to be used on a laptop. It also has a different data structure; ESS is a cross-sectional survey of individuals whereas SHARE is a household panel survey. It should therefore be considered as a standalone tool.

1See Malter, F. (2013). Fieldwork Monitoring in the Survey of Health, Ageing and Retirement in Europe (SHARE).

Survey Methods: Insights from the Field. Retrieved from http://surveyinsights.org/?p=1974


http://surveyinsights.org/?p=1974


2.1.2 Software Functions

The FMMS app is intended to be used by interviewers in the field to collect and retrieve contact information about issued sample units (named individuals, households or addresses) and to transmit this information back to the survey agency. It should replicate the key features of the existing ESS Contact Form (CF) which interviewers currently use to manage their cases in the field and collect contact information about them. As a minimum the app should enable interviewers to:

Gain an overview of and manage their cases i.e. plan their workload in the field; Record details of the mode, time and outcome of every contact attempt made during

fieldwork; Record interviewer observations about the neighbourhood for each case; Complete of household and respondent selection; Edit address and contact information about sample units; Make notes about cases; Receive and transmit data to the survey agency via the FMMS central database.

The Centralised Case Management System (CCMS) will be used by survey agencies to manage ESS fieldwork. The CCMS should enable survey managers to:

Upload sample records to the database ; Allocate cases to interviewers (and reallocate cases to different interviewers part way

though fieldwork); Keep track of the progress of individual interviewers and sample units over the course of

fieldwork; Edit information about sample units.

The CCMS should also enable ESS stakeholders to monitor ESS fieldwork in a consistent way across countries. Based on the monitoring data collected using the FMMS stakeholders should be able to use the CCMS to:

Gain an overview of the current state of fieldwork in each country; Download summary reports on the current state of fieldwork in each country; Export a dataset containing case-level contact records for further analysis.

For a full list of the requirements specified for the FMMS app and CCMS at the start of development - and how these were addressed in the prototype tool - see Appendix I.

2.1.3 User Roles and Characteristics

The FMMS has four main user roles in addition to the administrator role (with full access and user rights to enable them to support other users). Survey interviewers will use the FMMS app to manage cases allocated to them in the field, perform respondent selection, record details of contact attempts made and neighbourhood observations for all sample units allocated to them and sync the data collected to the CMMS. Survey agency users are responsible for conducting fieldwork in a country. They will use the CCMS to manage ESS fieldwork, allocate cases to interviewers and receive back contact information from interviewers using the FMMS app in the field. The CCMS will be used to monitor fieldwork progress and to share information on fieldwork progress with NCs and the CST.



National Coordinators: oversee the implementation of the ESS in each country. They will use the CCMS to monitor fieldwork in the country for which they have responsibility.

ESS CST users: The ESS Core Scientific Team (CST) is responsible for the overall implementation of the ESS. The CST will use the CCMS to monitor fieldwork in a consistent way across all countries participating in the ESS.

The FMMS handles personal data. To maintain data security, it is therefore essential that differential access rights are set for each user group with users only having access to the minimum data necessary for their role (see section 2.3.3). This means that the survey agency and NC should only have access to data from their own country and that access to personal data should be restricted to the survey agency.

The user role and access rights of the four FMMS user groups are summarised in the table below.

Table 2: FMMS user roles

Interviewer Survey agency NC ESS CST

Description Interviewers

assigned by

survey agency to

work on ESS

(between 60 and

300 interviewers

per country)

1 agency per

participating

country

May be multiple

people per agency

requiring access

e.g. IT, Ops staff,

field manager

1 NC organisaton

per country

May be 2-3 people

within NC team in

each country

The NC may or

may not be within

the same

organisation as the

Agency

c. 8 fieldwork

monitors spread

across

4-5 CST

institutions

App or

CMMS?

App CCMS CCMS CCMS

Access

Countries Own country only Own country only Own country only All

Cases Only allocated

cases

All All All

Fields All All Deidentified (no

personal

identifiers, no

notes)

Deidentified (no

personal

identifiers, no

notes)

CCMS views - Cases

Interviewers

Cases (summary

only)

Cases (summary

only)

Organizations

Sample types

CCMS Roles

Import - Y

View - Y Y Y

Edit - Y

Export - Y Y Y



2.1.4 Operating Environment

The FMMS app is intended to be used on handheld mobile devices such as tablets and smartphones so that interviewers can collect contact data in real time on the doorstep. So that the app is compatible with survey agencies’ existing hardware, it should be possible to install the app on Windows 10 (phone and desktop), Android or iOS (Apple) devices running the latest operating systems. The CCMS is a PHP based web application running on a WAMPP or LAMPP stack. Data transmitted to and from the app will be stored in a central database hosted and maintained by CentERdata on a secure cloud-based server. ESS stakeholders will access the database via the web (using known communication keys (see Section 2.3.3 on security requirements).

2.1.5 Design and Implementation Constraints

The following constraints to the design and implementation of the FMMS were identified from the start of development and informed the software design:

The FMMS must be implementable across all countries and organisations responsible for carrying out ESS fieldwork. This means, first, that the tool must be compatible with different mobile devices and any existing fieldwork monitoring software that agencies may use. Second, the FMMS must be usable by organisations with different levels of technological capabilities and IT support; it should operate as a standalone system and not be reliant on existing capabilities. Third, the FMMS must be sufficiently flexible to accommodate differences in fieldwork practices across countries, most notably it has to accommodate the used of both individual named and address-based sampling frames.

The FMMS app is designed to be used out in the field. This means that usability and reliability are priorities. It also means that an internet connection may not always be available and that it must be possible to input and retrieve data from the app offline.

The FMMS handles personal data and must therefore transmit and store all data securely in a way that is compatible with national and European data protection regulations.

2.1.6 User Documentation

Two user guides, one for the FMMS app and one for the CCMS, have been produced to accompany the prototype tool. These are available to download at: https://seriss.centerdata.nl

2.1.7 Assumptions and Dependencies

The FMMS depends on the Cordova library managing the deployment on the various mobile platforms. Any decision by Cordova to not support a specific version of such mobile platform will have the consequence that the FMMS will not work on that platform. Therefor the windows platform 8.1 is not supported. The CCMS is built upon CakePHP. This is an open source framework for creating web applications. A stable PHP version is mandatory to work correctly.


https://seriss.centerdata.nl/


2.2 External Interface Requirements

2.2.1 User Interfaces

A user interface is required for both the app and CCMS. The user interface for both components must be intuitive to use. The app will be used outdoors, in all weathers and in different visibility conditions (bright sunlight, after dark). Contrast will therefore be important and ideally, the interface should adapt to different light conditions. The app will be used in multiple countries and IWERS and survey agency personnel will not necessarily speak English. It should be possible for users to customise the language in which they use the tool.

2.2.2 Hardware Interfaces

The FMMS will support multiple mobile devices (phones and tables) if the correct software operating system (Android, IOS or Windows Universal apps)is installed and a network (cellular or wifi) is enabled. The CMMS needs to work on a server attached to the internet listening on port 443 (https) via an Apache server. The database can be on the same server or running on another server in the same network.

2.2.3 Software Interfaces

Survey agencies will be responsible for importing sample records into the CCMS. Data will be imported as a .csv file. It should be possible to export summary progress records and case-level contact records (composed of data collected via the app) from the CCMS. Data can be exported as a .csv file. Future developments may support data import and export in other formats, for example SQL. The FMMS is currently intended to operate as a standalone tool. The possibility of developing APIs to make the FMMS compatible with other fieldwork management and monitoring software tools, and/or Computer Assisted (CAPI) interview programmes, already in use with survey agencies was considered. However, as the software agencies use varies both within and between countries, and survey agencies can change from one round of the ESS to the next, such an interface would (at least in the first instance) need to be developed on a case-by-case basis in close cooperation with a specific agency. It is therefore not currently part of the specification for the FMMS.

2.2.4 Communication interfaces

The FMMS app needs to communicate with the CCMS and allow data to be transferred to and from interviewers in the field on a regular basis i.e. at least daily. Data transfer will be via the internet using secure protocols (HTTPS). However, when interviewers are in the field they may not have access to the internet and be able to transfer data straight away so local storage on the mobile device is also necessary. Data must not be lost if an internet connection (or power) is lost during data transfer.



2.3 Other Non-functional Requirements

2.3.1 Performance Requirements

The FMMS must be capable of handling a large amount of data. Each round of ESS fieldwork involves collecting contact data for around 100,000 sample units with multiple contact attempts (an average of 2-5, but sometimes 10+) to be logged for each unit.

Multiple users will need to access the system simultaneously. If rolled out across the ESS the FMMS must be able to support data collection in around 25 countries. The app will be used throughout the day by multiple interviewers in each country (between 60 and 300 assigned per country). The CCMS is also likely to be accessed simultaneously by multiple users (up to 15) within each country’s survey agency (as, for example, responsibility for fieldwork monitoring is shared among regional supervisors) as well as by the NC in each country and CST users.

Response times are important, especially within the app, which the interviewer will be using to collect data on the doorstep in real time. Response times for the app should be no more than 2-3 seconds.

2.3.2 Safety Requirements

Data entered into the FMMS needs to be retained during and beyond the ESS fieldwork period. The central database must be backed up regularly to minimise the risk that any contact data collected is not lost accidentally.

The intention is that data stored within the CMMS will be backed up daily by CentERdata’s ISO 9001 and ISO 27001 certified hosting partner, using a round robin backup strategy (with a daily frequency and a one-week cycle). A separate daily backup will also be carried out on a different server (hosted by the same organization and located within the Netherlands) as an additional safeguard against server failure or other technical problems.

2.3.3 Security Requirements

The FMMS will be handling personal data and so data security is of paramount importance. The FMMS along with all ESS ERIC activities must demonstrate compliance with the ISI’s Declaration of Professional Ethics2. Its use on the ESS will also need to be approved by the ESS ERIC Research Ethics Committee. The FMMS must treat data in accordance with EU data protection requirements (currently General Directive 95/46/EC3, to be replaced by the European Data Protection Regulation from May 2018). As the headquarters of the ESS ERIC, which acts as the Data Controller for the FMMS is based in the UK, the tool must also comply with the UK Data Protection Act 1998.4 Where national data protection regulations in individual ESS countries impose requirements over and above those imposed by EU regulations, the more stringent requirements should also be adhered to. Finally, the FMMS will also need to comply with any organisational requirements imposed by the survey agencies responsible for data collection and/or the organisations responsible for supplying sample records.

As a minimum, access to both the app and CCMS should be via secure login, data should be encrypted with decryption possible only on supply of known communication keys, and all data

2 https://www.isi-web.org/index.php/activities/professional-ethics/isi-declaration 3 http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:31995L0046:en:HTML 4 http://www.legislation.gov.uk/ukpga/1998/29/contents


https://www.isi-web.org/index.php/activities/professional-ethics/isi-declaration

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:31995L0046:en:HTML

http://www.legislation.gov.uk/ukpga/1998/29/contents


transfer should be via HTTPS. Differential access rights for the four user groups will be necessary to ensure that access to personal data is kept to a minimum.

Data stored on mobile devices should not be accessible by unauthorised use. Each time the app is opened the app should trigger a login screen to authenticate the user and the app should lock after a short amount of time left inactive. The data should not be accessible via other applications on the same device. No data tracking or personal communication (e.g. telephone calls) will take place within the app so as to minimise the possibilities of hacking or the risk of invading the interviewer or respondent’s privacy through tracking.

Data within the CCMS will be stored on a virtual private server, hosted by CentERdata’s ISO 9001 and ISO 27001 certified hosting partner based in the Netherlands (and so subject to Dutch data protection legislation). The server and associated data security procedures are in line with Certified Information Systems Security Professional (CISSP) guidelines.

Data will only be retained within the system for the current fieldwork round. It must be possible for the system to be emptied at the end of each round of ESS fieldwork. To comply with the “right to be forgotten” it must also be possible for data collected for individual sample units to be deleted during fieldwork if requested.

2.3.4 Software Quality Attributes

Attributes that should be prioritised when developing the software are: Reliability - the FMMS must work reliably in the field to ensure that data can be input at any

time required; Usability - the tool, especially the app, must be intuitive to use whilst the user is

concentrating on other tasks, for example obtaining an interview with a potentialrespondent;

Responsiveness - interviewers must be able to enter data in the field in real time; Interoperability - the tool must be usable by survey agencies located in different countries,

using different hardware, and with different levels of technical capacity.

3. Usability Specification

To inform development and testing of the FMMS, use cases were developed for each of the key functionalities listed in Section 2.1.2 and grouped together in scenarios to illustrate the typical workflow within the FMMS. The scenarios - and the use cases associated with them - are described below. Full details of the use cases can be found in Appendix 2.

Scenario 1: At the start of fieldwork, a survey agency (SA) user successfully logs into CCMS and uploads to the CCMS a) sample records b) details of the interviewers assigned to work on the ESS. The CCMS syncs with IWER devices so that IWERS receive their cases upon logging into the app and are ready to start making contact attempts.

User(s) involved SA IWER

Preconditions User accounts set up Details of the survey agency and sample type input to the CCMS by administrator IWER has internet access (to sync with CCMS)



Use cases SA logs in securely to CCMS (CLOG01) SA imports interviewer records for interviewers assigned to work on

ESS to CCMS (IMPT01)

SA imports sample records to CCMS (IMPT02)

The SA is able to manually allocate cases to interviewers (ALLC01) IWER logs in securely to app (ILOG01) The app syncs with the CCMS and the IWER picks up new cases

allocated to them (SYNC01)

Scenario 2: An IWER issued with a sample of addresses plans their workload for the day, deciding to visit address x. They makes contact with someone at the target address, perform respondent selection (1 household, 2 residents 15+) and find that the selected respondent is not at home. The IWER makes a note to call back on Thursday. User(s) involved IWER

Preconditions User accounts set up

Details of the survey agency and sample type input to the CCMS by administrator SA has imported sample data into the CCMS and allocated cases to interviewer IWER has internet access (to sync with CCMS)

Use cases IWER logs in securely to app (ILOG01) IWER browses the list of cases allocated to them (OVER01) IWER searches for a specific case (OVER02) IWER logs a contact attempt at the address

o Logs date, time and mode of visit (ATTM01) o Records an appointment (ATTM06)

IWER carries out household and respondent selection o Household selection with 1 household (SELC01) o Respondent selection with multiple respondents (SELC02)

IWER makes a note of appointment (NOTE01) IWER syncs app with CCMS (SYNC02)

Scenario 3: A (male) IWER makes a call at an address, makes contact with the named respondent who refuses to take part. The IWER logs the necessary information about the contact attempt including reason for refusal, estimated likelihood of participation) and records neighbourhood observations about the address. They then request that the case be reallocated, making a note that it should be reallocated to a female IWER. The survey agency receives the request and reallocates case to another (female) IWER who, on their first visit, manages to convert case into an interview.



User(s) involved SA IWER 1 and IWER 2

Preconditions User accounts set up Details of the survey agency and sample type input to the CCMS by administrator SA has imported sample data into the CCMS and allocated cases to interviewer IWER has internet access (to sync with CCMS)

Use cases IWER1 logs in securely to the app (ILOG001) IWER1 log a contact attempt, recording a refusal and completing the necessary follow up questions (ATTM04) IWER1 requests that the case be reallocated to another IWER (ALLOC2) IWER1 makes a note about the reallocation request (NOTE01) SA manually reallocates the case to IWER2 (ALLC03) IWER2 logs in securely to the app (ILOG01) IWER logs a contact attempt, recording a completed interview (ATTM03)

Scenario 4: An IWER issued with a sample of named individuals calls at an address and makes contact with someone who is not the respondent. That person tells the IWER that the named respondent has moved just round the corner (still within the IWER’s area) and gives the IWER the new address. The IWER changes the address information within the app and goes to make a contact attempt at the new address. There is no contact with anyone. The IWER completes the neighbourhood observations for the new address. User(s) involved IWER


Details of the survey agency and sample type input to the CCMS by administrator SA has imported sample data into the CCMS and allocated cases to interviewer IWER has internet access (to sync with CCMS)

Use cases IWER logs in securely to the app (ILOG01) IWER records change of respondent address in the app (ATTM07) IWER logs a contact attempt (no contact with anyone) (ATTM05) IWER completes neighbourhood observations (NBQ01) IWER manually syncs app with CCMS (SYNC02)

Scenario 5: An IWER syncs the app before leaving home. They go to make a face to face visit at an address with no internet connection available, find that the house is actually split up into two self-contained flats (A and B) and so has to select a household. The IWER records in the app that there are two households and the app makes a selection. The IWER knocks on the door of the selected flat, makes contact with someone who says they live there alone. The person agrees to be interviewed and a completed interview is achieved.



User(s) involved IWER

Preconditions User accounts set up Details of the survey agency and sample type input to the CCMS by administrator SA has imported sample data into the CCMS and allocated cases to interviewer

Use cases IWER logs in securely to the app (ILOG01) IWER logs a contact attempt at a selected address

o Logs date, time and mode of visit (ATTM01) IWER carries out household and respondent selection

o Household selection with 2 households (SELC01) o Respondent selection with one respondent (SELC02)

IWER logs a completed interview (ATTM03) App functions – and data is saved – even though it is offline

Scenario 6: An IWER makes a face to face visit, discovers that the visited address is invalid (has been demolished) and returns home. Later that day they recall that they forgot to log the visit into the app. They log into the app, enter the case to log a visit and manually overwrite the time of the visit to be 2 hours earlier that day. They then sync the app with the CCMS to transmit the contact information to the survey agency. User(s) involved IWER


Details of the survey agency and sample type input to the CCMS by administrator SA has imported sample data into the CCMS and allocated cases to interviewer

Use cases IWER logs in securely to the app (ILOG01) IWER logs a contact attempt at a selected address

o Logs date, time and mode of visit (ATTM01 – alternative course)

o Records outcome of visit as “Address invalid” (ATTM02) IWER syncs app with CCMS (SYNC02)

Scenario 7: The SA decides to issue cases into the field in two batches. Part way through fieldwork, and just before they are due to issue the second batch of cases, the SA has concerns that several interviewer may be underperforming and decide to investigate further. They export a contact data file from the CCMS and examine the IWERs’ cases. They decide to reallocate the IWERs’ unproductive cases different IWERs. They create a new sample file (with different IWER allocations) for import upload this to FMMS. The SA also uploads the second batch of cases into FMMS for IWERS to start working on. IWERs login to app to retrieve new cases.



User(s) involved SA

IWER

Preconditions User accounts set up Details of the survey agency and sample type input to the CCMS by administrator SA has imported sample data into the CCMS and allocated cases to interviewer IWERs have recorded some contact information using the app and synced to the CCMS

Use cases SA logs in securely to the CCMS (CLOG01) SA exports a case-level data file from the CCMS (EXPT01) SA uploads new sample records to CCMS

o Uploads batch 2 cases (IMPT03) o (re)loads some batch 1 addresses with cases assigned to

different IWERS (IMPT04) New cases are synced to app (SYNC01) IWERs log in securely to app to retrieve new cases (ILOG01)

Scnenario 8: A CST member logs in to the CCMS to run a summary fieldwork report for a given country X and look at the latest contact data prior to a meeting with other CST members to discuss fieldwork progress across countries. After the meeting, the CST member then calls the NC to discuss fieldwork progress in CNTRY X. User(s) involved CST

NC

Preconditions User accounts set up Details of the survey agency and sample type input to the CCMS by administrator SA has imported sample data into the CCMS and allocated cases to interviewer IWERs have recorded some contact information using the app and synced to the CCMS

Use cases CST logs in securely to the CCMS (CLOG03) CST downloads a summary progress report for Country X (REPT01) CST exports case-level data file for Country X (EXPT01) NC logs into CCMS (CLOG02) NC downloads a summary progress report for Country X (REPT01)



4. System Architecture and Design

4.1 Introduction

4.1.1 System Overview

The system has a client server architecture where the clients on a pc uses a web browser to interact with the application. For the mobile devices a native app is developed who will connect to the same application for downloading and uploading data. Figure 2: FMMS system overview



4.1.2 Design Constraints

The design constrain used for developing the software are: Import and export of data should happen via a known format useable by different software

tools The webserver used is limited to Apache Database support is limited to MySQL The graphical user interface is created via html with the help of the HTML,CSS and

Javascript framework Bootstrap The PHP and CakePHP guidelines for development are followed. Security and Anonymize measures should be implemented application wide and not for

each functionality.

4.2 System Architecture

4.2.1 System Hardware Architecture

The current hardware architecture is partitioned in 3 main servers with the following specifications: One Apache web server Two MySql database servers

The Apache web server host the main system interface. It handles the different data downloading and uploading via https.

The MySql database servers store the data provided by the web server. One MySql server is the master server where all data is insert. The second MySQL server is used as a fall over mechanism for the master server and is used as a read server for load balancing.

4.2.2 System Software Architecture

The CMMS and FMMS app both use a Model View Controller architecture so the application can be more easily maintained.

The CMMS is programmed in PHP with the help of the CakePHP framework.

The FMMS app is programmed in Javascript with the help of AngularJS, ONSENUI and Cordova for deployment on mobile devices.

The architecture of the CMMS, also followed by the FMMS app, can be seen in Figure 3.


20

Figure 3: FMMS Class Diagram


4.3 File and Database Design

4.3.1 Database Management System Files

The entities used in the FMMS system are: Entity Description Address Address information for a certain case Case Unique case who needs to be contacted ContactOutcomeResultcode All codes that can be specified during a contact attempt Country List of countries known in the system and used for organizations

and samples Fieldwork To specify which round we are administering for a certain country Neighbourhoodquestion Questions asked for each address Notes Messages logged with a connect attempt Respondent Person who is interviewed Role Specify different roles within the system to use with the Acces

Control list User Table to identify user in the system The relations between these entities are shown in the Entity Relationship Diagram (Figure 4)

4.3.2 Non-Database Management System Files

The CCMS uses the following files for input: File Description Interviewer import Import file for importing multiple interviewers at once including personal and

login information Cases import Import file for importing the sample of a round into the system holding all

cases that needs to be interviewed The CCMS uses the following files for output: File Description Statistics export Export the current state of fieldwork which summarizes all cases to their

current final state for example interview, refusal not contacted etc. Cases export Export all information of the current sample listing all cases including their

respondents, contact attempts etc.


22

Figure 4: FMMS Entity Relationship Diagram


5. Software Testing

The prototype FMMS has undergone two main rounds of testing to ensure that it meets the needs of its intended users.

5.1 Researcher-led beta testing

Two rounds of researcher-led beta testing of the FMMS were carried out to test whether the prototype tool met the business needs of the ESS and to detect missing or defective functionalities. Testing was based around test case scenarios developed by the researchers in consultation with the developers. Role plays in which testers took on different user roles were used to group cases for testing and, as far as possible within an office-based testing environment, simulate how the tool would be used in practice.

Success criteria for each test case were identified with each case marked as pass/fail depending on whether the success criteria were met. Where a test case was technically passed but the possibility of improvements was identified these were noted as comments. Following testing, failed or skipped test cases and issues identified were assigned with fixes following a priority ranking based on MuSCoW prioritisation (Must, Should, Could, Would) i.e. issues that must/should be fixed as part of the SERISS project vs. those which could be fixed but are not a top priority. The lowest ranking “would” was used for features which “would be nice to have” but which were not considered essential. “Must” and “should” fixes were implemented before the tool underwent testing with ESS interviewers.

More information about how the testing was implemented, the testing scenarios used and the result obtained is available in a separate report:

Butt, S., Sommer, E., Kuijten, L., van der Wielen, I. and van Nieuwburg, B. (2016) Fieldwork Management and Monitoring System (FMMS) for the European Social Survey: Report on test case scenarios. Deliverable 4.8 of the SERISS project funded under the European Union’s Horizon 2020 research and innovation programme GA No: 654221.

Available at: www.seriss.centerdata.nl or www.seriss.eu/resources/deliverables

5.2 Classroom based testing with survey interviewers

Interviewer testing focused on the FMMS app. The app was tested on mobile devices (tablets) in a one day classroom-based testing session with a small group of interviewers from NatCen Social Research, the ESS National Coordinator and the survey agency responsible for conducting ESS Rounds 7 and 8 in the UK. The goal of the testing day was to gather end user feedback on the workflow within the app and its user interface. Testing was based around test case scenarios developed by the researchers to simulate - as far as possible within a classroom environment – how the tool would be used in practice on the doorstep and to test multiple features within the app. During the testing session, multiple testers were collecting and transmitting data in the tool simultaneously, thereby also providing an opportunity to test that syncing or data exchange between the app and the CCMS was quick and reliable. Finally, the testing session provided an opportunity to observe how quickly interviewers picked up how to use the app, and to obtain





feedback on the training materials provided to interviewers (for example, the user manual) in preparation for any future roll out of the tool. More information about how the testing was implemented, the testing scenarios used and the result obtained is available in a separate report. Butt, S., Sommer, E., Kuijten, L. and van der Wielen, I. (2016) Fieldwork Management and Monitoring System (FMMS) for the European Social Survey: Report on interviewer testing of the final mobile application and central database. Deliverable 4.9 of the SERISS project funded under the European Union’s Horizon 2020 research and innovation programme GA No: 654221. Available at: https://seriss.centerdata.nl/ or www.seriss.eu/resources/deliverables Further testing will be required before the tool can be considered ready for roll out: A proper field test should be carried out to ensure that the tool is sufficiently user friendly, responsive and reliable for use in real time. Ideally this test should take place in multiple countries. The tool also needs to undergo both security threat testing and stress testing to ensure that it complies with data protection requirements and can cope with multiple users entering and transferring data simultaneously (with the FMMS needing to be capable of handling records for up to 100,000 sample units in each fieldwork round.

6. Version Description

The current version of the FMMS, made available at www.seriss.centerdata.nl on 28th February 2017, is a prototype. The prototype is publically available free of charge for the duration of the SERISS project. The tool is provided for illustration and exploration purposes; it is not yet field-ready. Anyone interested in finding out more about the tool and its possible application in the field can contact CentERdata via [email protected].

6.1 Inventory of Materials Released

The following software is available for exploration:

A web-based version of the FMMS app that works in Chrome or Safari. A Windows version of the app that works with windows 10 (Windows Universal app) An Android version of the app that works with Android Kitkat (4.4) or higher A web-based Centralised Case Management System (CCMS)

The final field-ready release of the tool will also include an IOS version of the app.

6.2 Installation Instructions

The web-based version of the FMMS app can be accessed via Chrome or Safari web browsers. No installation or special settings are required. To install the Windows or Android versions of the app on a mobile device use the following instructions:





mailto:[email protected]


6.2.1 Windows 10

First enable the developer mode on your Windows 10 machine. This is done via searching “developer” in the Windows Search Box. Then Select "Use developer features". A new window opens where you can select "Developer mode"

Then download the correct installer from https://seriss.centerdata.nl/seriss/ and the app will automatically be installed.

6.2.2 Windows Phone

For windows phone the procedure is the same i.e. you need to enable ”developer mode”. This settings is available via the “update and security” section of the “settings” app.

6.2.3 Android

For android it highly depends on the manufacture of your phone/tablet. The procedure for an Android stock device is given; most manufacturers follow similar procedures.

Go to the settings of your device and choose “Security” then enable “Unknown sources” which lets you install apps from sources other than the google play store. Now you can download the app file from https://seriss.centerdata.nl/

The CCMS requires no installation. The CCMS is a web-based tool located on a secure cloud-based server hosted and maintained by CentERdata in the Netherlands. To access the tool all users require is a reliable internet connection and an up to date browser using HTML5 protocols.

Dummy interviewer and sample data have been pre-loaded into the prototype tool for two countries, country 1 using an individual named sample and country 2 using an address-based sample (more information about the data imported). Test login details are provided in the user manuals and given below.

User User name Password FMMS app Interviewer in Country 1 (individual named sample)

EEIW1 EEIW1

Interviewer in Country 2 (address-based sample)

UKIW1 UKIW1

CCMS Survey agency in Country 1 Agency1 welcome Survey agency in Country 2 Agency2 welcome NC in Country 1 nc1 welcome NC in Country 2 nc2 welcome CST ess1 welcome

The CCMS prototype operates on a secure server and the transfer of all information between the app and the CCMS is carried out using HTTPS protocols. However, the prototype has not undergone full security testing. Therefore, the prototype should only be used with the dummy data provided.


https://seriss.centerdata.nl/seriss/


https://seriss.centerdata.nl/seriss/import/Import%20files.zip


6.3 Known omissions

There are no known errors in the prototype software.

A few features of the tool outlined in the original specification are not fully operational within the prototype version (see Appendix 1 for further details). In particular note that:

The IOS version of the app has not been made available on the prototype website (IOSapps can only be made available through Apple stores);

The CCMS is intended to produce and store summary reports of fieldwork progress on adaily/weekly basis. It is possible to download a summary report of the current fieldworksituation using the CCMS prototype but it is not currently possible to retrieve reports fromearlier in the fieldwork period. Before the tool is launched in the field an algorithm will beput in place to produce and store updates automatically;

It is possible to export a case-level data file for analysis. However, the same file - includingsome personal identifiers - is exportable by all CCMS users, regardless of access rights i.e.regardless of whether they should have access to personal information. The next versionof the tool will need to address this issue with fields containing personal identifiers notavailable to NC and CST users;

The FMMS is currently available in English only. Further work is needed to develop amultilingual version of the tool.


27

Appendix 1: FMMS Specification

A.1 FMMS requirements

Table 1: FMMS mobile application

Features are grouped into four broad categories: Usability, Security, Data transfer and Fieldwork processes

Group Feature Essential or

optional Comments / observations Status In prototype v1.0

StUsability Simple and easy to start, use

and navigate through

Essential Implemented (and tested with

survey interviewers)

Usable in all countries –

regardless of which type of

smart phone or tablet is used

Essential Partially implemented: Available for

Windows and Android devices (IOS

to follow)

Compatible with system(s)

currently used by fieldwork

organisations

Essential The FMMS will initially be developed as a

stand-alone tool with data

imported/exported in CSV format.

Work to integrate the FMMS with

organisations’ own systems would need to

be carried out on a case by case basis and

led by individual agencies.

Implemented as a standalone tool

with straightforward import/export

capabilities

Suitable for all interviewers –

regardless of whether using

PAPI or CAPI administration for

the main survey interview

Essential Implemented

28

Group

Feature

Essential or

optional

Comments / observations

Status In prototype v1.0

Compatible with samples of

address, household and named

individuals

Essential For address samples the interviewer is

prompted to carry out a household and

respondent selection.

For household based samples, the

interviewer is prompted to carry out

respondent selection.

This feature should be enabled / disabled

based on the sample type.

App will assume selection is done via KISH.

Last/first birthday selection methods will not

be supported.

Implemented

Data can be accessed and

entered both online and offline

Essential Internet connection required to sync with

central database but interviewer can

access and work on downloaded cases

even in absence of internet connection.

First login requires an internet connection

but possible to login offline subsequently

Fir

Implemented

Security Password protected secure log-

in for interviewers; no capacity

for the application to

‘remember’ a password


User timed-out of the application

after a defined period of time

Essential Period of time before app locks still to

be determined

Not implemented

Only details of cases assigned

to interviewer visible


Secure transfer of data

to/from central server

Essential Data is transferred via secure headers

and https protocol.

Implemented: Dedicated

security testing to be carried out

29


optional Comments / observations Status In prototype v1.0

Summary overview to

instantly identify the status of

a case

Essential The overview screen provides a simple

view of all cases and their current status.

Colour

coding and search and filter functions also

help the interviewer to navigate through the

cases.

Implemented

Colour coding to determine

status of case - based on

outcome code recorded

Optional Use a simple 3 colour traffic light system Implemented

Filters to manage cases using the

overview summary lists

Optional Implemented

Possibility to select individual

cases and to record/track their

status throughout fieldwork

Essential Cases can be selected individually and

viewed and edited accordingly by the

interviewer from the mobile application.

Case history visible for each individual case

Implemented

Interviewer can complete

household and respondent

selection (if applicable)

Essential Selection should be randomly generated

(no first/last birthday).

App should display info on

household/respondent selected to

interviewer and information on selection

should be stored within central database

Implemented

Log visit information and contact

attempts

Essential Date and time of visits logged will be generated automatically by app. However, interviewer also has capacity to overwrite the date and time.

Each visit logged should be attributed to a specific interviewer within central database. This should not change even if case is later allocated to a new interviewer

Implemented

30

Group

Feature

Essential or

optional

Comments / observations

Status In prototype v1.0

Record outcome codes for

contact attempts

Essential Options available mirror those available in

ESS Contact Form

Implemented

Complete follow up questions

for refusals i.e. reasons for

refusal, likelihood of future

cooperation etc.

Essential Follow up questions follow logic/routing of

ESS Contact Form

Implemented

Space to record new

address if respondent has

moved

moved

Essential Interviewer can enter new address and

state whether this new address remains in

their area.

Any new address details do not overwrite

the original address stored in the central

database.

Implemented

Log to record answers to

neighbourhood questions

Essential Available to be completed once for each

address

Implemented

Note making facility (for

interviewers to make notes for

themselves)

Optional Notes can be added at the individual case

level and a summary of all notes can be

seen

Implemented

Interviewer able to signal to

fieldwork organisation that they

have completed work on a case

and would like to return it to

office

Optional Implemented

Ability to send messages to

fieldwork organisation

Optional Will not be implemented - note making

facility can be used to communication with

SA

Not implemented

Checks/notifications to remind

interviewer to complete certain

tasks

Optional Examples could include:

- Reminder to complete interviewer

observations if not done so by time log first

visit

- Case locked/new outcome cannot be

recorded once completed interview recorded

- Reminder to edit address if record that

respondent has moved

Not implemented

Include a calendar to help

interviewers organise their

workload

Optional Will not be implemented in first prototype Not implemented

A map or travel distance

indication feature to help

interviewers manage cases

e.g. by

location of respondent

Optional Googlemap view of address will be visible

via app

Not implemented

GPS of interviewer Optional Will not be implemented. Data protection

considerations

Not implemented

31

Table 2: FMMS centralised case management system (CCMS)

Features are grouped into four broad categories: Usability, Security, Fieldwork processes and Outputs


optional

Comments Status in prototype FMMS v1.0

Usability Compatible with system(s)

currently used by fieldwork

organisations

Essential The FMMS will initially be developed as a

stand-alone tool with data

imported/exported manually in .csv format.

Work to integrate the FMMS with

organisations’ own systems would need to

be carried out on a case by case basis and

led by individual agencies.

Implemented as a standalone

tool with straightforward

import/export capabilities

Compatible with a range of CAPI

programs

Optional This feature will not be implemented in

initial prototype. Future work would be

needed to scope the CAPI system(s)

used by agencies, and how these could

communicate with the FMMS.

Not implemented

Compatible with samples of

address, household and named

individuals

Essential CCMS will allow import of individual

and address sample files

Implemented

Some features available only to relevant users (e.g. Survey Agency (SA) are only users to import data)

Essential Differential access rights to be assigned on

login

Implemented

Security Access to CCMS via

personalized secure login Essential Differential access rights to be assigned on

login

Implemented

Access to personal data (e.g.

name and address of target

respondent) restricted to SA

Essential

Differential access rights to be assigned on

login

Partially implemented: Except with

respect to data export feature (see

below)

32


optional


Access of NCs and SA restricted

to data from their own country Essential Differential access rights to be assigned on

login (see A.3 FMMS Access Rights).

Implemented

Secure cloud based storage of

data within CCMS

Essential As currently designed all ESS contact

form data (including personal data) will

be stored within central database

hosted in the Cloud

Implemented: Dedicated

security testing remains to

be carried out

Secure transfer of data

to/from central server

Essential Data is transferred via secure headers

and https protocol.

Implemented

Fieldwork

processes

Import case records for

individual, household and

address samples

Essential Import allows for a mix of compulsory,

optional and user defined fields

Implemented

Assign a unique reference

identifier to each case

Essential Case ID will be supplied by fieldwork

agency as part of sample import file

Case IDs should start with a two digit

country ID to avoid duplication

Implemented

Offer the possibility of

transferring information to/from

mobile application

on a frequent basis

Essential App and central database sync when

interviewer logs in (online) or completes

manual sync

Implemented

Handle multiple as well as

single stages of case

allocation to interviewers

Essential There should be scope to import sample in

batches i.e. to add new cases to those

already stored in CCMS

It should also be possible to upload a new

sample file - with cases reissued to different

interviewers - mid way through fieldwork

Partially implemented: Batch

import is possible. Reimporting

updated sample records i.e. with a

different interviewer assigned is

not possible without

33


optional


Possible for cases to be

reassigned to interviewers

during fieldwork e.g. for refusal

conversion or if respondent has

moved out of area

Essential Interviewers can be assigned to cases during

sample import phase or manually within

CCMS

Central database keeps a record of all the

interviewers that have worked on a case and

which interviewer logged which visits

Implemented

SA can monitor fieldwork

progress on individual cases and

by individual interviewers

Essential CCMS overview screen provides summary of

key info (e.g. outcome and date of last visit,

number of visits) for all cases

Possible to filter cases by interviewer or

outcome code

Survey Agency (SA) can enter an individual

case to find out more info e.g. history of

contact attempts, read interviewer notes

Implemented

SA can edit case level data

within CCMS

Essential Ideally SA should be able to:

- Delete case record if respondent

requests

- Designate a case as office refusal

- Correct errors identified in data entered

by interviewer

Implemented

Data outputs Aggregate level progress

summaries available during

fieldwork

Essential Available to all users (CST, NC, SA) on a

country by country basis.

Partially implemented: reports are

available within CCMS and can be

downloaded.

Option to retrieve past updates

from earlier in fieldwork not yet

available

Case level output file containing

contact form data available to

Essential Ideally there should be an option for user

defined exports e.g. specifying certain

Partially implemented: case-level

export file is available to export

34


optional


download cases/fields

Data should export in user friendly format

e.g. excel/SPSS (or CSV).

Output files need to be usable during

fieldwork for bespoke progress

monitoring/analysis and to provide basis for

full contact form dataset at end of fieldwork

However, exports are not tailored

to different user groups

35

Appendix 2: FMMS use cases

Secure login and differential access rights to CCMS (CLOG)

Use case identifier CLOG01

Description SA can log in securely to the CCMS and gains access rights suitable to

role

Actor SA

Course of events SA users goes to CCMS URL in web browser

Enters username correctly

Enters password correctly

Clicks login

Outcome SA gains access to CCMS home screen

Menu options visible correspond to SA view i.e. no option to choose a

country in left hand menu, admin functions not visible.

Import and export functions available

SA can see only data for their own country

Alternative course If login credentials are entered incorrectly, access is denied. The user can

contact administrator for password reminder.


Description NC can log in securely to the CCMS and gains access rights suitable to

role

Actor NC

Course of events NC goes to CCMS URL in web browser



Clicks login

Outcome NC gains access to CCMS home screen

Menu options visible correspond to NC view i.e. no option to choose a

country in left hand menu, admin functions not visible, IWER info not

available.

Export function available. Import function not available.

36

NC only sees cases from their country




Description CST user can log in securely to the CCMS and gains access rights

suitable to role

Actor CST

Course of events CST user goes to CCMS URL in web browser



Clicks login

Outcome CST user gains access to CCMS home screen

Menu options visible correspond to CST view i.e. option to choose a

country, admin functions not visible, IWER info not available.

Export function available. Import function not available.



Secure login to app (ILOG01)

Use case identifier ILOG01

Description IWER can log in securely to the app

Actor IWER

Course of events IWER opens the app



Clicks login

Outcome App opens to show case overview and display cases allocated to that

IWER

This will be blank if no cases yet allocated

Alternative course If login credentials are entered incorrectly, access is denied. The IWER

will get a message telling them to contact the survey agency for a

password reminder

37

Importing data into CCMS (IMPT)

Use case identifier IMPT01

Description SA successfully imports an IWER data file into the CCMS

Actor SA

Course of events SA completes use case CLOG01 and gains access to CCMS

SA clicks on “import” function

SA uploads IWER data file as a .csv file using the file template provided

Outcome (Assuming the file format matches the agreed specification)

Data successfully imported

Message appears to confirm this

Imported data is visible in the CCMS

Alternative course If the data file uploaded does not match the specification (e.g. if data is

missing from required fields or if fields are in the wrong order) the file wil

be rejected and an error message will appear.


Description SA successfully imports a sample data file into the CCMS

Actor SA

Course of events SA completes use case CLOG01 and gains access to CCMS

SA completes use case IMPT01 (IWER data has to be imported before

sample data)


SA uploads sample data file as a .csv file using the file template

provided




Imported data is visible in the CCMS


missing from required fields or if fields are in the wrong order) the file will


38

Note that there will be two templates: One for countries using samples of

named individuals and another for countries using samples of

households/addresses.


Description SA successfully imports a second batch of sample cases into CCMS

Actor SA

Course of events SA completes use case IMPT02

SA completes use case CLOG01 and gains access to CCMS


SA uploads second batch of sample as a .csv file using the file template

provided




Second batch of cases is visible in the CCMS alongside previously

imported cases




All cases must be assigned a unique id to prevent cases already in the

CCMS from being overwritten.


Description SA successfully imports an updated sample file into CCMS to reallocate

(some) cases to different IWERs

Actor SA


SA completes use case CLOG01 and gains access to CCMS


SA uploads


39



Relevant details (e.g. allocated interviewers) are updated within CCMS.

Any data already entered for the case e.g. previous contact attempts

remains visible




Allocation of cases to interviewers (ALLC)

Use case identifier ALLC01

Description SA is able to manually allocate a case to a specific interviewer

Actor SA


Locates record for a specific sample case

Allocates the case to an interviewer

Outcome The case will show up on home screen as having been allocated to

IWER

The next time the IWER logs in to the app online, the allocate case will

appear in their workload

Alternative course It should be possible to allocate cases to one interviewer at the start of

fieldwork and then to reallocate cases to another interviewer(s) part way

through fieldwork (see ALLC03)


Description IWER requests that an unproductive case be unassigned from them (i.e.

returned to office)

Actor IWER

Course of events IWER completes use case ATTM04

Locates case used in ATTM04 from list cases screen

Clicks on selected case

40

Clicks on “request to be unassigned”

IWER completes manual sync with CCMS and logs out

Outcome Unassign box visible for case within app should change to “Cancel

request to be unassigned”

On login to CCMS SA should receive notification that there has been a

request for case to be unassigned

(SA can then decide to grant request or not and for e.g. reallocate to

another IWER)

Until case has been un- or reassigned by agency, it should still show up

in IWER workload.


Description SA can reallocate a case to a different IWER part way through fieldwork

Actor SA

Course of events Use cases CLOG01 and ALLC02 completed

SA clicks on case to be reallocated

Clicks on IWER box and selects new IWER from dropdown list

Next time original IWER logs in to the app online the reallocated case

will not be visible in their case listing

Next time the new IWER logs in to the app online the reallocated case

will be visible in their case listing

Outcome Case is reallocated to a different IWER

Information of previous IWERS working on the case should be retained

within CCMS

New IWER should have access to details of previous contact attempts

as logged by the original IWER

Syncing app with CCMS (SYNC)

Use case identifier SYNC01

Description CCMS syncs with app: Interviewers can pick up their allocated workload

Actor SA

41

IWER

Course of events SA imports data into CCMS and allocates cases to interviewers (Use

cases IMPT01 and IMPT02)

SA logs out of CCMS

IWER logs into app whilst online (ILOG01)

Outcome Cases allocated to IWER in CCMS (and only cases allocated to the

IWER) should be visible within app

Alternative course Sync will only occur if the IWER has internet access when logging into the

app. IWERs will need to log in at home (or elsewhere) to make sure they

pick up the latest cases before going out into the field

Use case identifier SYNC02

Description App syncs with CCMS: Contact information entered into the app is

returned to the SA

Actor IWER

SA

Course of events IWER logs into the app (ILOG101) and uses the app to log contact

attempts for one or more of their allocated cases (ATTM)

(whilst online) IWER manually syncs the app with the CCMS using

“sync” function

Outcome IWER receives a “sync complete” message

The synced information should be visible to the SA within the CCMS

Alternative course Sync will only occur if the IWER has internet access. Whilst the app is

offline all data collected and entered will be saved and stored locally.

IWER can either sync manually or syncing will happen automatically the

next time the IWER logs into the app online

Managing cases within app (OVER)

Use case identifier OVER01

Description IWER can see an overview of all cases allocated to them to use in

managing their workload

Actor IWER

42

Course of events Data has been imported into the CCMS with some cases allocated to

IWER (Use cases IMPT01 and IMPT02)

IWER logs in online to the app (ILOG101)

Outcome Cases allocated to IWER (and only cases allocated to the IWER) should

be listed in the app

Alternative course Sync will only occur if the IWER has internet access


Description IWER can search for a specific case within the app

Actor IWER

Course of events IWER completes use case OVER01

Enters caseid (or other info about the case e.g. postcode) in the available

search box

Outcome App displays info for searched case only

Alternative course It must subsequently be possible to clear the search results and return to

the full case listing


Description IWER can filter cases by certain fields e.g. outcome code

Actor IWER

Course of events IWER completes use case OVER01

Clicks on filter icon

Chooses field by which cases should be ordered

Outcome App displays cases in selected order e.g. by batch number, postcode

etc.

Performing household and respondent selection wihtin app (SELC)

Use case identifier SELC01

Description IWER can complete household selection

43

Actor IWER (working a household/address-based sample)



IWER completes use case ILOG01

Clicks on “Perform respondent selection”

Enters number of households when prompted

(If more than 2 households) enters details to identify each household

Clicks “Perform household selection”

Outcome App performs a random selection and selects household

Info on selected household displayed to IWER and stored/visible to

IWER within app

Alternative course If only one household at address selection should still be performed but

IWER should not need to enter further info about the household

Use case identifier SELC02

Description IWER can complete respondent selection

Actor IWER (working a household/address-based sample)



IWER completes use case ILOG01 and use case SELC01

Clicks on “Perform respondent selection”

Enters number of eligible respondents in household when prompted

Lists names of eligible respondents

Clicks “Perform respondent selection”

Outcome App performs a random selection and selects a respondent

Info on selected respondent displayed to IWER and stored/visible to

IWER within app

Logging contact attempts within app (ATTM)

Use case identifier ATTM01

Description IWER can log date, time and mode of contact attempt

44

Actor IWER





Clicks on log a contact

Checks date and time of contact attempt (automatically generated by

app)

Enters mode of visit

Outcome Date and time display correctly

Date and time of last contact are updated in “case” tab and show up in

case history

Alternative course Interviewer should be able to start a contact attempt, then either:

a) Continue to log a contact attempt

b) Move to a new tab e.g. to perform respondent selection. When

IWER returns to case overview, data already entered on date,

time etc, should be saved

Interviewer should be able to manually edit date and time of contact

attempt within app (in event that they forget to log the visit on the

doorstep)

IWER should be able to log multiple contact attempts for each case


Description IWER can record the outcome of a visit as “address invalid”

Actor IWER







app)


Records outcome as “Address invalid” and reason why address is invalid

Outcome App follows routing of ESS Contact Form

45

Outcome of last contact is updated in “case” tab and show up in case

history


Description IWER can record outcome of contact as completed interview

Actor IWER







app)


Records outcome as “Completed interview”



history


Description IWER can record outcome of contact as refusal

Actor IWER







app)


46

Records outcome as “Refusal” and completes follow up questions from

ESS Contact Form re: reasons for refusal, likelihood of future

cooperation and estimates of target respondent’s age and gender



history


Description IWER can record outcome of contact attempt as “no contact”

Actor IWER







app)


Records outcome as “No contact”



history


Description IWER can make an appointment for a future contact attempt

Actor IWER






47


app)


Records outcome as “Appointment” and enters date and time of

appointment



history


Description IWER can enter new address information for a named target respondent

who has moved house

Actor IWER





Clicks on “edit address”

Enters new address

Answers follow up questions from ESS Contact Form on whether target

respondent has moved out of area and/or to an institution


Address details are updated within app

Other app functions

Use case identifier NBQ01

Description IWER can complete neighbourhood observations for selected case

Actor IWER





48

Clicks on “About neighbourhood”

Completes the observation questions

Clicks on “Save”

Outcome Order and wording of neighbourhood observation questions follows that

in the ESS Contact Form

Interviewer observations are saved within the app and visible to the

IWER throughout fieldwork

Alternative course Neighbourhood questions need only be completed once for each address

Use case identifier NOTE01

Description IWER can complete record notes about a case within the app

Actor IWER





Clicks on “Notes”

Types a note

Clicks on “Save”

Outcome Note is saved within the app and visible to the IWER throughout

fieldwork

Note will also be shared with SA via CCMS

Data export (EXPT)

Use case identifier EXPT01

Description CCMS user can export a case-level data set containing contact

information received via app for further analysis

Actor SA, NC, CST

Course of events IWER (s) have used the app to log some contact attempts and sync

collected data with CCMS

CCMS user logs into CCMS (CLOG01, CLOG02 or CLOG03)

49

Clicks on “Export” function within CCMS

Makes a selection of which fields/cases they wish to export data for

(default is all cases and all fields)

Submits export request

Outcome .csv file containing contact records for (selected) cases loaded into

FMMS is available for download and can be saved external to CCMS

Alternative course Data exported should be consistent with CCMS user rights

a) SA should be able to see all fields for cases within own country

b) NC should be able to see deidentified data for cases within own

country

c) CST user should be able to see deidentified data for all countries

Summary progress reports (REPT)

Use case identifier REPT01

Description CCMS user can obtain a summary of the current fieldwork situation

Actor SA, NC, CST

Course of events IWER (s) have used the app to log some contact attempts and sync

collected data with CCMS

CCMS user logs into CCMS (CLOG01, CLOG02 or CLOG03)

Clicks on “Summary report” function within CCMS

Views (and can download) a summary of the current fieldwork situation

(based on last contact attempt synced to CCMS from app) including

response rate and full breakdown of outcome codes

Outcome Summary progress report is visible within CCMS and can be

downloaded as a .csv file to be saved external to CCMS

Alternative course As well as downloading report on current fieldwork situation, user should

also be able to access reports from previous days/weeks of fieldwork

stored within CCMS

Reports should be produced on a per country basis. CST user should be

able to select a specific country for which to run a report

a fieldwork management and monitoring system … fmms_technical...h a fieldwork management and...

Documents