youress

8/3/2019 YourESS

1/43

Copyright 1990 2008 QUANTIFY! Ltd

Your

Employee Satisfaction SurveyBy David C Lusty CharteredMCIPD MMS(Dip) MIC CMC

You may print this document to read at your convenience but the .pdf contains many links; some to

other parts of the document itself, some to the glossary of terms on our web site, so we recommend

that if possible, you read this on a web enabled computer.

QUANTIFY London

18 Rodway RoadRoehampton VillageLondon

SW15 5DS

08452 41 41 60

[email protected]

QUANTIFY Yorkshire

4 West ParadeWakefieldWest Yorkshire

WF1 1LT

0845 241 3450

[email protected]

www.quantify.co.uk

COPYRIGHT NOTE: This is copyright material. We encourage you to share it with anyone you

know who may be interested but you may only do so by copying the entire document. Any other use

or reproduction of this document or any part of it without our written permission is a breach of our

copyright.

First published 2001

This edition 2006

Last revised 21 February 2011
mailto:[email protected]?subject=Your%20Employee%20Satisfaction%20Surveymailto:[email protected]?subject=Your%20Employee%20Satisfaction%20Surveymailto:[email protected]?subject=Your%20Employee%20Satisfaction%20Surveymailto:[email protected]?subject=Your%20Employee%20Satisfaction%20Surveyhttp://www.quantify.co.uk/http://www.quantify.co.uk/http://www.quantify.co.uk/mailto:[email protected]?subject=Your%20Employee%20Satisfaction%20Surveymailto:[email protected]?subject=Your%20Employee%20Satisfaction%20Survey

8/3/2019 YourESS

2/43


ContentsContents ................................................................ ........ 2About the author ........................................................... 4About the Book............................................................. 5

Kit components ......................................................... 5Information is Power .................................................... 6

Management Information ......................................... 7Financials .............................................................. 7People and their feelings ....................................... 7Information about peoples feelings ..................... 8Employee satisfaction survey ............................... 8Whats in a name? ................................................ 9

Plan the project ............................................................. 9Purpose ..................................................................... 9Benefits ................................................................ ..... 9

Checklist: Possible objectives of the survey ......... 9Cost ................................................................ ......... 10In house or outsourced? .......................................... 11

Checklist: Reasons to outsource parts of the

project ................................................................. 11Methodology ............................................... ............... 13

Web or paper ................................ .......................... 13Reason quoted ................................ .................... 13We say ................................................................ 13Checklist: Advantages of a web based survey .... 13Checklist: Disadvantages of a web based survey 14

Sample or census? .................................................. 14Checklist: Advantages of the Census approach .. 14

Develop / agree a questionnaire .................................. 15Reuse an old one? ................................................... 15

Checklist: Reasons for reusing an existing

questionnaire....................................................... 15Checklist: Reasons to develop a new questionnaire

................................................................... ......... 15Topic areas .............................. ............................... 16

Checklist: Possible survey topics ........................ 16The Development Process ...................................... 17

Checklist: An effective questionnaire ................. 17Focus groups ....................................................... 17Other preliminary research ................................. 18

Size ......................................................................... 18So What? ................................................... ............. 18Comments ............................................................... 19Classifications......................................................... 19

Checklist: Possible classification systems .......... 19Return address ........................................................ 21Pilot ............................................................... ......... 21

Pilot Procedure ................................ ................... 21

Design .................................................................... 21Translation .............................................................. 22Printing ................................................................... 22

Maximising response rate ........................................... 22Checklist: Maximising response rate .................. 23

Publicise the project ................................................... 23Checklist: Possible publicity routes .................... 23

Preferred completion route web or paper ............ 24Distribute questionnaires ............................................ 25

The questionnaire pack ....................................... 25The Covering Letter ................................. .............. 25

Checklist: The Covering letter ............................ 25Model text for a covering letter .............................. 25Closing date ............................................................ 26N members .......................................................... 27Business Reply Envelope ....................................... 27Home or work address? .......................................... 27

Checklist: Advantages of internal distribution ... 27Checklist: Disadvantages of internal distribution27

A Web Option? ....................................................... 28Access controlled, Confidentiality assured ........ 28Offer both routes and secure respondent

commitment in advance ...................................... 28Gather responses; Capture data; Analyse results ........ 30

Checklist: Reasons to outsource the processing to

QUANTIFY ........................................................... 30Progress report........................................................ 31Tabulation by location or department ..................... 31Analysing & reporting the results........................... 31

Quantitative results ............................................. 31Total percentage favourable ........................... 32Averages ......................................................... 33Percentiles ...................................................... 33Significance .................................................... 33Indices ............................................................ 34Importance ...................................................... 34Dont like numbers? ....................................... 34

Qualitative results ............................................... 34Anecdotal use ................................................. 34Analytical use ................................................. 35

Normative (Benchmarking) Comparisons .................. 35Caution ................................................................... 36

Expectation ......................................................... 36Wording .............................................................. 37Response frame .................................................. 37Sequence ............................................................ 37Conclusion .......................................................... 37

8/3/2019 YourESS

3/43


Take action .................................................... ............. 38Publish the results. .................................................. 38Agree initiatives ...................................................... 38Action ..................................................................... 38Measure again ......................................................... 38Add to your Management Information System (MIS)

................................................................ ................ 39Instruct us to help with your employee satisfaction

survey ................................................................ ......... 39When you instruct us, you get ............................ 39

You get .......................................... ................. 39So that ............................................................. 39

When you instruct us, YOU choose ............... .... 40

You choose ......................................................... 40

So that ................................................................ . 40QUANTIFY Menu of services ............................... 40

Employee Satisfaction Surveys .......................... 40Service Guarantee........................................... 41

More information ................................................... 42Other services ......................................................... 42

Employment Cycle Research ............................. 42Customer Satisfaction......................................... 42360 Feedback (Management Strengths) ............ 42

Off-the-shelf ................................................... 42Bespoke .......................................................... 43

Teamwork (Internal Customer Satisfaction) ...... 43Quantitative data from written remarks .............. 43

8/3/2019 YourESS

4/43


About the authorDavid Lusty began his career in Management Services, working for several local authorities before

moving into Personnel Management in 1974. After 11 years in local government,

he joined Avis Rent a Car Ltd in 1978, became head of the UK personnel

department in 1980 and was Director of Personnel and Management Services by the

time he left Avis in 1990 and founded QUANTIFY.

He is the author of several published articles about Employee Satisfaction Survey

research, an accomplished trainer in survey research methods and a sought-after

speaker at HR management and business performance events.

David is a Chartered member of the Chartered Institute of Personnel and Development, member of

the Institute of Consulting and the Institute of Management Services. He is a Certified Management

Consultant.

If you are reading the Portable Document Format (.pdf) version of this book on a computer

you can navigate using the bookmarks on the left, or links in the text or in marginal

comments like this one, set in green.

8/3/2019 YourESS

5/43


About the BookThis is the free version of the book. Our 80-page Employee Satisfaction Survey Research Kit edition

includes all the content of this version together with

Many enhanced and expanded sections including step by step instruction on Running a focus group to inform the development of the questionnaire Writing the questionnaire Analysing the results Assessing the significance of any differences you find Making normative (benchmarking) comparisons

Full instructions on using all the associated tools An Employee Satisfaction Survey Items (Questions) Bank including 1,700 separate questions in which you can

choose a subject area and see a number of model items (questions) you can use to measure how people feel

about it.

A template to use in MS Word to design a compact and professional questionnaire. A sample ready-made Employee Satisfaction Survey questionnaire for you to use or adapt A template for Excel comprising an easy-to-use survey data capture and analysis tool. A collection ofsimple statistical tools to help you answer the questions:

How big a sample do we need? What is a standard error, and why does it matter? Is the difference between twosubsetresults statisticallysignificant?

A normative table providing data on 51 core Employee Satisfaction Survey itemsKit components

YourESSRK.pdf; the Research Kit edition of this document, including full instructions for the othercomponents:

ItemsESSRK.xls; the survey items (questions) bank of 1,700 items Quantify.dot; a Word add-in providing a useful questionnaire editing toolbar QuaireRK.dot; the Word template for creating questionnaires Questionnaire.doc; a sample, ready-made questionnaire to use or adapt Analysis.xlt; the template for Excel for data capture, subset analysis and statistical tests Normstable.xls; an Excel workbook providing normative data to put your results in context

The Kit costs 500 plus VAT and is delivered on CD-ROM. If you would like to order a copy,

please [email protected] call us on 020 8704 1296.
http://www.quantify.co.uk/resources/glossary.php#Subsethttp://www.quantify.co.uk/resources/glossary.php#Subsethttp://www.quantify.co.uk/resources/glossary.php#Subsethttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancemailto:[email protected]?subject=Order%20for%20your%20Employee%20Survey%20DIY%20Toolkitmailto:[email protected]?subject=Order%20for%20your%20Employee%20Survey%20DIY%20Toolkitmailto:[email protected]?subject=Order%20for%20your%20Employee%20Survey%20DIY%20Toolkitmailto:[email protected]?subject=Order%20for%20your%20Employee%20Survey%20DIY%20Toolkithttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Subset

8/3/2019 YourESS

6/43


Information is PowerHave you ever realised that something about the way your organisation isnt working as it should?

Perhaps the way things are currently arranged means that employees feel taken for granted, or

overworked and underpaid and that they might be better off with a different employer.

But when you suggest some change to deal with the problem you have identified, the response is

How do you know theres a problem? If all you have to go on is your judgement and one or two

specific employees whose cases you can quote, it is hard to convince others of the need to change

the way things have been done for the last so many years. After all, it seems to work OK and were

no worse than anyone else, are we? And each of your anecdotes represents a sample of one.

Wouldnt your arguments carry much more weight if you were able to back them up with objective

data demonstrating the problem?

An employee satisfaction survey will provide you with that evidence; management information

about how it feels to work in your organisation, in this department compared with that one; for this

manager compared with that one. It will show where the arrangements work well, and where they

need to be improved.

A well informed manager is an influential manager. An Employee Satisfaction Survey will

provide you with information which will give you the power to bring about change for the

better.

This book is all about how to conduct the survey in the most effective and economical way.

8/3/2019 YourESS

7/43


Management Information

Financials

Most organisations have a business plan of one kind or another and a way of comparing what

happens in reality with what had been planned. Often the plan for this year has been derived from

the actual performance a year earlier, and is expressed as a percentage improvement on prior year. A

system of reporting provides management with (fairly) up-to-date information on actual results,

usually month by month, with comparisons versus plan and prior year.

This Management Information System will include a line for each source of revenue, all added

together to arrive at total revenue; then a line for each cost item followed by total costs and theconsequent profit or loss. Often, there are analysis lines showing revenue per unit or cost per unit

what the unit is depends on the nature of the business. There may be other statistics which help to

pinpoint the reasons for any difference between actual performance and the plan or prior year figure.

Hotels measure their occupancy rate the percentage of available rooms which were occupied; car

rental companies measure utilisation the percentage of available cars which were rented out. There

may be employee cost statistics, too; say employee cost per guest night, or employee cost per rental.

In 99% of organisations, all these numbers have one thing in common. They are based on money or

transactions, or other data which is gathered necessarily in the process of finding and serving

customers, billing them and getting paid; finding, hiring and paying employees. These reports, often

referred to as the financials are the main tool by which the enterprise is managed and yet they only

include numbers which happen to be available anyway in the business system. Is that all theinformation needed to manage a modern enterprise?

People and their feelings

People, and the way they feel about what they do profoundly affect the effectiveness of an

organisation and its success in selling its goods and services. If your employees care about the

success of the business, if they are proud of the quality of its outputs and their part in creating them,

the business is likely to flourish. If they regard their employer as an adversary, an exploiter who

deserves no loyalty or effort from them, the business will suffer. Similarly, if customers trust the

organisation, like its products and have good feelings about their contact with it they will be easier

to sell to again and again than if they dread the next time they may be forced to deal with it.Peoples feelings affect the bottom line just as much as ingredient cost per cover or energy cost per

unit. But the data doesnt exist anywhere in the financial system, so most organisations are managed

by watching all the measures which are easy to count, rather than all the measures which matter. It is

a bit like steering a supertanker in a fog with a big chunk of the radar screen covered up.

If you really mean it when you say Our people are our greatest asset, why does your management

information system tell you so little about this crucial asset and how they feel?

8/3/2019 YourESS

8/43


Information about peoples feelingsThere are two main ways of finding out how people feel about things. You can study what they say,

or observe what they do.

Getting someone with suitable training to observe and report on all your people isnt practical for a

number of obvious reasons.

You might get some useful information by talking to employees, or more to the point, listening to

them. Managers should be doing this all the time. But depending on the individual managers

relationship with them, employees with a criticism to express, however constructive, may hold back

for various reasons:-

Risk of offending the manager, or incurring his or her wrath Risk of being perceived as a troublemaker, or negative Feels a bit wet to complain e.g. about not getting pats on the backEven if people do open up to their managers, you cant express what they learn in numbers which

could be used as a management statistic, and even if you could, the individual interpretation each

manager put on what they had found out would make the measure too subjective and inconsistent.

What you need is a consistent, objective way of measuring employees feelings.

Employee satisfaction survey

A survey can gather information using a consistent set of questions, allowing employees to remain

anonymous, and thereby freeing them to tell the truth. It can produce a single measure of satisfaction

overall or you can extract various indices including:-

Engagement Commitment Satisfaction with

Working conditions Pay and benefits

Training & development CommunicationThere is a more comprehensive checklist of topics you might want to include in a survey under

Checklist: possible survey topicsbelow.

Whatever measures you decide you want, they can be calculated for the whole organisation as well

as separately for each location, department, and cost / profit centre (to coincide with the way the

financials are produced. You probably shouldnt attempt to measure monthly, like the financials.

An annual main survey sent to every employee can be supplemented by quarterly tracking sample

8/3/2019 YourESS

9/43


surveys using just a few key questions, so you can update the employee feelings measures in theManagement Information System quarterly.

Whats in a name?

Perhaps a rose by any other name would smell as sweet, but the name you give your survey will

affect the way it is perceived. If your organisation makes a distinction between staff, who work in

offices, and other workers, who dont, you wont want to call it a staff survey, because some

employees will feel it doesnt apply to them.

The traditional description attitude survey isnt ideal. The word attitude has acquired negative

associations, and it probably isnt wise to suggest that you think your employees have an attitude.

Especially in an organisation which emphasises the importance of customer satisfaction, why notgive the message that employee satisfaction is important, too, by calling the employee survey a

satisfaction survey, even if some of the measures will be about more than satisfaction? The word

employee is a bit clunky, but at least it is all-embracing so unless you have a better word which

takes in everyone, we suggest

Employee Satisfaction Survey Or use your organisations word for your people

Colleague Satisfaction Survey Partner Satisfaction Survey Team Member Satisfaction Survey

Plan the project

PurposeBefore you begin developing a questionnaire or deciding on a methodology, you should be clear

about your reasons for carrying out the survey. How will you use the data you get out of it? What

benefit do you expect it to produce and will the benefit exceed the cost? Costs and benefits might

both be difficult to predict but you should at least have a rough idea.

BenefitsSpecify the purpose or purposes for which the employee survey is to be run.

Checklist: Possible objectives of the survey

Improve communication Demonstrate managements

commitment to listening to employees

views

Discover aspects of communicationemployees see as failing

Provide management information onwhich to base action to

8/3/2019 YourESS

10/43


Improve employee performance /productivity

Improve employee retention Reduce employee sickness and other

absence

Improve quality of service Speed Courtesy Accuracy

Improve customer satisfaction

Study the correlation betweenemployee satisfaction measures and

customer satisfaction employee performance / productivity employee costs bottom line

Monitor the effect of change Reorganisation Acquisition / merger

Required by external authority orparent company Government or other body Group requirement

Required for accreditation ISO quality accreditations Investors in People

Fulfil commitment made to TradeUnion or other employee representative

body

CostWe are often asked How much does it cost to run an employee satisfaction survey? And of course,

there is no single answer to that question. It depends on the answers to a long list of questions,

including

How many people do you employ? How many do you plan to include in the survey i.e. will you run acensussurvey, including

everyone, or a sample, in which only selected people receive a questionnaire?For a discussion of the census or sample issue, seeSample or censusbelow.

How many questions will there be in the questionnaire? Precoded ones can be answered with a tick in one of several boxes you provide, and are

relatively cheap to process

Free text prompts, where you ask people to write in an answer are much more expensive toprocess

Do you already have a satisfactory questionnaire you can reuse, or adapt slightly, or will youhave to develop one from scratch?
http://www.quantify.co.uk/resources/glossary.php#Censushttp://www.quantify.co.uk/resources/glossary.php#Censushttp://www.quantify.co.uk/resources/glossary.php#Censushttp://www.quantify.co.uk/resources/glossary.php#Census

8/3/2019 YourESS

11/43


As a rough guide, you should reckon to spend between 500 and 5,000 on getting a consultant tohelp you develop a new questionnaire, depending on the complexity of what you want to measure,

and the rigour with which you decide to conduct the preparatory research and the testing of the draft

questionnaire.

Spending more than this might further improve the questionnaires validity (ability to measure what

it sets out to measure) or reliability (ability to measure what it measures consistently each time you

use it) but the law of diminishing returns applies. Is it worth spending twice as much to get a

questionnaire which is 5% better? You can reduce the consultants fee by carrying out much of the

work yourself but you would be wise to get some experienced, professional input at some stage, to

avoid falling into the many pitfalls which lie in wait for the inexperienced questionnaire author. You

may find that the cost of your time to do the work will actually exceed what you would have paid aspecialist.

Once you have a questionnaire, allow about 2 per head to cover the cost of distributing, gathering,

analysing and reporting the results of your survey. This is a very rough rule of thumb and the actual

cost may be more or less depending on the size of the questionnaire and the complexity of the

analysis you will require.

Finally, you should allow for feeding back the results to management and employees and for taking

action to initiate change which may be indicated by the results of the survey.

As you proceed with the planning of the project and you know the answers to more of the questions,

you or your external supplier will be able estimate the cost of the project more accurately.

In house or outsourced?Much of the work involved in running a survey can perfectly well be done in house, if there is

someone who can devote the time required. You may have the capacity and the skills in house to

handle

Planning the project Drafting the questionnaire Printing, mailingOther jobs are probably best done by a specialist and involving an independent external professional

provides some advantages you cant get any other way.

Checklist: Reasons to outsource parts of the project

Know-how Questionnaire design. A specialist will help iron out any wrinkles in your draft questionnaire

8/3/2019 YourESS

12/43


Systems. A specialist has systems and procedures all worked out which anyone in housewould have to figure out as they go along so theyll take much longer, which might cost more

than the specialist would have billed

Experience. A specialist will have conducted many surveys and knows what works and whatdoesnt

Web platform. An external supplier might offer the option to publish your survey on the webWe do!

Professionalism. You might not have the expertise in house to handle the technicalities ofsampling, or assessing thesignificanceof results

Instant questionnaire. An external supplier may offer an off-the-shelf questionnaire you canuse at lower cost than is involved if you develop one of your own

Confidentiality. Some employees will fear that an outspoken comment on their part mightlead to unpleasantness if it becomes known to one of their bosses. So people will be much

more willing to give full, frank input if you provide an independent

person to facilitate focus groups for developing a questionnaire destination to return survey responses to facility to analyse the data

Normative data. Your external supplier might be able to put your survey results in context bymaking comparisons with data gathered from other employee surveys

Authority. When communicating the results, and convincing colleagues that some change isrequired, an external professionals view often carries more weight than anyone in house.

This means it is often best to use outside help to handle

Focus group facilitation Questionnaire drafting and design Response handling, data capture

Results analysis Interpretation and presentation of

conclusions

Some suppliers might expect to take over the whole project as a package. Most will be willing to doso if that is what you require but you might want to find a supplier who, like QUANTIFY, will be

pleased to take on any parts of the project you cant handle, or dont choose to, while letting you

stay in control and handle the rest.
http://www.quantify.co.uk/resources/glossary.php#Samplinghttp://www.quantify.co.uk/resources/glossary.php#Samplinghttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Sampling

8/3/2019 YourESS

13/43


Methodology

Web or paperOrganisations are increasingly using the web for surveys. The main reasons people give are listed

below with our view of them.

Reason quoted We say

More reliable distribution

channel

Maybe, depending on the way you choose to distribute paper

questionnaires.

You can have response rate

updates while the survey is

live

We provide daily progress reports whether clients choose paper or

web distribution

Eliminates manual keying

from questionnaires

True, but extra set up costs might outweigh the saving. You probably

need to cut out the data capture for at least 1,000 responses to recoup

the extra costs involved in setting up your survey on the web.

Saves paper True, it might save some trees but dont assume that it will save you

money.

Allows quicker reporting

when the survey closes

If paper responses have been returning separately to us during the life

of the survey, they are almost all keyed by the time you decide to

close it, so reporting is just as quick either way.

Facilitates feedback to

employees

You can put your survey results on the web or your intranet if you

wish, regardless of how the data were gathered.

It is the current fashionable

way of doing things

True. Some of your people might look down their nose at a paper

questionnaire, these days.

We think the web approach is neither better nor worse than paper, just different. That means it may

suit some organisations but not others, so what matters is which is suitable for you.

We suggest that there can be a very significant advantage in using both routes. See A web

option?

Checklist: Advantages of a web based survey

Eliminates the need to print all those paper questionnaires Many employees prefer it, so might enhance response rate Reduces the data keying workload

8/3/2019 YourESS

14/43


Modern, quick and green.On the negative side, although we save on postage and the data keying we don't have to do, the extra

set up work together with the web hosting cost might exceed the saving. You cant fit as many

questions on one screen as you can on one page of a paper questionnaire, so there are more virtual

page turns involved in completing a questionnaire on the web than on paper and each one takes

longer, even with the fast software and server we use. Also, people mistrust the web, believing that

everything they do or view can be traced. If access to your survey requires the use of a key or login,

people will take it that you can pick out their individual response.

Checklist: Disadvantages of a web based survey Often more expensive than paper and pencil May heighten employees concerns about their confidentiality Takes longer to complete on the web than on paper Requires a PC so

People cant do it on the bus or the train on the way to or from work Some people wont be comfortable using a PC Not suitable for people with no access to a PC

Sample or census?Should you give every employee the opportunity to participate (census), or only ask some

employees (sample)?

Most employee surveys adopt the census approach, which has the following advantages:

Checklist: Advantages of the Census approach

Bigger sample size makes results more convincing Adequate sample sizes even when the results are sliced up intosubsetsto compare one group

with another. A sample survey might not provide statisticallysignificantdata on specific

subsets

Nobody feels excluded, or receives the message that their views dont matterA sample approach might be appropriate to

Preventsurvey fatigue, particularly if you decide to measure employee satisfaction quarterly.If everyone gets a questionnaire to fill out every three months, they will soon get fed up with

it andresponse ratewill decline. You may choose then to conduct an annual census using a
http://www.quantify.co.uk/resources/glossary.php#Censushttp://www.quantify.co.uk/resources/glossary.php#Censushttp://www.quantify.co.uk/resources/glossary.php#Censushttp://www.quantify.co.uk/resources/glossary.php#Samplinghttp://www.quantify.co.uk/resources/glossary.php#Samplinghttp://www.quantify.co.uk/resources/glossary.php#Samplinghttp://www.quantify.co.uk/resources/glossary.php#Subsethttp://www.quantify.co.uk/resources/glossary.php#Subsethttp://www.quantify.co.uk/resources/glossary.php#Subsethttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Surveyfatiguehttp://www.quantify.co.uk/resources/glossary.php#Surveyfatiguehttp://www.quantify.co.uk/resources/glossary.php#Surveyfatiguehttp://www.quantify.co.uk/resources/glossary.php#Responseratehttp://www.quantify.co.uk/resources/glossary.php#Responseratehttp://www.quantify.co.uk/resources/glossary.php#Responseratehttp://www.quantify.co.uk/resources/glossary.php#Responseratehttp://www.quantify.co.uk/resources/glossary.php#Surveyfatiguehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Subsethttp://www.quantify.co.uk/resources/glossary.php#Samplinghttp://www.quantify.co.uk/resources/glossary.php#Census

8/3/2019 YourESS

15/43


comprehensive questionnaire, and in the three other quarters sample a different 1/3 of thepeople using a short questionnaire with just the key tracking measures.

Reduce cost.The cost reduction available by sampling might not be as much as you imagined, because a census

can be distributed in batches and managers asked just to give everybody a pack (questionnaire,

covering letter and reply envelope). A sample needs to be selected from an employee list, and

personally addressed, so the distribution process is more expensive. All the same, there may be a

saving to be made so if you employ a large number of people, and are satisfied that you can deal

with the perception of those excluded, sampling might be worth considering.

Develop / agree a questionnaire

Reuse an old one?You may have a questionnaire already, which has been used for previous surveys. Will you reuse it

or develop a new one?

Checklist: Reasons for reusing an existing questionnaire

Ability to compare with previous results to show improving / declining trend Avoids cost of developing new questionnaireChecklist: Reasons to develop a new questionnaire

This is to be the first survey conducted Old questionnaire doesnt gather required data Old questionnaire is poorly designed

Doesnt address the current issues Asks two or more questions in one, so staff cant respond to specific points Uses inappropriateresponse frames Finds out problems but gets no clue to solutions Staff cant understand questions, or give meaningful responses Asks about things which cant or wont be changed

Old questionnaire is simply out of dateIn practice, it is often possible to retain sufficient elements of an old questionnaire to provide for

trend comparisons whilst incorporating new items to keep the questionnaire topical, relevant, fresh

and interesting.
http://www.quantify.co.uk/resources/glossary.php#Responseframehttp://www.quantify.co.uk/resources/glossary.php#Responseframehttp://www.quantify.co.uk/resources/glossary.php#Responseframehttp://www.quantify.co.uk/resources/glossary.php#Responseframe

8/3/2019 YourESS

16/43


Topic areasYou need to decide what you need to measure. The following checklist is fairly comprehensive, but

if you try to include all these topics you will probably end up with too big a questionnaire.

Try to group the questions you ask under several different headings, each of which covers one

aspect of the service you deliver. Then, as well as asking employees how satisfied they are with each

area, you can ask how important each of them is to people.

Your employees may be very dissatisfied with some aspect of their experience at work but if they

regard that aspect as unimportant, this might not matter as much as another area which people are

not dissatisfied with to quite the same degree, but which they see as very important. Before you

invest effort and resources in dealing with an issue, it is best to know how important it is to yourpeople as well as how satisfied or dissatisfied they are about it.

Checklist: Possible survey topics

Communication Teamwork / relationships

with Manager with Colleagues

with other Departments Management / supervision

Strategic direction and leadership Leadership / Management skills Recognition / motivation Technical competence Management of change

The job Variety

Authority Responsibility Stress Sense of achievement Job satisfaction

The organisation Culture Equal opportunities Job security

Career progression

Learning and development Terms and conditions

Working environment Compensation Benefits

Fulfilment of Personal goals Attitude to quality Attitude to customer satisfaction Identification with Organisation & its

goals Commitment Engagement

Confidence in the survey process andits outcomes

8/3/2019 YourESS

17/43


The Development ProcessPeople often think that anyone can write a questionnaire and of course they are right, anyone can.

Not everyone can write one which works, though. One which works fulfils the following criteria.

Checklist: An effective questionnaire

People fill it in and return it People understand the questions the way you intended It covers the topics

you want to measure employees want to report on

Peoples ticks in boxes represent their responses to the questions i.e. they dont tick a box youare interpreting as meaning something different from what they intended

The responses can be summarised in a meaningful way The summarised results

diagnose specific problems help you decide what needs to be changed

Most employee surveys are conducted on paper or on the web using self-completion questionnaires,the most exacting kind to prepare, because there will be nobody available to explain what was meant

if an informant misinterprets a question.

Writing questionnaires which work requires training and experience and comprehensive instruction

in questionnaire writing would fill a small book, but not this one.

At QUANTIFY we run a one day seminar for clients who want to develop their own

questionnaires. It covers the basic dos and donts and will equip someone who already has a

good command of English to produce competent questionnaires.

Focus groups

An invaluable part of the process of developing a questionnaire is to get some input from the

proposed target group. A good way to do this is to conduct one or more meetings where a

reasonably representative group of employees can talk freely about the issues to be included in the

survey.

Employees are less likely to talk freely if they think that their boss might hear about any criticism

they offer. If people think they might suffer as a consequence, or be challenged and expected to

justify any comments, they will most likely keep quiet. We find that focus groups facilitated by an

independent person from outside the organisation, with no employer representative present, work
http://www.quantify.co.uk/services/training-seminars.phphttp://www.quantify.co.uk/services/training-seminars.phphttp://www.quantify.co.uk/services/training-seminars.php

8/3/2019 YourESS

18/43


best. The outputs from focus groups often lead to things being included in questionnaires whichwouldnt have been there otherwise. This avoids producing a questionnaire which asks questions

about lots of issues excepting the one people most want to get a message to management about.

The people involved in the focus group can also become advocates of the process, encouraging their

colleagues to participate.

The Research Kit edition of this book has a 4 page section here on Running a focus

group including who should participate, choosing a venue and facilitator and a step-by-

step guide to handling the meeting.

Other preliminary research

It may be impractical for a number of reasons to bring employees together to run focus groups. Inthis case, the preliminary research to inform the development of the questionnaire may be carried

out via individual interviews, face to face or on the telephone.

SizeThe bigger the questionnaire, the greater is the risk that people will put it to one side, or in the bin.

On the other hand, a questionnaire with only three questions will cost much the same to print and

distribute as a bigger one so you might feel that you could get more information for your money by

using a bigger questionnaire. In practice, you probably want to ask about a wide range of issues, so

you wont have trouble thinking of more questions; you are more likely to have a problem keeping

the size down to avoid putting people off.

The Research Kit edition of this book has a 10 page section here on Writing the

questionnaire including; using the Research Kits items bank; brevity; use of language;

eliminating wasted words; leading questions; positively and negatively expressed items;

response frames, odd or even number of options in a rating scale and more.

So What?When you have assembled a list of proposed questions, look at each one and write down the answer

to this question.

If the response to this question is very unfavourable what would you do about it?

If the answer is that management would be unwilling to change anything in response, consider

removing the question because asking it will suggest to your employees a willingness to

contemplate change, and lead to frustration when the point is made in the survey results but no

change occurs. If it was an issue raised by employees in the focus groups, it is evidently important to

them. You need to have a very good reason for deciding that nothing can be done about it and you

need to convince the focus group members that it is a good reason, too. Otherwise, they might lose

all faith in the process and instead of being advocates they might become opponents of the survey.

8/3/2019 YourESS

19/43


If the answer to the question is that you wouldnt know what you could change to improve thesituation, consider whether the question is specific enough. You may not be able to make it any

more specific, in which case, you may have to compromise and include it as a diagnostic tool, a

question which might reveal a problem which you would then need to investigate further.

CommentsMost of the questions should allow response by ticking one of several boxes but you may want your

questionnaire to include prompts which invite free text response. Sometimes some space at the end

of the questionnaire headed Please add any further comments here will be all that is required.

Getting anything useful out of the answers is much more time-consuming and therefore much more

expensive than analysing ticks in boxes, though. There are two main ways to use the responses Anecdotally. When the survey results are available, and you are advocating some kind of

change, as well as presenting the results drawn from the tick-box answers, you quote some of

the remarks employees made. Often, they are pithy and persuasive, and you can point out that

it isnt you saying this, it is the employees.

Analytically. By paraphrasing and categorising every remark and counting how many thereare in each category, and how many expressing each single general sentiment, you can see

which thought was expressed most often.

If you are tempted to include many open questions, is that because you dont know what the issues

are, that employees are bothered about? You may be able to find out more by conducting focus

groups, if you havent already done so, and make sure that the questionnaire already covers the keyconcerns employees want to communicate to you.

The extra cost of developing a questionnaire with fewer free text prompts will give you a saving at

the analysis stage. The greater the number of participants in your survey, the bigger the analysis

saving will be. An external supplier should be able to advise on the trade-off here.

ClassificationsThe results will be more valuable if you provide for comparisons between different groups of

people, areas, departments etc. This requires that you include on the questionnaire some items where

you ask people to place themselves in one of severalcategoriesyou offer.

Depending on your organisation and the subdivision of results which would be helpful in

interpreting the messages the survey provides, you may want to ask employees to tick one of several

options you provide under some or all of the following headings.

Checklist: Possible classification systems

Department Location
http://www.quantify.co.uk/resources/glossary.php#Subsethttp://www.quantify.co.uk/resources/glossary.php#Subsethttp://www.quantify.co.uk/resources/glossary.php#Classificationsystemhttp://www.quantify.co.uk/resources/glossary.php#Classificationsystemhttp://www.quantify.co.uk/resources/glossary.php#Classificationsystemhttp://www.quantify.co.uk/resources/glossary.php#Classificationsystemhttp://www.quantify.co.uk/resources/glossary.php#Subset

8/3/2019 YourESS

20/43


Job type / function Job grade / level Age group Length of service

Gender Ethnicity Employee / Contractor / Implant Working hours

These classifications are generally, if loosely, known as demographic items.

The department or location split can be very useful because it often divides people up according to

the manager they report to. You could do this explicitly, of course, by listing the managers and

asking people to tick the one they report to. When you get the results, you might well see somestriking differences between the groups managed by different people and the information will allow

you to identify the aspects of different managers styles which work well, and those which dont,

and target development activity accordingly.

For a more specific way of providing feedback to managers on how their style is perceived by their

team and others, a360 Feedbackor Management feedback system will be helpful. Find more about

these on our web site at www.quantify.co.uk.

Some employees will be anxious, despite any promises to the contrary, that classification questions

may be used to identify them, so you may be tempted to keep the classification systems as wide as

possible and to skip some altogether. This is to deny yourself the chance to use the survey results to

their best advantage, however, so it is a decision which should not be taken lightly.

The first time you conduct a survey using a new questionnaire, the results dont mean much by

themselves. Whatever the pattern of responses to a question, you probably wont know if the overall

result is good, bad or indifferent. You may have access to some normative comparisons for some of

the questions, but probably not for all of them.

The most useful data you can get out of any of the questions will be to see where one group of

employees is significantly happier, or less happy about the issue than another. This allows you

to identify areas where more effective practices are in place, which you may be able to identify and

replicate elsewhere, and find the key areas which would benefit from improvement, whether it

concerns people in a particular department, with a specific age, or length of service range, of a

specific gender or ethnicity, or whatever it is. Without the classification questions in thequestionnaire, you cant do the analysis, so you will never know.

Even if you include no classification questions at all, the cynical employee who doesnt believe your

assurances that the process is anonymous and confidential will still be convinced that each

questionnaire has an invisible or disguised identity mark on it so that you can see how specific

people replied.

It is wise to put the demographics section at the end of the questionnaire. This way, someone who

has already invested the time required to fill out the rest of the questionnaire might decide not to
http://www.quantify.co.uk/surveys/360-appraisal.phphttp://www.quantify.co.uk/surveys/360-appraisal.phphttp://www.quantify.co.uk/surveys/360-appraisal.phphttp://www.quantify.co.uk/surveys/360-appraisal.php

8/3/2019 YourESS

21/43


answer these questions but they will probably still send it back. So we can at least include it in theoverall results. If the same person encountered these questions at the beginning of the questionnaire,

they might just throw it in the bin.

Return addressPeople lose reply envelopes, or mix them up with other mailings they happen to have lying around

at the same time, so even though you plan to provide a reply paid envelope, print a reply address on

the questionnaire, too. Use a FREEPOST address independent of your organisation if you can.

Pilot

If a newly developed questionnaire includes many questions that havent previously been used inother questionnaires, it may be worth conducting a pilot exercise. This takes time, and costs money,

adding to the development cost, so the more people you plan to include in the survey the easier it

becomes to justify the cost of a pilot.

Pilot Procedure

Choose a small but reasonably representative group of employees. If you used focus groups to help

develop the questionnaire, some of the focus group members would be a good group to use for the

pilot. Ask them to complete the draft questionnaire, then gather their feedback on the questionnaire.

At this stage, they should understand that you are not gathering their answers to the main survey,

just testing the questionnaire.A good way to gather feedback is to meet up to six pilot targets at a time and work through the

questionnaire line by line asking them to say how they understood the instruction, or question, how

they went about responding, what the appearance of the questionnaire suggested to them. You are

looking for any indication that the questionnaire might be interpreted in any way you hadnt

intended.

Encourage them to raise any different interpretation they think is possible. Then you can amend any

ambiguous item to ensure that as many people as possible will understand it the way you intended.

Design

As well as the content, the appearance of the questionnaire should be fresh and appealing. Manypeople dont read the instructions, if any, so the design must make it obvious how they should

complete the questionnaire.

The physical size of the resulting document is important. Even if you decide you want 100

questions, which is really a few too many, it is possible to fit them on four sides of A4.

Alternatively, you could space them out more and make a pretentious booklet of 32 sides. The

version on four sides of A4 (printed on A3 and folded to make a nice neat booklet) is much more

likely to be completed and returned, though, because the thud the 32 pages make when they hit the

desk makes it seem like a much bigger chore to fill it out.

8/3/2019 YourESS

22/43


The Research Kit edition of this book has a 6 page section here on Designing thequestionnaire including; using the Research Kits Word template for questionnaire

design; inserting a logo; headings,; items and sub-items; creating neat response frames;

arranging demographic items; fitting it in .

TranslationIf the questionnaire needs to be translated, now is the time to do it. To facilitate processing, it is best

to keep the layout and page breaks the same in all versions, so that the equivalent question is always

in the same place, on the same page. Make sure the translators dont reverse the headings on the

response scales!

PrintingFor a paper questionnaire, printing and production should be to a standard appropriate to the

importance of the project but not extravagant; in keeping with your house style, but distinct from

any literature which requires no response. Paper a bit heavier (say 100gsm) than normal office paper

(usually 80gsm) invests the survey with a little bit more importance. Make sure that the paper can be

written on with ball-point or fountain pen. Glossy paper sometimes resists any attempt to write on it,

which defeats the object of the exercise, however pretty the questionnaire looks.

Maximising response rateDo everything you can to encourage people to participate. The response rate you should expect

depends on the nature of the organisation, the work people do and the relationship they have with

the organisation. Casual catering staff might not be as ready to respond as dedicated full-time

professionals, for example. As a rough rule of thumb, an organisation where people are interested in

their work and identify with the employers corporate objectives should get 70% to 80% response.

Less than 50% almost always indicates a serious breakdown between employees and their employer.

Some employers actually get all their people together (in groups, not all at the same time) and ask

them to complete the questionnaire then and there. Usually they provide everyone with an envelope

to seal their completed questionnaire in, and a bag, or tray to put it in to be parcelled up forprocessing (i.e. to be sent off to us). This certainly achieves a high response rate but some people

think it at the expense of honesty, because employees might imagine that someone will see their

questionnaire, or even open it to look at before sending it for processing. This practice contravenes

the Market Research Society Code of Conduct for Employee Satisfaction Surveys and we do not

advocate putting this level of pressure on informants.

Achieving a good response rate is not just about the administration phase. There are things you can

do right through the project, and after, which affect response for this survey or the next one.
http://www.quantify.co.uk/resources/glossary.php#gsmhttp://www.quantify.co.uk/resources/glossary.php#gsmhttp://www.quantify.co.uk/resources/glossary.php#Responseratehttp://www.quantify.co.uk/resources/glossary.php#Responseratehttp://www.quantify.co.uk/resources/glossary.php#Responseratehttp://www.quantify.co.uk/resources/glossary.php#Responseratehttp://www.quantify.co.uk/resources/glossary.php#gsm

8/3/2019 YourESS

23/43


Checklist: Maximising response rate Involve representatives of the target

group in the project

Publicise the survey in advance Make sure the content is relevant to the

people you ask to respond

Produce the questionnaire to anappropriate quality

Make its size manageable; notintimidating

Promise confidentiality

Allow employees to complete in worktime if they wish

Provide free postage for return Provide an independent external agency

to handle responses confidentially

Set a suitable time limit Issue a reminder

Promise a donation to charity or thelocations employee social fund for

each response receivedAnd if you decide not to guarantee anonymity

Promise entry in a prize draw for eachresponse received

Promise a voucher or gift for eachresponse received

Publicise the projectHave you ever received correspondence telling you that you would shortly be receiving six numbers

in a Readers Digest prize draw? You probably threw it in the bin but it achieved what it was

intended for. It created expectation. No matter that you didnt cancel all appointments for the next

week and sit in waiting for the postman, when the real mailing arrived, you were expecting it.

Professional direct marketers like Readers Digest dont do things like this for fun; they do it

because it enhances response rate. You can do the same. The letter can be based on the model

covering letter shown below but you can afford to go into a bit more detail, especially about the

reasons for conducting the survey and what you plan to do with the results.

If there is an in house employee newspaper, run an article in it about the survey. Include the points

set out in the checklist for a covering letter below, but take the opportunity to go into more detail

than you can in a covering letter, which needs to be brief, and to the point. If there have been focus

groups as part of the questionnaire development process, you could interview one or two of the

focus group members, not about the input they made to the meeting that is confidential but about

the process. If you choose well-known people whose enthusiasm for the process will be respected by

their colleagues, the article should leave readers more likely to complete and return their

questionnaire.

Checklist: Possible publicity routes

8/3/2019 YourESS

24/43


Articles in corporate Newsletter /magazine

Feature on in house radio / TV Note enclosed with pay-slips Email messages

Personalised mailing from ChiefExecutive / Head of HR

Topic in team briefings / other meetings Feature on the intranet web pages Posters

Preferred completion route web or paperIf you are making the survey available on the web, this provides an opportunity to go beyond

creating expectation and get some active buy-in from most employees.

We describe the procedure for web surveys in more detail at A Web Option? below.

To run a survey on the web, you need to address email invitations personally, and they need to

include a unique access code for each participant. This prevents anyone who finds the web site by

accident from completing your survey, and means that the people you want to complete it can only

do so once. In most organisations, there are still some people who cant, or who prefer not to

complete a survey on the web. So if the main survey takes place on the web you probably will still

need to provide paper questionnaires to some people. Make an asset of this by writing to every

employee before the survey is published; tell them about the survey and when it will be out, and ask

them to respond using a tear-off slip, or by email etc. to say which route they prefer to use tocomplete the survey. This clearly involves more work and expense. The mailing needs to be

prepared and the responses recorded so that the survey administration itself can be divided

according to the preferred completion route nominated by each employee.

Some people will fail to respond to the first mailing so you need to decide on a rule to decide which

route people will be allocated to by default, failing a response indicating their preference. Perhaps

anyone for whom you have an email address on file would default to web completion, others to

paper. Or you may want to set up a different rule. The result is that everyone has had the Readers

Digest style advance notice and most people have bought into the process by saying which way they

prefer to complete the questionnaire.

You noticed that we didnt offer the option of not filling it in at all. This is what sales people call analternative close. Asking Would you like the green one or the blue one? or Would it more

convenient for me to call on you on Tuesday or Thursday often gets the sale, or the meeting where

would you like to buy one or can we arrange a meeting wouldnt.

8/3/2019 YourESS

25/43


Distribute questionnaires

The questionnaire packEach participating employee should receive a pack comprising

A covering letter The questionnaire

Abusiness replyenvelope

The Covering LetterWe are aiming for clarity and maximum response rate, so the covering letter should be only as long

as it has to be to cover the points in the following list

Checklist: The Covering letter

Confidentiality results aggregated forN employees or more

External analysis, individual responseswill not be seen by anyone in the

Company / organisation

Quick and easy to complete

Response is post paid Promise publication and action on

results Closing date Urgent and important do it now

Model text for a covering letter
http://www.quantify.co.uk/services/index.php#Business%20reply%20servicehttp://www.quantify.co.uk/services/index.php#Business%20reply%20servicehttp://www.quantify.co.uk/services/index.php#Business%20reply%20servicehttp://www.quantify.co.uk/services/index.php#Business%20reply%20service

8/3/2019 YourESS

26/43


Sometimes, clients ask us to provide a further covering letter on our letterhead to reassure

employees that we really exist, and are promising to protect their confidentiality.

Closing dateThe closing date you publish in the letter and elsewhere should be at least a week after the day you

expect employees to receive the pack but not be much more unless shift or rostering arrangements

mean you cant predict when people will see the questionnaire. The aim is to create sufficient sense

of urgency to encourage many people to do it straightaway, so it doesnt get put to one side and

forgotten, but not so great a sense of panic that people regard it as unreasonable, and bin it without

even considering joining in.

Dear Colleague / Employee Name

Employee Satisfaction Survey

I am writing to invite you to participate in a survey. This is a confidential,anonymous way for every member of the Company Name team to have a sayabout how it feels to work here. The results will help us to make sure that anychanges we make will be changes for the better.

Confidential

Please complete the questionnaire and return it in the envelope provided to the

independent company, Quantify, who will handle the analysis for us and protectyour confidentiality. You can complete the survey just by ticking boxes if youchoose, so it need not take more than a few minutes. You dont need to use astamp. Please post your response to arrive by closing date.

No one at Company Name will see your completed questionnaire. Quantify willprovide reports to us, which show the views of different groups of people. Toensure your complete confidentiality, no group with less than [N] members willhave their views reported.

We will let you know what the main results of the survey were, and we will take

action to deal with any issues which emerge.Yours sincerely

Chief Executive

PS Why not do it right now and be sure to have your say?

8/3/2019 YourESS

27/43


We suggest you extend the deadline later, when you issue a reminder, see Progress Reportbelow.

N membersWe will never report to a client the results of any group smaller than three individuals. You may

choose to instruct us to apply a different minimum subset size. The results from a subset smaller

than about ten will have very big sampling error, anyway, so might not be of much value.

Business Reply EnvelopeThe business reply service allows us to distribute envelopes for people to use to post things back to

us. The local delivery office count how many they deliver and bill us for the postage, but only forthe ones which are sent back. We pass this cost on to clients with a modest mark-up for providing

the service. There are two versions of the envelope, one for use in the UK and a different one for use

internationally, so an employee almost anywhere in the world can have a reply paid envelope and

clients pay postage only for the ones that are used.

Home or work address?If you mail to home addresses, this suggests that you expect the questionnaire to be completed in

employees own time. Many people will resent this. Many will also see work matters mailed to their

home as an unwelcome intrusion. Unless you have very good reasons for using home addresses, we

recommend distribution at work, which has the following advantages.

Checklist: Advantages of internal distribution

Need not be a personalised mailing, soless costly

No postage cost Group dynamic helps to get people

filling them out when they see others

doing so

No impact on confidentiality peoplecan take them home if they wish, and if

you have provided a business reply

envelope, they can post their response

in any pillar box. We find response rates tend to be better

There are some possible disadvantages, though.

Checklist: Disadvantages of internal distribution

Employees might be distracted fromessential work

Employees might collaborate, ratherthan giving their own view

Managers (paranoid ones?) sometimesforget to distribute questionnaire

packs, although we can usually spot

this when the responses start coming in

and tip you off so that you can

8/3/2019 YourESS

28/43


encourage them. SeeTabulation bylocation or departmentbelow.

Group dynamic might lead to acollective decision not to participate

Some people dont have a workplaceaddress

A Web Option?You may want some or all informants to complete your survey on the web. Here is how we handle

this.

For a discussion of the advantages and disadvantages of using the web, seeWeb or Paper

above.You can visit a sample web survey which demonstrates the speed of our system, provides

links to selected outputs illustrations and describes the many aspects which can be

configured to suit your preference. Clickhereto visit the survey.

Access controlled, Confidentiality assured

We need to ensure that only authorised informants can participate while still hoping to satisfy them

about their confidentiality. In the QUANTIFY system, each informant gets a unique randomly

generated respondent key to use to get into the survey.

When they are ready to complete the survey, employees visit the site, usually just by clicking on the

link provided in an email invitation. The survey opens with the clients logo on the welcome page,then presents the survey items for them to complete by clicking buttons or typing if text responses

are required. When we are ready, we retrieve the data directly from the server and load it into our

analysis software along with any data we have keyed from paper questionnaires.

The email invitations are generated from data where each employees name and email address

record is allocated one of the unique access keys. When we download the response data, the access

keys do not download so neither we nor our client can relate survey responses to individual

informants.

Offer both routes and secure respondent commitment in advance

There are still only a few organisations in which every employee has access to a connected PC witha browser and feels comfortable using it. So the web may be the best medium for many of the

informants for your survey but it may not be suitable for all of them. Some of the targets for your

survey might not have access to a web enabled PC, or perhaps some people will suspect that their

responses won't be as anonymous on the web as they would be on paper. So why not let people

choose which way they prefer to participate in the survey?
http://embrace.asponlinesoftware.com/rkey.asp?u2nkh4woJP3ddxVv7sRYhttp://embrace.asponlinesoftware.com/rkey.asp?u2nkh4woJP3ddxVv7sRYhttp://embrace.asponlinesoftware.com/rkey.asp?u2nkh4woJP3ddxVv7sRYhttp://embrace.asponlinesoftware.com/rkey.asp?u2nkh4woJP3ddxVv7sRY

8/3/2019 YourESS

29/43


Step 1 Advance notice

Our client provides us with a list of all the participants with name, workplace (internal mailing)

address and email address, if they have one. We send an advance notice to every member of the

target group to say that the survey will be happening. As usual, this message includes explanation of

the reasons for conducting the survey, commitment to take action on the results and assurances of

anonymity and confidentiality. Crucially, though, it also asks each participant to choose whether

they wish to complete the survey on paper, or on the web. They can respond by using a tear-off

strip, or by sending email directly to us.

This process has the usual positive effect on response rate which any advance publicity will achieve,

but this is heightened by the commitment people make by choosing to participate by one route or

another.

We use the participant list the client provided to create a preference register, in which each person is

allocated as a paper or a web participant. People who don't respond to the initial request are

allocated to one route or another using a rule we agree with the client. This could be "Paper unless

they opt for the web", or "Web if we have an email address on file, paper if not" or any other

appropriate rule.

Step 2 Invitations

We generate invitations to participate, as follows:-

Paper Participants: Web Participants:

A personalised covering letter, with the

printed questionnaire and a QUANTIFY

business reply envelope. We can mail these

direct, or deliver them to the client for

distribution.

A personalised email invitation to participate,

including a link to the survey web site, with a unique

respondent key for this participant. We send these

directly to participants at the email addresses provided

by them, or the client employer.

Step 3 Data gathering

Responses return to us by post or accumulate on the web site, and we send the client daily progress

reports showing responses received so far.

Step 4 Reminders

We issue a reminder. For paper participants it may be just a memo asking managers to remind all

their people. For web participants, we send an email reminder to everyone, (we don't know who has

and who hasn't responded) with their unique access link repeated. If they have already completed the

survey but they try again, they just get a message saying "This respondent key has already been

used."

8/3/2019 YourESS

30/43


Step 5 Closure & analysis

When the client agrees with us that the survey should close, we download the response data from the

web site, combine it with the keyed data from the paper participants, and proceed with analysis and

reporting in the usual way.

Gather responses; Capture data; Analyse

resultsSomeone needs to sort out the survey responses from the rest of the mail; open the envelopes,

extract and unfold the questionnaires and arrange them in a neat pile. Someone must configure acomputer to handle the responses and key in all the replies from the paper questionnaires or transfer

the data from a web site. Theyll need to work out a way to group, summarise and average the

responses so you will be able to compare the views of one group of informants with those of another

(and decide which differences aresignificantand which are just the result ofsampling error), design

a way to present the results, analyse and produce the outputs.

Even if you have chosen to do the development, design and distribution of the questionnaire

in house, we would like you to instruct us to handle the responses, data capture, analysis and

reporting. We think this would be a wise decision from your point of view, because we have

procedures and systems in place, and do this all the time. Naturally, we would like to be

more involved, and be able to send you a bigger bill, but we are very happy to provide justthe support you choose to outsource, no more, no less.

Checklist: Reasons to outsource the processing to QUANTIFY Ourreport styleshave evolved through years of client feedback, and are robust and easy to

understand

Our reporting clearly shows you which differences aresignificant, and which are not, so youwont spend time and money on fixing imaginary problems

We can provide normative (benchmarking) comparisons for many survey questions to putyour results in context

Externally prepared reports have more authority than internal data If we do it, you can offer the independent destination for replies and confidentiality promise

which will enhanceresponse rate

We will finish the job faster, (usually within days of the final closing date) so youll getcurrent data, not out of date stuff

We can supply daily progress reports and tabulation by location or department, so you cantarget reminders in the right places
http://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Samplingerrorhttp://www.quantify.co.uk/resources/glossary.php#Samplingerrorhttp://www.quantify.co.uk/resources/glossary.php#Samplingerrorhttp://www.quantify.co.uk/pdf/samples.pdfhttp://www.quantify.co.uk/pdf/samples.pdfhttp://www.quantify.co.uk/pdf/samples.pdfhttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Responseratehttp://www.quantify.co.uk/resources/glossary.php#Responseratehttp://www.quantify.co.uk/resources/glossary.php#Responseratehttp://www.quantify.co.uk/resources/glossary.php#Responseratehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/pdf/samples.pdfhttp://www.quantify.co.uk/resources/glossary.php#Samplingerrorhttp://www.quantify.co.uk/resources/glossary.php#Significance

8/3/2019 YourESS

31/43


Anyone doing this work in house will have to reinvent all our systems and procedures, so youwill probably spend more in-house than we would bill you

We are always pleased to talk over your project and give an estimate for whatever services

you require. There is no charge for an initial discussion.

The Research Kit edition of this book has a 10 page section here on Analysing and

reporting the results including configuring the included analysis tool for your survey;

handling incoming responses; data entry; subset analysis; the relationship between

sample sizes and the accuracy of the result; assessing the significance of apparent

differences of opinion and more.

Progress reportAs the originally published closing date approaches, you should consider extending the closing date

and issuing a reminder. If we are handling the replies, we will be sending daily progress reports by

email, tracking the responses received each day and the cumulative response so far. The progress

report is illustrated in our Report Styles Illustrations document,samples.pdf.

It is usually worth extending the closing date by at least a further week, so people have altogether

had two weeks or more to respond. Reminders, whether personally addressed or not, can announce

the extension of the closing date, and will produce some more responses. There is a trade-off

between waiting for more responses, which will lend more weight to the results when you get them,

and closing the survey, which will allow you to get results while they are still current, not stale

news. You choose.

Tabulation by location or departmentIn a confidential, anonymous survey you obviously cant target reminders on individuals but it can

be helpful to know which areas the responses have already come from. If we are handling the

responses, we can provide, on request, a tabulation which shows you. This means that you at least

know which areas on which you need to concentrate your encouragement to complete the

questionnaires.

Analysing & reporting the results

Quantitative results

Any question which people have responded to by ticking boxes can be summarised using numbers.

This is the sort of question people mean when they refer to quantitative research.

The simplest way to summarise is to count how many people ticked each of the available boxes.

Converting the counts topercentagesbased on all the informants makes it easier to compare one

question with another.
http://www.quantify.co.uk/pdf/samples.pdfhttp://www.quantify.co.uk/pdf/samples.pdfhttp://www.quantify.co.uk/pdf/samples.pdfhttp://www.quantify.co.uk/resources/glossary.php#Percentagehttp://www.quantify.co.uk/resources/glossary.php#Percentagehttp://www.quantify.co.uk/resources/glossary.php#Percentagehttp://www.quantify.co.uk/resources/glossary.php#Percentagehttp://www.quantify.co.uk/pdf/samples.pdf

8/3/2019 YourESS

32/43


If the prompt wasHave you had an appraisal meeting in the last year? and you report that 42% ofinformants ticked Yes, everyone knows what that means. If you compare two different groups, and

you find that for one group the result was 32% and for another it was 50% we know what that

means, too.

What if the prompt wasI found my last appraisal meeting useful, and you asked people to tick one

of the following boxes?

Strongly

disagree Disagree

In

between Agree

Strongly

agree

I found my last appraisal meeting useful

Two different groups might produce results like these:

I found my last appraisal meeting useful Strongly

disagree Disagree

In

between Agree

Strongly

agree

Group 1 18% 18% 12% 20% 32%

Group 2 12% 20% 32% 18% 18%

We can still compare, but it is a bit harder with more numbers to look at, so to make it simpler, we

aggregate the numbers one way or another.

Total percentage favourable

People often report the answers to questions like this by giving the total percentage who ticked the

favourable responses, Agree and Strongly agree in this case. This way, group 1 above scores 52%

and group 2 only 36% and we have no difficulty deciding which is the happier group.

Now look at these two sets of results.

I found my last appraisal meeting useful Strongly

disagree Disagree

In

between Agree

Strongly

agree

Group A 12% 20% 32% 18% 18%

Group B 32% 20% 12% 20% 16%

Using the total favourable ticks approach, they both score 36% but we dont need to spend long

looking at the numbers to see that group B is less happy than group A. This way of reporting results

disregards important part of the data we went to so much trouble to collect. A Strongly agree

response is different from an Agree, so it is barmy to treat them as if they were the same. And the

8/3/2019 YourESS

33/43


Strongly disagree; Disagree and In between responses are all different, so it makes sense torecognise this and treat them differently.

Averages

Here is a way to work out a single number to represent each groups result, taking all the options

into account. Give each reply a value, or score, according to the box the person chose to tick, as

follows.

A vote for

is worth

Strongly

disagree

1

Disagree

2

In

between

3

Agree

4

Strongly

agree

5

When we work out the average score for each of the above groups we get 3.1 for group A and 2.7

for group B, numbers which reflect the difference between the groups.

Percentiles

Most people are happier thinking about scores out of 100, so we would convert these averages to

express them as if the scale had been from 0 to 100 instead of 1 to 5. We can do this with all the

results, whatever scale they use, and this makes all the results comparable. After the conversion, we

report thepercentile results for the two groups like this.

I found my last appraisal meeting usefulGroup A

Group B

Percentile result52.5

42.5

This makes it easy to compare the two groups and conclude that group A are happier than group B.

The Research Kit edition of this book includes detailed instructions here on how to

convert average results to percentiles.

Significance

We might be right to reach this conclusion, but the apparent difference might be just the result ofsampling error. To decide whatsignificancewe can attach to the difference, we need to calculate the

sampling error, which depends on the number of people in each of the groups we are comparing and

the standard deviation of their answers. If we dont do this, we might put a lot of resources into

dealing with an apparent problem which wasnt a problem in reality, just a product of the sampling

process, so-called sampling error.

For people who are interested in this sort of thing but dont know how to do the necessary

calculations, we offer a one day seminar entitledGetting facts out of figures; A beginners

practical guide to descriptive statistics.
http://www.quantify.co.uk/resources/glossary.php#Percentilehttp://www.quantify.co.uk/resources/glossary.php#Percentilehttp://www.quantify.co.uk/resources/glossary.php#Percentilehttp://www.quantify.co.uk/resources/glossary.php#Samplingerrorhttp://www.quantify.co.uk/resources/glossary.php#Samplingerrorhttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Standarddeviationhttp://www.quantify.co.uk/resources/glossary.php#Standarddeviationhttp://www.quantify.co.uk/resources/glossary.php#Standarddeviationhttp://www.quantify.co.uk/services/training-seminars.phphttp://www.quantify.co.uk/services/training-seminars.phphttp://www.quantify.co.uk/services/training-seminars.phphttp://www.quantify.co.uk/services/training-seminars.phphttp://www.quantify.co.uk/services/training-seminars.phphttp://www.quantify.co.uk/services/training-seminars.phphttp://www.quantify.co.uk/resources/glossary.php#Standarddeviationhttp://www.quantify.co.uk/resources/glossary.php#Significancehttp://www.quantify.co.uk/resources/glossary.php#Samplingerrorhttp://www.quantify.co.uk/resources/glossary.php#Percentile

8/3/2019 YourESS

34/43


IndicesBy averaging the responses for several related questions, you can work out a rating for thetopic(or

cluster) they represent. Express it as apercentile, and you can call it a satisfaction index. If you have

several questions about communication, for example, you can average them and express the result as

a percentile to provide a Satisfaction with communication Index. You could do the same with each

topic your questionnaire covers. If you want some questions to have more effect on the final index

than others, the average can be a weighted average.

Importance

If you also included in the questionnaire a block of questions asking how important each topic is to

customers, you can work out a Priority for action index for each topic. Take the satisfaction index

for the topic away from 100 to convert it into a dissatisfaction index. Then multiply by the topicsimportance index and divide by 100. The result is a priority for action index. The following

illustration shows how the thing people are most dissatisfied with isnt always the top priority for

action.

Topic (cluster)

Satisfaction

index

Dissatisfaction

index

Importance

index

Priority for

action index

Communication 60 40 85 34

Physical

environment 52 48 52 25

Recognition and

reward 81 19 78 15

Dont like numbers?

Many people dont enjoy playing with numbers like this. They need help, and may find it in house,

or get it from an ex

youress

Documents