Abstract Number: 020-0186

Creating an Enabling Tool for Facilitating Engagement in Continuous Innovation Programmes

Dr Helen T Wagner*1, Dr Susan C Morton1, and Prof Chris J Backhouse1

1 Manufacturing Organisation Group, Wolfson School of Mechanical & Manufacturing Engineering, Loughborough University, UK

POMS 22nd Annual Conference, Nevada, USAApril 29 to May 2, 2011

Abstract

Although Lean Manufacturing is an established concept in both academia and industry,

consideration of the stages that follow a company-wide Lean initiative has received far less

attention. Pursuing continuous innovation (CI) takes commitment from all involved, and the

gap between knowing about continuous innovation and actually doing it needs to be filled. To

facilitate organizational CI, a need has been identified for a bespoke tool that will enable

managers to understand their people and support problem solving activities, at the

supervisory/team management level in particular. Having identified five main constructs that

contribute to successful engagement of employees in CI and their respective diagnostic tools,

the process of questionnaire development was researched. This offered guidelines for

planning, question wording, ordering and presentation, which were actioned in the

development. The work has resulted in a bespoke tool for facilitating engagement that will

add to the information available to managers and academics alike.

Introduction* Corresponding author:Dr H T Wagner Research Associate: NIMBLE ProjectWolfson School of Mechanical & Manufacturing EngineeringLoughborough University, LoughboroughLeicestershire, UK LE11 3TUEmail: h.t.wagner@lboro.ac.uk

Lean Manufacturing is a well established concept in both academia and industry; however,

what is required when moving on from a lean initiative to attain further benefits is not such

an established field. What lies beyond Lean in relation to performance improvement requires

further investigation; the gap between knowing about continuous innovation (CI) and doing it

also needs to be removed, or reduced at the very least. To facilitate continuous organizational

innovation, a requirement has been identified for a new diagnostic tool for use by managers

to assess all levels of the organization and to assist with problem solving at the level of

supervisory/team management in particular.

Although the information needed could be collected by interviewing workers within an

organization, several advantages to using a standardized questionnaire exist. Studies show

that people are often more honest when completing a self-administered questionnaire [1, 2].

They provide an easy and time effective route to collecting data from many people, provide

anonymity and limit researcher bias, and the structured format ensures each respondent reads

and answers the exact same questions, which makes for robust analysis [3].

Devising a new questionnaire is not an easy task [Error: Reference source not found] and

often researchers underestimate what is required, thinking that because they have knowledge

of a topic they are capable of developing a good questionnaire [4]. In fact, it is a highly

complex and time consuming process [5] that cannot be shortcut, no matter how tempting it

may be [Error: Reference source not found]. The process requires not only thorough

knowledge, but attention to detail [Error: Reference source not found] and a ‘stringent and

scrupulous’ approach to ensure the data collected provides what is required in a usable form

[Error: Reference source not found]. This is essential, as the consequences of the decisions

made during the design phase impact directly on the results obtained [6] and, therefore, the

findings and validity of the study.

It is, therefore, the purpose of this paper to introduce the process undertaken to create a new,

bespoke questionnaire, created to assess the factors affecting employee engagement in

continuous innovation programmes.

Existing Tools that measure the constructs affecting employee engagement in CI

There are five identified constructs that affect the potential for employees to engage with the

CI programme, as outlined by Wagner et al. [7]. In order to incorporate each of the constructs

of creativity, empowerment, leader-member relationship, team role and leadership style, into

the new questionnaire, the existing tools that measure these constructs were assessed.

Job Diagnostic Survey

The Job Characteristics Model (JCM) proposed by Hackman and Oldham [8] is a tool for

analysing the satisfaction and motivating potential offered by a job role. The quantitative

calculation of each component of the model is facilitated by the Job Diagnostic Survey (JDS)

[9], which was developed for use “in research and evaluation activities aimed at assessing

the effects of redesigned jobs on the people who do them” [10].

Questions are set out to identify the core characteristics using two techniques; standard

questions and reverse scored questions. Some researchers have experienced problems with

this approach and have sought to make improvements [11], but when their revised questioning

was tested by others, in a direct comparison, no improvement was found and the authors

recommended continued use of the original questionnaire [12]. The results of the JDS are

entered into the Motivating Potential Score (MPS) equation, also created by Hackman and

Oldham [Error: Reference source not found], where each component in the equation is scored

from 1 to 7, with results ranging from 1 to 343 and scores commonly around 150 [13].

Leader-Member Exchange (LMX)

Leader-Member Exchange (LMX), with its roots in Social Exchange Theory [14], is based on

the two-way, dyadic relationship [15] between a leader and an individual subordinate [16].

Each relationship becomes differentiated [Error: Reference source not found,17] based on

factors affecting the level of interaction, communication, understanding and trust [18] between

the two; constraining or facilitating the development of the relationship [19]. The measure of

the multidimensional relationship [Error: Reference source not found] is based on the

perceptions of both the leader and subordinate, and so in its study, it is vital to view it

objectively from both sides [20].

The LMX-7 questionnaire, put forward by Graen and Uhl-Bien [21], assesses the quality of

the relationship of the supervisor with each individual team-member. The tool is made up of

seven items that exemplify different aspects of the leader-subordinate working relationship

[Error: Reference source not found] and is measured on a five-point scale.

The Belbin team roles model

The Belbin team roles model [22] identifies the nine potential roles each individual could

exhibit when working in a team, outlining the specific behaviours and skills each brings to

the team dynamic.

The Self Perception Inventory (SPI) [23] tool comprises seven questions, which ask the

respondent to distribute ten points between ten different response options. The points can be

allocated where they like, but it is important to use all ten points. Subjects explored include a

person’s contribution to a team and what they feel they lack, their approach to tasks and

problems, and working with others or in a group. Participants completing the questionnaire

indicate their own perceptions of their behaviour in each situation. This can be complemented

by the addition of the Observer Assessment [Error: Reference source not found], where other

team members or the team supervisor provides their perceptions of the participant, to give an

outsider view.

Although some researchers question its validity, many support the tool. They [e.g. Fisher et

al. [Error: Reference source not found] and Partington and Harris [Error: Reference source

not found]] suggest that it has made a significant contribution to understanding [24], also

suggesting its value in use is more important than its psychometric validity [25] and

recognizing that to set aside the work because of doubt would be a great pity [Error:

Reference source not found].

Research Tools – KEYS to Creativity

The KEYS to Creativity instrument (KEYS) was developed by Amabile et al. [26], to meet the

need for research in organizational theory and practice, by using theoretical knowledge from

literature to create a tool that would test real organizational settings. It looks at creativity

within the working environment [27], examining the intrinsic motivation of individuals to be

creative and assessing perceived barriers and enablers to creativity [28]. It is said that “the

value of KEYS lies in its capacity to accurately identify the conditions necessary for

innovation to occur” [29]. Aimed at assessing all levels within an organisation, from the shop

floor team to supervisory and organizational, KEYS concentrates on the effects of

environmental factors on an individual’s perceptions, which influence the creativity of their

work [30].

The KEYS instrument itself is made up of 78 questions assessed using a four point scale

[Error: Reference source not found]; purposely designed to force a response by not offering a

neutral option [Error: Reference source not found]. Of the 78, 66 are related to the work

environment with the remaining 12 assessing performance in terms of creativity and

productivity [Error: Reference source not found]. The work environment factors are split

between management practices that encourage and those that inhibit creativity [Error:

Reference source not found], encompassing: organizational encouragement; supervisory

encouragement; work group supports; sufficient resources; challenging work; freedom;

organizational impediments; and workload pressure.

As a tool it has been extensively empirically tested [Error: Reference source not found] and

has been shown to be both robust and rigorous [31], demonstrating its validity and reliability

[Error: Reference source not found] through research with more than twelve thousand

research subjects [Error: Reference source not found].

Extant Literature on Developing Questionnaires

The Oxford English Dictionary defines a questionnaire as: “A formulated series of questions

by which information is sought from a selected group, usually for statistical analysis; a

document containing these” [32], however, some see it as much more. Labaw [33] sees it as not

only a series of questions or just a series of words [34], but a layered structure, where “it is a

totality, a gestalt that is greater than the sum of its individual questions... with each part vital

to every other part and all parts must be handled simultaneously to create this whole

instrument” [Error: Reference source not found].

Although there may be more to it, the basics of the dictionary definition also hold true; as a

tool for data collection in written format, suitable for large numbers of respondents [Error:

Reference source not found] and a series of attitude or opinion statements and questions

developed to elicit a response, which can then be used to measure the variable being studied

[Error: Reference source not found]. Well designed questionnaires can provide an

understanding of the details of an organization’s manufacturing strategy [Error: Reference

source not found] and can aid in driving-in and measuring the success of organizational

change [Error: Reference source not found]. An ideal questionnaire should be clear,

unambiguous and suitable to collect the data required to test the research question or

hypothesis set [Error: Reference source not found]. In order to meet these requirements it

must be designed with the respondents in mind; this will dictate the type of questions,

wording and concepts that can be explored [Error: Reference source not found].

The Process of Development

Before question writing begins there is much work to be done; this starts with knowing what

is the purpose of the research [Error: Reference source not found] and what you wish to

accomplish [Error: Reference source not found]. Questionnaire construction takes place in

stages, which begin with setting objectives [Error: Reference source not found], clearly

defining what will be studied [Error: Reference source not found] and to what level of detail

and accuracy [Error: Reference source not found]. This should involve reviewing appropriate

literature [Error: Reference source not found]. Once all of this is known, research questions

or hypotheses should be developed [Error: Reference source not found]. It is likely that this

initial planning phase will take up a third to a half of the development time of the

questionnaire [Error: Reference source not found].

Another fundamental part to the early stages of questionnaire development, that must run

parallel to both the planning and question development stages, is that of analysis design

[Error: Reference source not found]. A questionnaire must be designed with analysis as an

integral part; so that it can be assured that the data collected will be suitable for analysis

[Error: Reference source not found]. This statistical analysis will allow researchers to study

data on individual respondents or questions, but will also facilitate the presentation of results

and testing of hypotheses [Error: Reference source not found]. It must, however, be

remembered that data collection is paramount and no amount of statistical manipulation can

make up for poor questionnaire design [Error: Reference source not found]. With this in

mind, the practicalities of questionnaire design must next be examined.

Question Wording

The consideration of question wording is one that receives much attention in extant literature.

While all recommend it be given careful consideration, some think that the specific wording

of questions has a much greater impact than others. Brigham [35] suggests that wording has

considerable effect on results; a belief that is supported by Synodinos [Error: Reference

source not found] who found that even small changes in wording can produce response

effects. However, this belief is not shared by all. Labaw [Error: Reference source not found]

suggested that wording variations have little impact on the stability of results, a position that

was corroborated by the findings of Gendall [Error: Reference source not found] who stated

that it is possible to ask the same question in different ways with no effect on respondents’

understanding.

Further contention exists in the phasing of attitude statements. Murray [Error: Reference

source not found] considers that all such statements should be worded positively; however,

Gendall [Error: Reference source not found] found no evidence that wording positively or

negatively has any influence on response. He did find that the strength of a word had an

impact on response, with words such as ‘forbid’ being less acceptable than ‘not allow’ [Error:

Reference source not found].

It is sometimes taken for granted that the respondent reads and understands the question as

the researcher intends, unfortunately this is not always the case [Error: Reference source not

found]. With this in mind, a table of the suggestions for questionnaire wording has been

compiled, outlining the advice of many sources (Table 1).

In addition to question wording, answer format must be considered. It should provide a clear

structure so that respondents know what is required of them [Error: Reference source not

found]. Closed questions offer a fixed choice of answers that come in several different

formats. Yes/No formats are commonly used, but should be limited to avoid guessing [Error:

Reference source not found]. Checklists where the respondent is asked to tick all that apply

[Error: Reference source not found] can be used when multiple answers may be applicable. In

this instance, the use of an ‘other’ box is also recommended in case a possible option has not

been thought of [Error: Reference source not found], although this may not entirely make up

for omissions [Error: Reference source not found]. Category answers are possible [Error:

Reference source not found] as are quantities or bands of figures [Error: Reference source not

found].

Table 1: Advice for wording questions

Rules for wording of questions

Use closed questions where possible to ensure the context is the same for all [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]

Questions should be simply worded and structured, unambiguous, focussed and short

[Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]

Less than 20 words [Error: Reference source not found]

Less than 12 words [Error: Reference source not found]

Questions should be clear and precise and not woolly, so all understand and interpret as intended

[Error: Reference source not found, Error: Reference source not found,

1 Marshall, G. “The purpose, design and administration of a questionnaire for data collection”, Radiography, Vol.11, pp131-136, 2005

2 Kelly, P. “Questionnaire design, printing, and distribution”, Government Information Quarterly, Vol.17, No.2, pp147–159, 2000

3 Gillham, B. Developing a Questionnaire, Real World Research, Continuum, London, 2000

11 Idaszak, J.R. and Drasgow, F. “A revision of the Job Diagnostic Survey: Elimination of a Measurement Artefact”, Journal of Applied Psychology, Vol.72, pp69-74, 1987

12 Kulik, C.T., Oldham, G.R. and Langner, P.H. “Measurement of Job Characteristics: Comparison of the Original and the Revised Job Diagnostic Survey”, Journal of Applied Psychology, Vol.73, pp462-466, 1988

Error: Reference source not found, Error: Reference source not found, 36]

Use language appropriate to the target population [Error: Reference source not found]

Phrase to the lowest education level of respondents [Error: Reference source not found,Error: Referencesource not found]

Do not patronise or make too elitist [Error: Reference source not found]

Do not use jargon, unusual words, acronyms, and abbreviations [Error: Reference source not found, Error: Reference source not found]

Avoid unfamiliar, difficult words or words that sound similar to others [Error: Reference source not found]

Consider of words have an alternative meaning [Error: Reference source not found]

Avoid double negatives [Error: Reference source not found, Error: Reference

14 Harris, K.J., Wheeler, A.R., and Kacmar, K.M. “Leader-member exchange and empowerment: Direct and interactive effects on job satisfaction, turnover intentions and performance”, The Leadership Quarterly, Vol.20, pp. 371-382, 2009

15 Tierney, P. “Work relations as a precursor to a psychological climate for change”, Journal of Operational Change Management, Vol.12, No.2, pp. 120-133, 1999

16 Kim, B.P. and George, R.T. “The relationship between leader-member exchange (LMX) and psychological empowerment: A quick casual restaurant employee correlation study”, Journal of Hospitality and Tourism Research, Vol.29, No.4, pp. 468-483, 2005

21 Graen, G.B., and Uhl-Bien, M. “The relationship-based approach to leadership: Development of LMX theory of leadership over 25 years: Applying a multi-level, multi-domain perspective”, Leadership Quarterly, Vol.6, No.2, pp. 219-247, 1995

source not found]

Double barrelled questions should be separated into single concept questions

[Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]

Avoid leading or loaded questions [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]

Avoid assuming/presuming questions [Error: Reference source not found]

Imprecise conditions such as frequently, generally, normally should be avoided

[Error: Reference source not found, Error: Reference source not found]

Questions should not challenge the respondents’ knowledge, only asking what they are easily able and to answer

[Error: Reference source not found, Error: Reference source not found, Error:

32 Oxford English Dictionary Online, “definition of questionnaire”, http://www.oed.com accessed 15 Dec. 10

33 Labaw, P. Advanced Questionnaire Design, Abt Books, Cambridge, Massachusetts, 1985

34 Gendall P. “A Framework for Questionnaire Design: Labaw Revisited”, Marketing Bulletin, Vol.9, pp28-39, 1998

Reference source not found, Error: Reference source not found]

Do not ask respondents to think too far back, not more than 6 months [Error: Reference source not found, Error: Reference source not found]

Hypothetical questions are difficult to answer and should be avoided [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]

Questions that ask people to predict the future should be used with caution [Error: Reference source not found]

Although different questions require different styles of response formats, it is the attitude or

opinion statements that seem to be the most contentious. Ranges of mutually exclusive

answers, such as strongly disagree to strongly agree [Error: Reference source not found,

Error: Reference source not found] are commonly used to quantify these questions. However,

Gendall and Hoek [Error: Reference source not found] suggest that agree-disagree questions

are the most likely to be affected by question wording and, therefore, the answer format

should be a forced choice. In further work, Gendall [Error: Reference source not found]

reinforces that there should be no mid-point or neutral alternative offered, in order to measure

intensity of feeling, but states that a no opinion option should always be included.

Question Order

Question order is another issue that must be considered during the design phase of a new tool.

Unlike question wording, most authors are in agreement as to the best way to order questions.

The first and most fundamental point is to establish whether each question is in fact necessary

to complete the study [Error: Reference source not found], as the length of the questionnaire

should be kept to its optimal minimum. Once the questions are deemed to be necessary they

can be ordered based on the generally accepted advice.

The questionnaire should begin with easy, basic questions that are neither sensitive nor

threatening, in order to ease the respondent into the process [Error: Reference source not

foundError: Reference source not found, Error: Reference source not found, 37]. Questions

should then develop logically [Error: Reference source not found, Error: Reference source

not found, Error: Reference source not found], be grouped by theme [Error: Reference source

not found, Error: Reference source not found, Error: Reference source not found] and flow

smoothly from one to the next [Error: Reference source not found]. Questions that are more

sensitive or embarrassing should be left until late on in the order [Error: Reference source not

found, Error: Reference source not found, Error: Reference source not found]. Advice is

divided on where important questions should be placed, with some feeling that these

questions should be first as later responses could impact on these issues, but others suggest

that important questions should be approached slowly [Error: Reference source not found].

Similarly, division is found in the positioning of demographics questions. While Synodinos

[Error: Reference source not found] recommends that some screening questions be placed at

the end of the introduction section, he thinks that demographics questions are likely to be the

most sensitive in the questionnaire so, in line with previous advice, these should be

positioned at the end. Oppenheim [38] concurs on the positioning, although his reasoning

comes more from the desire not to dissipate the initial enthusiasm by diluting it with

questions not related to the main topic of the questionnaire. This is in direct opposition to

Drummond et al. [39], who found that placing demographics questions first actually increased

response rates in a postal survey. The effects of question order on response rates were also

found by Synodinos [Error: Reference source not found] and Dunn et al. [Error: Reference

source not found], and it was thought that this could be influenced further by the gender of

the respondent [Error: Reference source not found].

Questionnaire Presentation Formatting

Whilst much of the advice concerning questionnaire presentation is aimed at self-

administered tools that are completed as part of a postal survey, there are lessons to learn to

improve the presentation of all questionnaires based on these findings. Jepson et al. [40] found

that the overall length of the questionnaire had a direct impact on response rates with a

response rate at 60% for a questionnaire of 849 words, but only 16.7% when the words are

increased to 1800, concluding that there is an acceptable threshold for questionnaire length.

Although response rate is not an issue in organizational studies with full participation, the

findings on questionnaire length may help to ensure that focus can be maintained by those

completing the questionnaire.

Two things are likely to create an immediate impression on respondents, which makes them

vitally important. The first is the introduction, which is needed to build rapport with the

respondent [Error: Reference source not found]; it sets the scene and can build interest in

completing. Second is the graphic design of the tool itself [Error: Reference source not

found], which has the potential to either arouse interest or discourage respondents from

taking the time to complete [Error: Reference source not found].

To maximise the likelihood of completion, practical advice on questionnaire format is offered

by many, and is summarised in Table 2.

Table 2: Advice for presentation of questionnaires

Rules for presentation of questionnaires

Begin by assuring respondents of confidentiality of answers and results [Error:Referencesource notfound]

Use a large, distinct typeface/font, appropriate to the target population [Error:Referencesource notfound, Error:Referencesource notfound]

Use a different, clear typeface/font for instructions [Error:Referencesource notfound]

Instructions should be clear and explicit – illustrated with an example if necessary

[Error:Referencesource notfound]

Questions should be numbered and not split across page breaks [Error:Referencesource notfound]

Pages should be numbered, with PTO at the bottom of each page (where further pages exist)

[Error:Referencesource notfound]

Only place questions on one side of the paper [Error:Referencesource notfound]

End by thanking participants [Error:Referencesource notfound, Error:Referencesource notfound]

The Process of Development – The FACETS Questionnaire

Developing the FACETS questionnaire forms part of a two year study on employee

engagement in continuous improvement, which began with much background research on the

constructs that affect engagement and how these can be synergistically combined to create a

more successful improvement program. At the outset, the objectives and research questions

were set, purpose of the work defined and the final output agreed upon. Literature suggests

that the initial planning phase will likely take up a third to a half of the development time of

the questionnaire [Error: Reference source not found], and this has certainly proved to be true

for this questionnaire, with early phases used not only for literature review, but also initial

testing using the tools developed and recommended by previous authors.

Developing a questionnaire when well established tools exist has proven a challenging

exercise. It is essential to ensure that the new tool measures all of the topics considered

important from the existing questionnaires as effectively as the original without using their

questions. In most cases there were substantial parts of the existing questionnaires that were

considered unnecessary for this study, as they went into aspects not thought to link directly to

engagement in the CI process. In order to ensure effective assessment, the existing questions

were reviewed, but this was supplemented by the background research on the constructs and

factors that affect and make them up. Aligning the two, allowed new questions to be written

that have the potential to still measure the construct without plagiarising the work of extant

tool developers. Therefore, creating a new and, more concise tool that measures only the

aspects considered to directly influence behaviour.

Throughout the early development stages, designing the data analysis to be undertaken

[Error: Reference source not found] has always been under consideration. This will ensure

that the final questionnaire not only meets its purpose in providing knowledge on the

individual and their role in the CI program, but also that it facilitates the proof required to

validate the model and theory developed on the interconnectivity of the constructs considered

to effect CI [Error: Reference source not found].

Question Wording

Wording of questions has been carefully considered to meet as many of the suggested

guidelines as possible. At this early stage all statements have been worded positively, as per

the advice of Murray [Error: Reference source not found], but this will be carefully analysed

to ascertain if it has had any effect on results.

A decision was taken to align all sections of questions to a standard answer format. Rather

than questions, as such, attitude/factual statements were developed to measure strength of

feeling or level of agreement; all in a closed format [Error: Reference source not found]. A

five point scale was chosen, to allow for a middle position, although this is labelled as

moderate as opposed to neutral, in order that the participant not see it as a opt out. Congruent

to this, no ‘don’t know’ option has been allowed, with the assumption that if it is possible to

answer, participants will choose and if it is not they will simply leave the question

unanswered; as outlined in the instructions.

Referring back to Table 1, the steps undertaken were reviewed in-line with advice in extant

literature, to produce Table 3.

Table 3: Actions on advice for wording questions

Actions taken on wording of questions

Only used closed questions [Error: Reference source not found,

Error: Reference source not found, Error: Reference source not found]

Questions kept concise and simple [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]

100% of questions below 20 words [Error: Reference source not found]

92.5% of questions 12 words or below [Error: Reference source not found]

Questions kept clear and not confusing [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, 41]

Language kept simple but not patronising, accessible to all [Error: Reference source not found, Error: Reference source not found,

Error: Reference source not found]

No jargon, acronyms or abbreviations used [Error: Reference source not found, Error: Reference source not found]

Words carefully selected for singular meaning and common usage [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]

No questions contain double negatives [Error: Reference source not found, Error: Reference source not found]

Double barrelled questions only used when combination effects sought [Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]

Questions are neither loaded or assuming/presuming [Error: Reference source not found, Error:

Reference source not found, Error: Reference source not found, Error: Reference source not found]

Imprecise conditions not used, conditions such as regularly considered acceptable

[Error: Reference source not found, Error: Reference source not found]

All questions based on current knowledge and strength of feeling, easy to answer

[Error: Reference source not found, Error: Reference source not found, Error: Reference source not found, Error: Reference source not found]

Limited hypothetical/future questions used to illustrate some vital aspects of [Error:

6 Sinclair, M.A. “Questionnaire design”, Applied Ergonomics, Vol.6, No.2, pp73-80, 1975

7 Wagner, H., Morton, S., Backhouse, C., Burns, N., and Dani, S. “Building a model of the Synergistic Effects of Constructs Affecting Engagement in Continuous Improvement Programs”, (in review - Human Resource Management Review, 2010)

8 Hackman, R., and Oldham, G.R., Work Redesign, Addison-Wesley Publishing Company, Reading, Massachusetts, 1980

9 Munz, D.C., Huelsman, T.J., Konolold, T.R., and McKinney, J.J. “Are There Methodological and Substantive Roles for Affectivity in Job Diagnostic Survey Relationships?”, Journal of Applied Psychology, Vol.81, No.6, pp795-805, 1996

10 Hackman, R., and Oldham, G.R. “Development of the Job Diagnostic Survey”, Journal of Applied Psychology, Vol.60, pp159-170, 1975

20 Cogliser, C.C., Schriesheim, C.A., Scandura, T.A., and Gardner, W.L. “Balance in leader and follower perceptions of leader-member exchange: Relationships with performance and work attitudes”, The Leadership Quarterly, Vol.20, pp. 452-465, 2009

relationship Reference source not found, Error: Reference source not found, Error: Reference source not found]

Wording effects will only really be measurable once the questionnaire has been pilot tested

on a representative sample of the population, at which stage any felt to be affecting results

will be amended.

Question Order

22 Belbin, R.M. Management Teams: Why They Succeed or Fail. Heinemann, London, 1981

23 Belbin Associates, “What are Belbin Team Role profiles?” available online (accessed 08 Jan. 10) http://www.belbin.com/rte.asp?id=10

26 Amabile, T.M., Conti, R., Coon, H., Lazenby, J., and Herron, M. “Assessing the work environment for creativity”, Academy of Management Journal, Vol.39, No.5, pp. 1154-1184, 1996

27 Mathisen, G.E., and Einarsen, S. “A Review of Instruments Assessing Creative and Innovative Environments Within Organisations”, Creativity Research Journal, Vol.16, No.1, pp. 119-140, 2004

28 Morton, S.C., Burns, N.D., and Michaelides, R. “OPEN innovation in operation: Organisational Performance and Engineering Networks”, Proceedings of the Third World Conference on POM, Manufacturing Fundamentals: Necessity & Efficiency, Tokyo, Japan, August 5-8, 2008

29 CCL, KEYS® to Creativity, Centre for Creative Leadership, KEYS publicity brochure, available at http://www.ccl.org/leadership/ assessments/KEYSOverview.aspx accessed 20 May, 2009. 2007

30 Morton, S.C., and Burns, N.D. Understanding and Overcoming Resistance to Innovation, in Creating Wealth from Knowledge: Meeting the innovation challenge, Bessant J. and Venables, T. Editors. Edward Elgar, Cheltenham. 2008

35 Brigham, F.R. “Some quantitative considerations in questionnaire design and analysis”, Applied Ergonomics, Vol.6, No.2, pp90-96, 1975

36 Gendall P. and Hoek J. “A Question of Wording”, Marketing Bulletin, Vol.1, pp25-36, 1990

The combination of previous questionnaires utilized in the early phase of research, resulted in

participants being asked to complete an arduous 237 questions. In light of this, a target was

set of 100 questions for the final tool. At this early stage more questions (120) have been

written, to be refined after initial analysis takes place [Error: Reference source not found].

40 Jepson, C., Asch, D.A., Hershey, J.C., and Ubel, P.A. “In a mailed physician survey, questionnaire length had a threshold effect on response rate”, Journal of Clinical Epidemiology, Vol.58, pp103–105, 2005

41 Gendall P. and Hoek J. “A Question of Wording”, Marketing Bulletin, Vol.1, pp25-36, 1990

4 Synodinos N.E. “The ‘art’ of questionnaire construction: some important considerations for manufacturing studies”, Integrated Manufacturing Systems, Vol.14, No.3, pp221-237, 2003

5 Murray, P. “Fundamental issues in questionnaire design”, Accident & Emergency Nursing, Vol.7, pp148-153, 1999

13 Arnold, J., Robertson, I. and Cooper, C. Work Psychology. Pitman Publishing, London, 1991

17 Tse, H.H.M., Dasborough, M.T., and Ashkanasy, N.M. “A multi-level analysis of team climate and interpersonal exchange relationships at work”, The Leadership Quarterly, Vol.19, pp. 195-211, 2008

18 Scandura, T.A. and Pellegrini, E.K. “Trust and leader-member exchange: A closer look at relational vulnerability”, The Journal of Leadership and Organizational Studies, Vol.15, No.2, pp. 101-110, 2008

19 Aryee, S. and Chen, Z.X. “Leader-member exchange in a Chinese context: Antecedents, the mediating role of psychological empowerment and outcomes”, Journal of business research, Vol.59, pp. 793-801, 2006

24 Fisher, S.G., Macrosson, W.D.K., and Sharp, G. “Further evidence concerning Belbin team role Self-Perception Inventory”, Personnel Review, Vol.25, No.2, pp.61-67, 1996

25 Partington, D., and Harris, H. “Team role balance and team performance: An empirical study”, Journal of Management Development, Vol.14, No.6, pp.35-41, 1995

31 Adams, R., Bessant, J., and Phelps, R. “Innovation Management Measurement: A review”, International Journal of Management Reviews, Vol.8, No.1, pp. 21-47, 2006

37 Dunn, K.M., Jordan, K., and Croft, P.R. “Does questionnaire structure influence response in postal surveys?”, Journal of Clinical Epidemiology, Vol.56, pp10–16, 2003

38 Oppenheim, A. N. Questionnaire Design, Interviewing and Attitude Measurement, Continuum, London, 2000

In a questionnaire that is primarily based on attitudes, it is difficult to ascertain which

questions are likely to be most sensitive for participants, with the potential for questions to

affect some more than others. In light of this, the questionnaire begins with a general section

on continuous improvement, to both ease the participant into completion [Error: Reference

source not foundError: Reference source not found, Error: Reference source not found, 42]

and provide a clear, contextual start on which to base future answers. Further sections are

then used to move through the constructs being measured in a logical [Error: Reference

source not found, Error: Reference source not found, Error: Reference source not found]

themed way [Error: Reference source not found, Error: Reference source not found, Error:

Reference source not found]; progressing from topics considered less to more challenging

[Error: Reference source not found, Error: Reference source not found, Error: Reference

source not found]. However, within each section the questions are purposely randomized so

that questions on each sub topic are split up, to encourage them to be read carefully each time

rather than creating a lead from one to the next.

The division shown on the placement of demographics questions [Error: Reference source not

found, 43, 44] has lead to an approach that allows participants in the pilot phase to choose

whether they complete these questions before or after the main body. Results of this will

dictate the final positioning of the demographics section.

Questionnaire Presentation Formatting

During the development phases, practical information offered in extant literature, as outlined

in Table 2, was kept in mind and the measures taken to meet these recommendations are

summarised in Table 4.

39 Drummond, F.J., Sharp, L., Carsin, A-E., Kelleher, T., and Comber, H. “Questionnaire order significantly increased response to a postal survey sent to primary care physicians”, Journal of Clinical Epidemiology, Vol.61, pp177-185, 2008

Table 4: Actions on advice for presentation of questionnaires

Action taken on presentation of questionnaires

Introductory section assuring confidentiality and anonymity [Error:Referencesource notfound]

Questions in Arial font [Error:Referencesource notfound, Error:Referencesource notfound]

Instructions given in Calibri font, headings Arial Bold [Error:Referencesource notfound]

Clear instructions given at the start of the questionnaire, covering all sections which are completed in the same way

[Error:Referencesource notfound]

All questions are numbered and not split across page breaks, column headings for answer options on top of each page

[Error:Referencesource notfound]

Pages numbered, PTO considered unnecessary [Error:Referencesource notfound]

Printing only on one side of the paper [Error:Referencesource notfound]

Participants thanked at end of introduction and after questions [Error:Referencesource notfound, Error:Referencesource notfound]

Although much can be learned from the literature on postal survey response rates,

information on how number of words affects response rate was considered non-influential in

this case, as the questionnaire will always be administered as part of a company based study

with participants completing on company premises, in company time. However, as with

number of questions, overall length of the questionnaire was generally considered throughout

the development stages.

The introduction to the questionnaire was given much attention, due to its potential influence

on participants [Error: Reference source not found]. Formatted on a single sheet, it outlines

the purpose of the study, what it will measure, and how the information will be presented to

management. It assures confidentiality and anonymity of data and results. The different types

of questions (demographic and research) are explained, along with instructions on how to

complete both sections, giving the choice of completion order. The introduction ends by

thanking participants.

Graphic design was also considered important [Error: Reference source not found, Error:

Reference source not found]. General appearance was kept simple, with questions

formatted in a tabular structure, using a highlight for every other question, in order to both

distinguish between questions and to draw the eye across to the correct answer line for a

question. Color was also deliberated. It was decided that the paper should be white, partly as

this was shown to make little difference in previous studies [45] and partly as it creates a clear,

simple background [46, 47]. Highlight colors were researched for their potential meaning and

their practical application in printed form.

Shortlisted colors were pale blue, pale green, yellow, orange and grey, which was included

for simplicity of questionnaire reproduction. Each has a defined meaning and possible effect

on participants and was chosen for these advantages. Blue is “often associated with depth

and stability. It symbolizes trust, loyalty, wisdom, confidence, intelligence” [Error: Reference

source not found], “Green calms and soothes the mind, stimulates creativity, is easy on the

eye” [Error: Reference source not found], “Yellow stimulates mental activity, and attracts

attention” [Error: Reference source not found], whereas “Orange increases oxygen supply to

the brain, produces an invigorating effect, and stimulates mental activity” [Error: Reference

source not found].

Piloting the Questionnaire

As literature suggests [Error: Reference source not found], an initial pre-pilot was undertaken

using staff and PhD students within the university. This allowed for early detection of basic

errors in typing and grammar. It also highlighted a small number of issues in regard to the

42 Dunn, K.M., Jordan, K., and Croft, P.R. “Does questionnaire structure influence response in postal surveys?”, Journal of Clinical Epidemiology, Vol.56, pp10–16, 2003

43 Oppenheim, A. N. Questionnaire Design, Interviewing and Attitude Measurement, Continuum, London, 2000

44 Drummond, F.J., Sharp, L., Carsin, A-E., Kelleher, T., and Comber, H. “Questionnaire order significantly increased response to a postal survey sent to primary care physicians”, Journal of Clinical Epidemiology, Vol.61, pp177-185, 2008

45 Beebe, T.J., Stoner, S.M., Anderson, K.J., and Williams, A.R. “Selected questionnaire size and color combinations were significantly related to mailed survey response rates”, Journal of Clinical Epidemiology, Vol.60, pp1184-1189, 2007

46 Sibagraphics, “The Meaning of Colour in Web Design”, Available online at http://www.sibagraphics.com/colour.php accessed 24 January 2011

47 Color Wheel Pro. “See Color Theory in Action”, http://www.color-wheel-pro.com/color-meaning.html accessed 24 January 2011

instructions given and wording in a couple of questions. These issues were rectified and the

questionnaire was then considered ready to be used. Information from pre-pilot participants

led to the decision to use a questionnaire with blue highlighting during the pilot phase.

The next stage in the development will be a pilot at the organisation being studied, with a

small sample, representative of the general population to be studied. At this stage the

wording, question order and presentation will be assessed for their suitability in meeting the

requirements for a fully commercial tool.

From there, the questionnaire will be completed by large sections of the shop-floor

populations of two of the organization’s larger plants, in order to provide a wide testing

ground in different cultures to give the data needed to assess reliability and robustness of the

tool.

Acknowledgment

Research was supported by the EPSRC Loughborough Innovative Manufacturing and

Construction Research Centre http://www.lboro.ac.uk/imcrc

References