dissertation studies

SouthamptonEducation SchoolSouthamptonEducation School

Dissertation Studies

Data Collection Methods


Outline• Choosing your participants• Qualitative collection methods• Interviews• Focus groups

• Quantitative collection methods• questionnaires


Choosing your Participants• relevance or convenience methods• usually associated with qualitative methods but

can be applied to surveys or evaluations of programmes

• probability methods• usually associated with experimental and survey

methods


Choosing your Participants-relevance methods• It is the relevance to the research topic

rather than their representativeness which determines the way in which the people/subjects are chosen

• convenience (haphazard)• get any participants in any manner that is

convenient• person on the street interviews of media


Choosing your Participants-relevance methods• quota• get a pre-set number of participants in each of

several pre-determined categories that will reflect the diversity of the population.

• snowball• get cases using referrals form one or a few

participants, and then referrals from those people…and so on

• extreme/deviant case• get participants that substantially differ from the

dominant pattern


Choosing your Participants-relevance methods• theoretical• (sometimes called purposive sampling)• Get participants that will help reveal features

that are theoretically important about a particular setting / topic

• establish criteria for selection of participants• certain knowledge• certain experiences


Choosing your Participants-relevance methods• Number of participants?• depends on:• research purpose• type of data analysis• on population characteristics• number is far less important than sampling

method• saturation sampling• keep selecting / interviewing data/ participants

until no new information occurs


Choosing your Participants-probability sampling• Everyone in your population has an equal

chance of being selected• simple random• create a sample frame for all cases, then select

cases using a purely random process• pull names out of a hat• allocate random numbers to cases

• sample frame is a list of all possible participants


Choosing your Participants-probability sampling• stratified

• you may wish to ensure a balance of particular traits (gender or age etc.)

• divide (stratify) your sampling frame into groups according to your traits

• draw a random sample from each of the subgroups then combine the samples


• John• Peter• Toni• Rebecca• Amanda• Vicky• Andrew• Robert• Bart


John MalePeter PeterToni AndrewRebecca RobertAmanda BartVicky FemalesAndrew ToniRobert RebeccaBart Amanda Vicky


John MalePeter PeterToni AndrewRebecca RobertAmanda BartVicky FemalesAndrew ToniRobert RebeccaBart Amanda Vicky4 Random

2 Males randomly2 Females


Choosing your Participants-probability sampling• systematic• create a sampling frame• specify the total number of cases required. • divide the sampling frame by the no. of cases• choose a random starting place, select every Xth

case

Jane Anderson Cliodhna Devitt Calum JefferiesJohn Anderson Graeme Devlin Paul JensenStuart Anderson Anna Downes Linda JohnsonCaine Ansell Stephen Drew Louise JohnsonMark Antoni Judy Drew Jennifer JohnstoneLachlan Archibald Anna Duff Daniel JonesSandy Argo Carol Duffin Lara JordanDavid Argo Andrew Edwards James KellyTracey Argo Ruth Edwards Lisa KempChristine Argo David Farmer Joshua KernanJohn Armour Luke Fielden Sheryl-Lee KerrScott Armstrong David Fittell Alan KiddPeter Arnett Helane Fittell Julieanne KiddAmanda-Rose Arnett Frederick Fogarty James KingChris Bain Kevin Follon Anna KitchingRachel Baker Jane Forrest Wendy LavenderHayden Ballard John Forsyth Margaret LawrieAlison Bell Ben Fox Jennifer LeemingDavid Bell Scott Fraser Adam LeslieNicola Bennett Alexandra Frischmann Catherine LeslieRoss Bennion Neil Fullerton Jenna LetleyBeth Bennion Mark Gaffney Stuart LittleSean Black Paul Gardiner Glenn LoutitJohn Bodley Catherine Garner Colette LoutitPeter Bodley Emily Garrioch Karyn LumsdenDebbie Bodley Olivia Gavan Alan MacDonaldLynette Bodley Sue Gillespie Ross MacDougallAlan Bolt Michael Goodwin Gemma MacNeillKym Bolt Christopher Grant Ali MarshallEwan Bouch Daniel Gray Cecily Martina

1. a programme has 100 students2. you wish to select 10% e.g. 103. select every 10th student


Choosing your Participants-probability sampling• Cluster• identify sub groups (clusters) of your population

(usually geographical)• create a list (a sampling frame) of the clusters• randomly select an agreed number of clusters

• create a sampling frame for each selected cluster

• select a random sample from within each cluster

1

2

3

4


• suppose you were undertaking a satisfaction survey for a university library

• you need to cluster your sampling• time of day• day of week• quota for each sample period

Mon Tues Wed Thurs Fri Sat Sun

7-99-1212-22-55-77-10


Sample Size• depends on:• research purpose• type of data analysis• on population characteristics

• size is far less important than sampling method.• the larger the population the smaller sampling

ratio is required (95% confidence)


SamplingRecommended Sample Sizes for two different confidence levelsPopulation Size Sample size for

95% confidenceSample size for 90% confidence

Population Size Sample size for 95% confidence

Sample size for 90% confidence

10 10 275 163 7415 14 300 172 7620 19 325 180 7725 24 350 187 7830 28 375 194 8035 32 400 201 8140 36 425 207 8245 40 450 213 8250 44 475 218 8355 48 500 222 8360 52 1000 286 9165 56 2000 333 9570 59 3000 353 9775 63 4000 364 9880 66 5000 370 9885 70 6000 375 9890 73 7000 378 9995 76 8000 381 99

100 81 51 9000 383 99125 96 56 10000 385 99150 110 61 15000 390 99175 122 64 20000 392 100200 134 67 25000 394 100225 144 70 50000 397 100250 154 72 100000 398 100


response rate• 0-20% Your project has not succeeded - unless

representativeness doesn’t matter or somehow it is so homogeneous that a tiny sample is still representative

• 20-40% Too low - unless there are reasons. You must account for the low rate - it may represent too strong a bias

• 40-60% Bearable - but again you must account• 60-80% You can relax. As a formality: account for the

non response• 80-100% Good …but still account for non responses


Questionnaires• A questionnaire is a set of standardised

questions for gathering the same information from a group of individuals• Aim

• To make comments about the group• To generalise from the group to a wider population

• You can administer questionnaires by:• Mail• Telephone• Face-to-face interviews• Hand-outs• Electronically (e-mail or through Web-based

questionnaires).


Purpose & types• Advantages

• researcher doesn’t need to be present (low cost)• structured, often numerical data• relatively straightforward to analyse

• Disadvantages

• time consuming to develop, pilot and refine (high cost)

• limited scope of data (no visual cues)• elicits inflexible responses

(Wilson and McLean, 1994 in Cohen, Manion and Morrison, 2000)


Purpose & types• Descriptive

• The descriptive survey is designed to find out the extent of a particular phenomena within a population

• Typically descriptive surveys seek to determine 'how many' participate in a certain behaviour or hold a particular opinion

• Need to ensure the sample and sample size are representative

• Analytical• The analytic relational survey is set up specifically to

explore associations between particular concepts• It is less orientated towards representativeness and more

towards finding associations and explanations• It is more than often used in theory-building/testing

research therefore statistically staple sample sizes are more important

(Oppenheim, 2000)


Levels of measurement• we classify information from questions using

numbers and we can do this with varying amounts of precision. This process is called level of measurement and partly determines what statistical processes can be carried out

• nominal type questions (lowest)• ordinal type questions• interval type questions• ratio type questions (highest)


Basic statistical approaches• Category / Discrete• nominal

• Ordered• ordinal

• Continuous• interval• ratio differences between certain sub-

groups or conditionsrelationships between variablesprediction of variablesmodelling of relationships

description of a particular phenomenaassociations of particular behaviours and conditions


Levels of measurement• in nominal measurement the numerical

values just "name" the attribute uniquely. No ordering of the cases is implied.

• Jersey numbers in basketball are measures at the nominal level. A player with number 30 is not more of anything than a player with number 15, and is certainly not twice whatever number 15 is.

• Subject specialism of teachers in a secondary school, no implication of value of one subject over another is made.


Levels of measurement• in ordinal measurement the attributes can be

rank-ordered. Here, distances between attributes do not have any meaning.• for example, on a survey you might code highest

level of educational attainment using the national qualifications framework:• level 1 (5+ GCSE passes), 2 (5+GCSE A*-C), 3

(A-level/IB), 4 (HND, foundation degree), 5 (degree)

• In this measure, higher numbers mean more education. But is distance from 1 to 2 same as 4 to 5? The interval between values is not interpretable in an ordinal measure.


Levels of measurement• in interval measurement the difference

between the attributes have meaning on a numerical scale and are equal, but zero is not an absence of that attribute.

• for example in the Celsius (centigrade) temperature scale we know that the change from 5 to 10 C is the same as a change from 15 to 20 C. But 20 C is not twice as hot as 10 C.


Levels of measurement• in ratio measurement there is always an

absolute zero that is meaningful. This means that you can construct a meaningful fraction (or ratio) with a ratio variable. Weight is a ratio variable.

• In applied social research most "count" variables are ratio, for example, the number of clients in past six months. It is possible to have zero clients and it is meaningful to say that "...we had twice as many clients in the past six months as we did in the previous six months."


Basic statistical approaches• Category / Discrete• nominal

• Ordered• ordinal

• Continuous• interval• ratio differences between certain sub-

groups or conditionsrelationships between variablesprediction of variablesmodelling of relationships

description of a particular phenomenaassociations of particular behaviours and conditions


Question formats• Open question [write whatever you want]

Please use the space belowto tell us about your peanut butterexperiences.


Question formats• Open question [write what one wants]

What brand of peanut butterdo you use?_________________


Question formats• Nominal-type Questions• forced choice

Do you like peanut butter?o Yes o No


Question formats• Nominal-type Questions• inventory

Which brands of peanut butter do you like?Kraft oSunbeam oDaffodil oNature’s Own oTesco’s finest o


Question formats• Ordinal-type Questions• ranking

Please rank the following brands of peanut butterKraft oSunbeam oDaffodil oNature’s Own oTesco’s finest o


Question formats• Interval-type Questions• (scales)

To me peanut butter is the best food in the worldstrongly agree agree disagree strongly disagree

I think I would die if I didn’t have peanut butterAgree 1 2 3 4 5 disagree


Question formats• Ratio-type Questions• (behaviour/frequency)

• Please indict how times a week you would have a peanut butter sandwich _______________

• How many times a day do you dream of peanut butter ____________________

•


Response formats- Likert Scale

To me peanut butter is the best food in the worldstrongly-agree agree disagree strongly-disagree

I think I would die if I didn’t have peanut butterdisagree 1 2 3 4 5 agree


Response format- Likert Scale

• respondents indicate the extent of their agreement or disagreement with a statement

• usually 5 / 7 / 9 options provided

Problems• too many options confuse the respondent• what does the middle position represent• response set• Which direction? (increasing/decreasing?)• General rule of thumb… 5 or 7 options


Single or multiple QuestionsTo what extent are you satisfied with peanut butter

Very Unsatisfied Very Satisfied1 2 3 4 5

To what extent are you satisfied with the following aspects of peanut butter

Satisfaction with the colour 1 2 3 4 5Satisfaction with taste 1 2 3 4 5Satisfaction with the texture 1 2 3 4 5Satisfaction with the aroma 1 2 3 4 5Satisfaction with packaging 1 2 3 4 5

Pre-published questionnaires?• Advantages• Less time• Proven set of items• Reliability / Validity

tested• Clarity of directions• Potential for

benchmark comparisons

• Prior data analysis framework

• Disadvantages• Cost• Items may not fit

your research question (s)

• Inappropriate norms for your group (e.g. US vs. UK)

Develop my own questionnaire?• Advantages• Relevant set of

items• Involvement of

stakeholders• Low(er) Cost• Data analysis

control• Control over

report format

• Disadvantages• Time consuming-

development of items, formatting, printing, check reliability & validity, etc.)

• Unknown reliability/validity

• Have to consider length of questionnaire carefully

• Greater risk of poorly worded items


Tips• Length• Keep questions short < 25 words

• Phone questionnaires should be no longer than 10 minutes

• Face to face questionnaires < 30 minutes

• Self-administered questionnaires < 4 - A4

• Web-based questionnaires shouldn't require participants to have to take multiple steps to answer each question.


Tips• Language

• Use simple everyday language ( accessible, readability, validation process)• What particular aspects of the current

positivistic debate would you like to see reflected in the developmental psychology course

• Avoid irritating questions• Have you ever attended an in-service course of

any kind during your entire teaching career?


Tips• Language• Avoid loaded questions that suggests to the

respondent there is only one acceptable answer• Do you prefer abstract, academic courses or

down-to earth practical courses useful to your day-to-day life?

• Avoid double-barrelled questions• Vocational education is only available to the

lower ability students but it should be open to every student


Tips• Order

• No hard and fast rules• Be logical in the sequence• Avoid developing a ‘response-set’• Start with easy, move to harder questions• Early questions may inform later questions• Put least important questions last• Be sensitive to fatigue


What is an Interview?• Interviewing is a meeting between two

persons to exchange information and knowledge through questions and responses, resulting in a communication and joint construction of meaning about a particular topic

• (Janesick, 1998)


Assumptions• assumes research is subjective• (quantitative research assumes objectivity)

• the researcher brings his or her own values, history, and experiences into any situation

• assumes the participants are the experts on your topic that is why you are interviewing them

• consequently• focus on who ‘owns’ the research• focus on understanding the researcher


When do we use interviewing?• When the aim is to explore ‘in-depth’ a

particular topic or situation• more detailed knowledge• straightforward ideas are more complicated than

thought• surface appearance is misleading

• When there is limited ‘theory’ available about a topic

• When the population of participants is small• When the response mode of other methods is

inappropriate for the participants


Advantages• relatively unobtrusive• can be undertaken quickly• flexible can redirect• probing• can seek further explanation• can follow-up leads

• provide ‘rich’ information• empowerment of respondents• holistic• not just cognitive or behavioural or affective

Structured Semi Structured Unstructured

Standardised InterviewsSurvey interviewsClinical history taking

In-depth InterviewsSurvey InterviewsGroup Interviews

In-depth InterviewsClinical interviewsOral life history Interviews

Deductive

hypothesis

theory

test hypothesis

accept/reject theory

Inductive

identify patterns

observations

generalisations

theory

Structure

Semi-Structured UnstructuredStructured

Interviewer Q

uestioning Skills

and Technique

Question Content

Impo

rtanc

e


Structured (standardised)• an exact set of questions are specified• the order of questions is specified• responses usually short (or even multiple response)• the interview is very focussed• outcomes are somewhat directed• variation from structure is discouraged• Aim: to compare responses of various groups• Potential advantages

• control, reliability and speed (including analysis)• Potential disadvantages

• constrained, superficial and stilted

Structure


Interviewer Q

uestioning Skills

and Technique

Question Content

Impo

rtanc

e


Semi-structured(interview schedule)• topics and issues specified in advance• interviewer decides the order and wording of

questions during the course of the interview• the interview schedule helps to makes data

collection systematic for each respondent• interesting information can be followed up• Aim: to develop theory about your topic• Potential advantages

• rapport, flexibility and depth/richness• Potential disadvantages

• lack of control, danger of leading, analysis


Planning your interview• What is your Research Purpose

• From your research purpose state a Central Research Question

• From your central research question identify 5-10 Topic Questions

• For each topic question write 4 or 5 Individual Questions


Can you describe what your job involves?do they ever conflict with each other?when do they conflict?in what ways?

Who are the people you mix with in your job?what parts of your job are these people are related to?can you describe the way others expect you to

behave?how do you respond?

What is the difference between the way they see your role and the way you see your role?

how do you respond to the difference?how do they respond to the differences?do people outside the organisation have a different

view of your role (professional/peers)?Who are the people you report to?

formally? /informally?how much interest do these people take in the

organisation?in your opinion is this too much? too little?how does this affect you?

Structure


Interviewer Q

uestioning Skills

and Technique

Question Content

Impo

rtanc

e


Unstructured (interview guide)• no predetermination of questions or issues• questions emerge from immediate context, asked as

they arise• increases salience and relevance of questions• questions relate to circumstances of individuals• Aim: to let participants dictate the research problem

and develop theory• Potential advantages

• detail, faithful, participant engagement• Potential disadvantages

• narrow, high level of skill (questioning & analysis)


Unstructured - a Model(Wengraf, 2001)

Usually three subsections• Section 1

the interviewer asks a single initial question designed to elicit the full narrative

I would like you to tell me about your use of technology in your teaching, all the events and experiences that are important for youStart wherever you like. Please take the time you need. I will listen, I won’t interrupt, and I will just take some notes for afterwards.



• Section 2

During Section 1 the researcher takes notes about the main topics.

In Section 2 the researcher repeats those topics in the exact order and words and asks them to tell you more.

No new points are to be raised by the researcher only elaboration of the points raised.



• Section 3Usually follows about a week or more after

sections 1/2.

After transcribing and analysing your interview you will form questions about their ‘story’ that you wish to elaborate even further.

This section will be relatively structured but should leave room for new ideas to emerge and may be followed up.


Group Interviews(Focus Groups)• usually 2-10 people• usually semi-structured• tend to move from experiences/behaviour to

values/opinions • interviewer acts as a moderator/guide• Aim: draw out collective view of the group• Possible advantages:

• Insight into the collective nature of an issue• Possible disadvantages:

• moderator skills required to prevent ‘hijacking’ or turning into multiple individual interviews


Interview process• Before the interview• Choosing your participants• Location & Attire• Opening the interview• During the interview• Closing the interview


Question Types• Open questions• Closed questions• Paraphrasing• Probing• Using examples• Structural• Contrast/Comparison


Open vs. Closed• Open• Questions that cannot be answered with a yes /

no or a single word

• Q. How was your weekend? • A. Good (closed)

• vs.

• Q. Tell me about what you did on the week-end?

• A. …...


Probing• Probes are used to deepen the response to a

question

• a follow-up question• When did that happen?• Who else was involved?• Where were you during that time?• What was your involvement?• How did that come about?• Where did that happen?


Probing• or elaboration questions

• Can you tell me more about that?

• Would you elaborate on that?

• That is useful…can you tell me more about that?

• I am not sure what you mean by that … can you elaborate further?


Paraphrasing• repeating back to the participant your

interpretation of that they said

• This is used to see if you are understanding them correctly

• Or to encourage deeper conversation

• So what you are saying is…...• Let me get this right, you think …


Using Illustrative Examples• Draw on ideas from previous interviews to

put the participant at rest• other people have told us that they enjoy …

how do you feel about …?

• However avoiding leading the participant• we’ve been hearing a lot of bad comments

about … tell me how bad it really is?


Contrast / Comparison• Drawing comparisons between various points

the interviewee may have said

• You said that the teachers at the college treated you differently from your teachers at school. Can you elaborate on that a bit more for me? Perhaps you could compare some of your experiences.


Structural• Question to determine relations or order

between events / processes

• You stated that in order to be a good school leader a Head teacher needs to be a good teacher. How do you see these qualities belonging together?

dissertation studies

Documents

interviewing data participants

preset number of participants

sampling frame

cases sample frame

random sample

snowballget cases

main title

hatallocate random numbers