getting organized for the class projectfaculty.nps.edu/rdfricke/oa4109/lab 1 -- getting organized...

1

Getting Organized for the Class Project"

Professor Ron Fricker"Naval Postgraduate School"

Monterey, California"

1/1/13

Goal for Lab 1"

•  Discuss Technology Use in the NPS Classroom survey"–  One survey: NPS residents"–  Define and state survey objectives"–  Set up survey teams "

•  Design, fielding, and analysis"–  Review schedule and tasks"

•  Get team preference inputs"

1/1/13 2

Class Project"

•  Real-world survey: Use of technology in the NPS classroom by instructors "–  Compliment to surveys done at NPS in 2010 and

USNA in 2011about mobile learning devices"–  To be conducted for NPS CED-3 & ITACS"

•  Deliverable:"–  Final briefing to CED-3/ITACS on 18 March"

•  See “Class Project” in Sakai for draft schedule and background information"

1/1/13 3

Survey Objective"

1/1/13 4

Design Team Duties & Responsibilities"

•  Responsibilities:"–  Design/revision of questions and instrument,

including necessary background research "–  Obtain respondent contact information"–  Input of the above into SurveyMonkey"

•  Major milestones:"–  Background research: ASAP"–  Respondent information obtained: NLT 24 January"–  Draft/revise questions and instrument: 31 January"

1/1/13 5

Fielding Team Duties & Responsibilities"

•  Primary responsibilities: "–  Pretesting of questions and instrument"–  Field survey and collect results, non-respondent

follow-up"•  Major milestones: "

–  Fielding strategy complete: 3 February"–  Pre-testing complete: 5 February"–  Fielding complete: NLT 23 February"

1/1/13 6

Analysis Team Duties & Responsibilities"

•  Primary responsibilities: "–  Write data safeguarding and analysis plans"–  Analyze survey data"–  Brief final results"

•  Major milestones:"–  Data safeguarding plan complete: 31 January"–  Analysis plan draft: 31 January"–  Data analysis complete: NLT 14 March "

1/1/13 7

Draft Project Schedule"

1/1/13 8

Draft Project Schedule"

1/1/13 9

Design Team Tasks"

•  Next lab class: "–  Complete as much background research as

possible on classroom instructional technology"–  Draft survey outline and questions, brief class"

•  Read ahead to Dillman chapters 4-6 and next week’s lectures"

•  Obtain respondent contact information (i.e., NPS resident student e-mail addresses) "

•  Longer term:"–  Start thinking about SurveyMonkey and getting the

survey entered"

1/1/13 10

Fielding Team Tasks"

•  Upcoming:"–  Start thinking about fielding strategy (read ahead

to Dillman chapters 2 and 7)"•  Longer-term:"

–  Provide feedback to design team from the survey respondent’s perspective"

•  Start thinking about how you’re going to pre-test the survey – more on this in Week 4"

–  Draft announcement e-mail(s), survey cover information, etc. (see previous classes’ materials)"

1/1/13 11

Analysis Team Tasks"

•  Upcoming:"–  Draft the data safeguarding plan"–  Draft the analysis plan"

•  Longer-term:"–  Think about how you’re going to analyze the data

resulting from the Design Team’s survey design"–  Provide regular input to Design Team so resulting

instrument meets analytical requirements"

1/1/13 12

Some Notes About Survey Research (1)"

•  Surveying is always an iterative process"–  Decisions at/about each step affect the other steps"–  There are always multiple solutions"

•  Must assess trade-offs between solutions"•  Perhaps the one case where design-by-

committee adds value"–  Survey has to work for many types of people"–  Need many viewpoints for survey to be robust"–  Can’t direct a survey to be good or to get high

response rates"

1/1/13 13

Some Notes About Survey Research (2)"

•  Attention to detail critical"–  Small things affect respondents’ perceptions of the

survey – and hence data quality"–  A good survey has to get it all right"

•  Must be flexible and be open to change"–  Particularly during fielding phase, must be able to

adjust to what’s happening"•  But is also true in general during survey design

and analysis"•  Finally: It’s hard work…"

1/1/13 14

Finally, A Few Notes About Our Involvement On The Project…"

•  Project is a learning experience"–  We’re gong to let you guys figure out how to apply

the principles we learn about in class"•  We’re the Principal Investigators, so have

final say on product"–  However, functionally, we’ll be advisors and the

coordinator, but you will make the decisions"•  If we disagree, we will let you know, but will try

not dictate a solution"•  We learn by making mistakes "

•  Note: we won’t always have the best solution"–  See points on previous slides about multiple

solutions and design-by-committee"1/1/13 15

What We Have Just Learned"

•  Discussed Technology Use in the NPS Classroom survey"–  Set up survey teams (for each survey)"

•  Design, fielding, and analysis"–  Reviewed schedule and tasks"

1/1/13 16