Getting Started with Research

Karen Belfer, VCCTannis Morgan, BCIT

Introductions

Our experiences

Your interests

What brought you to this session today?

Why bother?

Motivations“research” vs investigation as part of own faculty developmentResearch in your discipline or educational research?

Overview

SOTL, applied research, and research

The research process

Research ethics

Tools

Funding

Questions?

Situating Research

SoTL Applied Empirical

Scholarship of Teaching and Learning (SOTL) Research:

Where it startsWho am I as an instructor?

Are my exam questions too hard?

Does a course weblog help students feel connected?

Is group work helping first year math students?

Are employers satisfied with graduates of our program?

Scholarship of Teaching and Learning (SOTL) Research

Teaching ExpandedPlanned reflection and analysisShortest “distance” from current practice

Systematic investigation or exploration via a research process

Manageable scope--scalable

Teaching context as a laboratory

Focus on student learning

Research Process (Carnegie criteria)

Adequate Preparation L

iterature review - 25% of time

Clear Goals R

esearch questions - derived from own experience, curiosity, lit review

Appropriate Methods

flow from research questions - “how” vs “what” determines methods to some degree

Significant Results

Analysis and discussion of what has been investigated, finding flaws in own research, need to investigate further if larger than thought, reflective critique of conducting the research, so what?

Reflective Critique

So what? what does this mean moving forward in your teaching? related to significant results, how does this affect the bigger picture

Effective Presentation

Sharing, communication, reporting

Preparation

Adequate Preparation - lit review - 25% of timeEngage librarians as a resourceGoogle scholar, interinstitutional loans, AskAwayKey wordsSOTL groups, use the networkResearch Ethics Board/ReviewOrganizational system: Refworks, cite-u-like

Literature Review

Theme Sources Methods My comments Do online discussions contribute to a greater sense of community among the students?

Swan et al (2001) Anderson et al 2003)

Surveys Content analysis

Interviews would have been helpful

Does teaching presence have a role in student satisfaction?

George (2003)

surveys Correlated with student perceptions of learning

Etc..

Clear GoalsWhat makes a good research question?

SimpleIt addresses a need or a problem that you encounter as a practitionerIt is researchable, meaning you are able to collect evidence that would answer the question.It is doable given your time and material constraints.It inspires you and has the potential to hold your interest over several months.It is not too general; that would result in a multitude of sub-questions.It is not too narrow; that would rule out the emergence of other possibilities.It cannot be answered Yes or No

How do you develop a research question? need, own experience, curiosity, lit review

Your questions

Share an observation, and what you would like to know

What would this look like as a research question?

Appropriate Methods

How are you going to investigate your question?

Flow from research questions - “how” vs “what” determines methods to some degree

Does your discipline influence your methods?InstrumentsProcess/research design

Organizing and documenting it all

Research Design & MethodsQualitative--How? Why?

Eg. Case studies, ethnographies, grounded theory

Quantitative--What?

Eg. (Quasi-)Experimental, correlational, surveys

Mixed--combination

Evaluation studies--typically program, institutional, or innovation focussed with a clear purpose of assessing the quality and effectiveness

Qualitative Methods

Used to “understand” more deeply. Typically results do not seek to generalize widely.

May involve one or more of the following:focus groups/interviewstext or discourse analysisobservations

Quantitative Methods

Typically seeks to generalize to a large population. It is “hypothetically” more objective and less interpretive.

Examples include one or more of the following:SurveysExperimental DesignContent analysis

Our SOTL/Ed Research Examples

Cardiology applied research - quantitative

surveys, validation of instruments

Net Gen learner - mixed methods

interviews/focus groups; survey

Teaching Presence and Voice Feedback - qualitative

discussion thread; interviews

Question Analysis with Clickers - quantitative

clickers, Item Response theory, Classical Theory of Tests

Matrix: Appropriate MethodsIssue Instruments Analysis Action Plan Instrument Reliability - Students’ responses to

the closed questions in the course experience instrument(s)

- Cronbach’s alpha A reliable instrument has a Cronbach of 0.80 or higher.

Instrument Validity - Students’ responses to the closed questions in the course experience instrument(s)

- Factor analysis A valid instrument will have distinct factors for course and instructor related elements.

Students’ perception of the quality of the: - learning experience (online) - course elements (content, materials, learning activities, instruction, clinical)

- Students’ responses to the closed questions in the course experience instrument(s).

Means, standard deviations

Courses with high values should be used slotted for a focus group; courses with low values should be reviewed.

Action plans - Students’ responses to the open questions in the course experience instrument(s)

Themed, codified and counted

Determine elements that are working well, and things that need to be changed.

Elements affecting the students’ learning experience

- Students’ comments during the focus groups sessions.

Categorize themes, compare themes to XYZ framework List of things that are working and things that need to be changes, and suggestions.

Strategic plan for elements needed to be revised in courses. Recognition plan for successful course deliveries.

ToolsData Gathering

Digital: Text, audio, video, surveys

Backups!

Security and privacy (ethics)

Data Analysis

SPSS/NVIVO/MaxQDA/Microsoft Word Notes

Data visualization tools

Brainstorm

Select one of your research questions and discuss with your partner/group how you would gather evidence/data

What are the challenges you would be faced with? What kinds of roadblocks would you anticipate?

Results

Analysis and discussion of what has been investigated

Finding flaws in own researchneed to investigate further if larger than thoughtreflective critique of conducting the researchso what?

Reflective Critique

So what?

What does this mean moving forward in your teaching?

How does this affect the bigger picture?

Effective Presentation

Sharing

Communication

Reporting

Research Ethics

Human dignity

Free informed consent

Vulnerable persons

Justice and inclusiveness

Minimizing (balancing) harm

Privacy and confidentialityAudio recording vs video recordingSecurity of data

FundingFunding Calendar (Science focussed)

http://www.bcit.ca/appliedresearch/funding/fundingcalendar.shtml

Funding Opportunities (Social Science focussed)h

ttp://ltcollaboratory.org/funding

Support

Peer support

Mentor support

Collaboration - with students, with other institutions

Role of Teaching and Learning Centres?

http://ltcollaboratory.org/

Summary

Think about your orientation (and your disciplinary orientation) to doing research

Think about your comfort level with qualitative and quantitative methods

Think about your question

Think about the forms of evidence/data you might want to access or collect

Parting WordsWe believe the time has come to move beyond the tired old "teaching versus

research" debate and give the familiar and honorable term "scholarship" a broader, more capacious meaning, one that brings legitimacy to the full scope of academic work. Surely, scholarship means engaging in original research. But the work of the scholar also means stepping back from one's investigation, looking for connections, building bridges between theory and practice, and communicating one's knowledge effectively to students.

E. L. Boyer

Thanks

View/download the presentation and access the resources at

http://researchworkshop.wordpress.com

Models

Boyer 1990

Examples in relation to model

Boyer 1990 model

Motivate Faculty TeachingDiscipline

InterdisciplineOutside

Awareness Raise awareness

Reflection Discuss and reflect about teaching practice

Investigation Identify SoTL research, define research strategies.

Application Analyze research finding, interpret and make changes to the classroom.

Communication Communicate findings

https://my.wsu.edu/portal/page?_pageid=177,280640&_dad=portal&_schema=PORTALWashington State U.

PROFESSIO NAL CONTRIBUTI ONS: Engages in literature of teaching and learning; initiates and/or participates in educational research to answer research questions related to teaching and learning; shares findings through presentations and archival publications; supports, mentors, or trains others in teaching and educational scholarship.

Developing (low impact) Advancing (moderate impact) Mastering (high impact) Performance Area 1 2 3 4 5 6 STATE OF

KNOWLEDGE Has little familiarity with important literature on teaching and learning

Aware of key references and authors important to teaching and learning

Familiar with foundational references, important current articles, and leaders in teaching and learning in relevant areas

APPLICATION OF

SCHOLARSHIP Literature on teaching and learning not used to guide innovation and improvement in teaching

Adopted some practices reported in literature to improve teaching and learning

Literature used to stimulate and guide innovations and to assess impacts on teaching and learning

ORIGINAL

RESEARCH Conducts or participates in little or no original research in teaching and learning; rarely shares results with others

Planning or conducts some research that may contribute to literature on teaching and learning

Regularly contributes to original research that contributes to archival literature on teaching and learning

RESEARCH GRANTS Has not submitted grant/gift proposals related to teaching and learning during recent five years

Has submitted one or more grant/gift proposals related to teaching and learning during recent five years

Has procured multiple grants or gifts (some competitive) for teaching and learning initiatives during recent five years

COMMUNITY

BUILDING Rarely collaborates with others on teaching and learning innovations or curricular development

Regularly collaborates with others on teaching and learning innovations or curricular development

Supports and trains others locally and beyond to enhance research and practice for teaching and learning

Resources

Good explanation of Evaluation research http://www.edu.plymouth.ac.uk/resined/evaluation/index.htm