qualitative methods in assessing libraries, users and ict applications dr. r. sevukan associate...
Post on 28-Dec-2015
214 Views
Preview:
TRANSCRIPT
Qualitative Methods in Qualitative Methods in Assessing Libraries, Assessing Libraries,
Users and ICT Users and ICT ApplicationsApplications
Dr. R. SevukanDr. R. SevukanAssociate Professor & HeadAssociate Professor & Head
Dept. of Library & Information ScienceDept. of Library & Information SciencePondicherry UniversityPondicherry University
Puducherry, IndiaPuducherry, Indiasevukan2002@yahoo.comsevukan2002@yahoo.com
NACLIN, 9-11 December, 2014NACLIN, 9-11 December, 2014
What is an Assessment?What is an Assessment?
It is the process of gathering It is the process of gathering information from the stakeholders information from the stakeholders using various methods to using various methods to systematically gauge the systematically gauge the effectiveness of an institution.effectiveness of an institution.
Why is assessment Why is assessment important?important?
Are we delivering what we think we Are we delivering what we think we are delivering?are delivering?
Are customers getting what they are Are customers getting what they are supposed to be getting?supposed to be getting?
Are customers satisfied with what Are customers satisfied with what they are provided?they are provided?
Is there a way to deliver the Is there a way to deliver the information better, thereby information better, thereby promoting satisfaction?promoting satisfaction?
Answers to these questionsAnswers to these questions Information sources needed for research, Information sources needed for research,
teaching, and learning; teaching, and learning; Reasons and frequency of library use; Reasons and frequency of library use; Campus computer network connectivity; Campus computer network connectivity; Use of electronic resources; Use of electronic resources; Instructional needs and effectiveness; Instructional needs and effectiveness; Satisfaction on the services available; andSatisfaction on the services available; and Enhance the quality of services rendered.Enhance the quality of services rendered.
Qualitative Vs. Qualitative Vs. QuantitativeQuantitative
Qualitative Research is primarily exploratory is primarily exploratory research used to gain an understanding of research used to gain an understanding of underlying reasons, opinions, and motivations. It underlying reasons, opinions, and motivations. It provides insights into the problem or helps to provides insights into the problem or helps to develop ideas or hypotheses for potential develop ideas or hypotheses for potential quantitative research.quantitative research.
Quantitative Research is used to quantify the is used to quantify the problem by way of generating numerical data or problem by way of generating numerical data or data that can be transformed into useable data that can be transformed into useable statistics. It is used to quantify attitudes, statistics. It is used to quantify attitudes, opinions, behaviors, and other defined variables – opinions, behaviors, and other defined variables – and generalize results from a larger sample and generalize results from a larger sample populationpopulation
Qualitative Methods: Qualitative Methods: Characteristics Characteristics
Naturalistic – Naturalistic – Investigate social processes as they unfold Investigate social processes as they unfold naturally rather than reasoning from either the conditions naturally rather than reasoning from either the conditions under which they occur or the outcomes that correlate with under which they occur or the outcomes that correlate with themthem
Emergent – Emergent – Willingness to adapt methods of analysis as Willingness to adapt methods of analysis as knowledge and understanding deepensknowledge and understanding deepens
Purposeful – Purposeful – Cases are selected because they are Cases are selected because they are “information rich”“information rich”
Source: Patton, Michael Q. Source: Patton, Michael Q. Qualitative Research and Evaluation MethodsQualitative Research and Evaluation Methods . 3. 3rdrd ed. (Thousand Oaks, CA: Sage, 2002), pp. 40- ed. (Thousand Oaks, CA: Sage, 2002), pp. 40-41. 41.
Data Collection Characteristics of Data Collection Characteristics of Qualitative MethodQualitative Method
Thick description – Thick description – The goal is deep understanding The goal is deep understanding about phenomena and experiences.about phenomena and experiences.
Personal experience and engagement – Personal experience and engagement – The The researcher is the primary instrument of data collection and researcher is the primary instrument of data collection and analysis.analysis.
Empathic stance – Empathic stance – Sensitivity, respect, awareness, Sensitivity, respect, awareness, responsiveness, and openness.responsiveness, and openness.
Attention to process – Attention to process – Situational and contextual Situational and contextual changes occur throughout the life of the study.changes occur throughout the life of the study.
Analytical Characteristics of Analytical Characteristics of Qualitative MethodQualitative Method
Unique case orientation – Unique case orientation – Every case is special and Every case is special and uniqueunique
Inductive and creative – Inductive and creative – Research process guided by Research process guided by analytical principles rather than rulesanalytical principles rather than rules
Holistic perspective – Holistic perspective – Phenomena under study are Phenomena under study are complex and cannot be reduced to a few variables complex and cannot be reduced to a few variables
Context sensitivity – Context sensitivity – Avoid broad generalizations; Avoid broad generalizations; investigation does not rest in isolation from the larger investigation does not rest in isolation from the larger social/ cultural/ economic/political environmentsocial/ cultural/ economic/political environment
Reflexivity – Reflexivity – Critical self-awareness of one’s own Critical self-awareness of one’s own presence in the research processpresence in the research process
Selective Applications inSelective Applications inLibrary SettingsLibrary Settings
To assess individualized outcomesTo assess individualized outcomes To explore internal dynamics of programs and servicesTo explore internal dynamics of programs and services To evaluate critical cases of program or service utilizationTo evaluate critical cases of program or service utilization To acquire details about program implementationTo acquire details about program implementation To obtain information about the nuances of program or service To obtain information about the nuances of program or service
qualityquality To personalize the evaluation process and/or outcomes To personalize the evaluation process and/or outcomes
assessmentassessment To discover effects of specific programs or servicesTo discover effects of specific programs or services To add depth, detail, and meaning to statistical dataTo add depth, detail, and meaning to statistical data To provide privileged access to processes, causes, and effects.To provide privileged access to processes, causes, and effects.
Source: Patton, Michael Q. Source: Patton, Michael Q. How to Use Qualitative Methods in EvaluationHow to Use Qualitative Methods in Evaluation. 1. 1stst ed. (Thousand Oaks, CA: Sage, 1987), pp. 41-42. ed. (Thousand Oaks, CA: Sage, 1987), pp. 41-42.
Some Techniques for Collecting Some Techniques for Collecting Assessment DataAssessment Data
In-Depth InterviewsIn-Depth Interviews – – speaking with and listening to speaking with and listening to our usersour users
ObservationObservation – – witness what is going on in a particular witness what is going on in a particular setting and explore the experiences of otherssetting and explore the experiences of others
Qualitative Case StudiesQualitative Case Studies – – holistic understanding of holistic understanding of social phenomena in library settingssocial phenomena in library settings
In-Depth Interviews In-Depth Interviews Outcomes for AssessmentOutcomes for Assessment
What is meaningful or important in the respondent’s What is meaningful or important in the respondent’s own voice?own voice?
Researcher’s knowledge, expertise, and interpersonal Researcher’s knowledge, expertise, and interpersonal skills can be used to explore interesting or unexpected skills can be used to explore interesting or unexpected concepts or issues raised by respondentsconcepts or issues raised by respondents
Opportunity for deep probing of issuesOpportunity for deep probing of issues
Source: Patton, Michael Q. Source: Patton, Michael Q. Qualitative Research and Evaluation MethodsQualitative Research and Evaluation Methods . 3. 3rdrd ed. (Thousand Oaks, CA: ed. (Thousand Oaks, CA: Sage, 2002)Sage, 2002)
Observation: Outcomes for Observation: Outcomes for AssessmentAssessment
Can facilitate multidimensional and long-term Can facilitate multidimensional and long-term imbeddedness in the “natural setting”imbeddedness in the “natural setting”
Melds together aspects of looking and listening Melds together aspects of looking and listening and watching and asking (connect what people and watching and asking (connect what people are saying to what they are actually doing)are saying to what they are actually doing)
Unmasks everyday interaction between people Unmasks everyday interaction between people and organizations; reveals the mundaneand organizations; reveals the mundane
Cases can be revisited repeatedlyCases can be revisited repeatedly
Case Studies Design ConsiderationsCase Studies Design Considerations IndividualsIndividuals – comparative sample of individual experiences – comparative sample of individual experiences
Groups – Groups – sample of individuals who share a common identitysample of individuals who share a common identity
Communities – Communities – sample of individuals with ethnic, cultural, sample of individuals with ethnic, cultural, spiritual or other types of characteristics in commonspiritual or other types of characteristics in common
Programs – Programs – comparative sample of projects or services comparative sample of projects or services intended to meet a public needintended to meet a public need
Policies – Policies – comparative sample of plans of actioncomparative sample of plans of action
Organizations – Organizations – comparative sample of a formal group of comparative sample of a formal group of people with one or more shared goalspeople with one or more shared goals
Qualitative Case StudiesQualitative Case StudiesApplications for AssessmentApplications for Assessment
IllustrativeIllustrative: Descriptive and intended to add “realism” and in-depth : Descriptive and intended to add “realism” and in-depth examples to other information about a program or policy.examples to other information about a program or policy.
ExploratoryExploratory: Also descriptive, but aimed at generating hypotheses for : Also descriptive, but aimed at generating hypotheses for later investigation rather than for purposes of illustrating.later investigation rather than for purposes of illustrating.
Deviant/Critical InstanceDeviant/Critical Instance: Examines a single instance of unique interest : Examines a single instance of unique interest or serves as a critical test of an assertion about a program, problem, or or serves as a critical test of an assertion about a program, problem, or strategy.strategy.
Program Implementation:Program Implementation: Investigates operations, often at several sites, Investigates operations, often at several sites, and with a subjective, value laden, and/or affective emphasis.and with a subjective, value laden, and/or affective emphasis.
Program Effects:Program Effects: This application uses the case study to examine This application uses the case study to examine causality and usually involves multi-site, multi-method assessments.causality and usually involves multi-site, multi-method assessments.
Cumulative:Cumulative: This brings together findings from many case studies to This brings together findings from many case studies to answer an evaluation question, whether descriptive, normative, or cause-answer an evaluation question, whether descriptive, normative, or cause-and-effect.and-effect.
The Challenge ofThe Challenge ofAssessment in LibrariesAssessment in Libraries
Traditional statistics emphasize inputs, Traditional statistics emphasize inputs, expenditures, acquisitions, holdings, etc.expenditures, acquisitions, holdings, etc.
Helping funding agencies understand success of Helping funding agencies understand success of investmentinvestment
No demonstrable relationship between No demonstrable relationship between expenditures and service qualityexpenditures and service quality
Lack of metrics describing outcomes: success Lack of metrics describing outcomes: success from the user’s point of viewfrom the user’s point of view
Need to redesign library services to better meet Need to redesign library services to better meet changing patterns of usechanging patterns of use
Building the climate, tools, and skill set for library Building the climate, tools, and skill set for library assessmentassessment
No Single Method!!!No Single Method!!!
““The difficulty lies in trying to find a single model or The difficulty lies in trying to find a single model or set of simple indicators that can be used by set of simple indicators that can be used by different institutions, and that will compare different institutions, and that will compare something across large groups that is by definition something across large groups that is by definition only locally applicable – i.e., how well a library only locally applicable – i.e., how well a library meets the needs of its institution. Librarians have meets the needs of its institution. Librarians have either made do with oversimplified national data or either made do with oversimplified national data or have undertaken customized local evaluations of have undertaken customized local evaluations of effectiveness, but there has not been devised an effectiveness, but there has not been devised an effective way to link the two.” effective way to link the two.”
Sarah Pritchard, Library Trends, 1996 Sarah Pritchard, Library Trends, 1996
““Let us replicate but not duplicate Let us replicate but not duplicate research”research”
--- H.K.Kaul--- H.K.Kaul
top related