a meta-assessment of statewide program evaluations
DESCRIPTION
A Meta-Assessment of Statewide Program Evaluations. Jeffrey Pomerantz Carolyn Hank School of Info. & Library Science UNC Chapel Hill @unc.edu. Charles R. McClure Jordon Andrade College of Information Florida State University @fsu.edu. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/1.jpg)
A Meta-Assessment of Statewide Program
EvaluationsJeffrey PomerantzCarolyn HankSchool of Info. & Library ScienceUNC Chapel Hill<pomerantz, hcarolyn>@unc.edu
Charles R. McClureJordon Andrade
College of InformationFlorida State University
<cmcclure, jca07d>@fsu.edu
Tapping the vast reservoir of human knowledge --Louis Round Wilson, founder, 1931
![Page 2: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/2.jpg)
http://is.gd/1eJQ
Tapping the vast reservoir of human knowledge --Louis Round Wilson, founder, 1931
![Page 3: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/3.jpg)
Research questions1. What methodologies are used in LSTA
evaluations?2. How do states’ goals map to LSTA goals?3. What, if any, correlations are there between
methodologies used and states’ and LSTA goals?
4. How successful are different methodologies in providing useful evaluation data about library programs?
![Page 4: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/4.jpg)
Background on IMLS and LSTA2008: IMLS awarded ~$161 M under Grants to
States program
LSTA funds < 15% of total state library fundingBut ≈ 97% of all federal funding to state libraries
IMLS encourages grantees to use Outcome Based Evaluation methods
![Page 5: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/5.jpg)
Rendon’s LSTA goal categoriesA: Establish or enhance electronic linkages among or between libraries.B: Electronically linking libraries with educational, social, or information
services.C: Assisting libraries in accessing information through electronic networks.D: Encouraging libraries in different areas, & encouraging different types of
libraries to establish consortia & share resources.E: Paying costs for libraries to acquire or share computer systems and
telecommunications technologies.F: Targeting library & information services to persons having difficulty using a
library & to underserved urban & rural communities, including children from families with incomes below the poverty line.
The Rendon Group (2003). National Profile: Analyses of the Five-Year Evaluations submitted to the Institute of Museum and Library Services by the State Library Administrative Agencies under the Grants to States program of the Library Services and Technology Act. Washington, DC: Institute of Museum and Library Services.
![Page 6: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/6.jpg)
MethodologyCollected 5-year plans & evaluation reports for 28 states
for 1998-2002 & 2003-2007.
From 5-year plans:1. State’s 5-year goals,2. Connection between state’s goals and LSTA’s goals.
From evaluation reports:1. Methodologies and data collection instruments used,2. Stakeholder groups that provided data,3. Accomplishment of LSTA goals,4. Recommendations.
![Page 7: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/7.jpg)
Methods used in LSTA evaluations
Methodologies used
Number of methods used
![Page 8: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/8.jpg)
Data collected from whom
![Page 9: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/9.jpg)
States’ goals mapping to Rendon
![Page 10: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/10.jpg)
Accomplishment of states’ goals
![Page 11: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/11.jpg)
Accomplishment of Rendon goals
![Page 12: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/12.jpg)
Methods used to evaluate goals
![Page 13: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/13.jpg)
DiscussionQuality and readability of eval reports varied
considerably.
Need more explicit mapping between state & LSTA goals, and eval methodologies.
Only 39% of goals accomplished completely.
![Page 14: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/14.jpg)
Recommendations• To libraries:– Need for clearer goals & measurable objectives.– Need for more appropriate methods.
• To IMLS:– Not just OBE.– Simplify the process.
• To both:– Need for more consistent use of terms.– Need for planning pre-eval & re-program implementation.
![Page 15: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/15.jpg)
Answers to research questions1. What methodologies are used in LSTA
evaluations? A lot of surveys.2. How do states’ goals map to LSTA goals?
Not very well.3. What, if any, correlations are there between
methodologies used and states’ and LSTA goals? Not many.
4. How successful are different methodologies in providing useful evaluation data about library programs? Not very.
![Page 16: A Meta-Assessment of Statewide Program Evaluations](https://reader036.vdocument.in/reader036/viewer/2022062814/568167e5550346895ddd5098/html5/thumbnails/16.jpg)
Thank you!
Jeffrey PomerantzCarolyn HankSchool of Info. & Library ScienceUNC Chapel Hill<pomerantz, hcarolyn>@unc.edu
Charles R. McClureJordon Andrade
College of InformationFlorida State University
<cmcclure, jca07d>@fsu.edu
Tapping the vast reservoir of human knowledge --Louis Round Wilson, founder, 1931