logic model swot analysis and action plan student learning outcomes three ways to evaluate your...

LOGIC MODEL

SWOT ANALYSIS AND ACTION PLAN

STUDENT LEARNING OUTCOMES

Three Ways to Evaluate your Program:

Dr. Patricia FullmerCollege Reading and Learning Association Annual Conference

November, 2011

Agenda

Overview of EvaluationLogic ModelSWOT Analysis and Action PlanStudent Learning Outcomes

Learning Outcomes

Gain an overview of evaluation.Create a logic model.Generate a SWOT analysis and action

plan.Craft student learning outcomes.

Overview of Evaluation: Types

Summative:Document the results of the

program.

Formative:Improve the program.

Why Evaluate?Required by Funding

Sources: Government Grants University/College funds

Required by Accrediting Organizations: Middle States, Southern

States, etc. Council for the

Advancement of Standards in Higher Education (CAS)

Self-Improvement: Determine how and

where to improve your program’s services.

Document Effectiveness: Demonstrate your

program is valuable and worthwhile!

How do you conduct Evaluation?

Evaluation Process:1. Who does the evaluation?

Evaluation committee/staff2. Evaluation Design:

Determine evaluation questions Determine what information to collect and when Determine the source, methods and

instrumentation of data collection Examine information: Analysis and

Interpretation Generate results

3. Use of evaluation results

Characteristics of Good Evaluation

Comprehensive Uses variety of methods Uses multiple sources Considers multiple perspectives Uses comparisons

SystematicContinuous

Direct MeasuresDirect Measures Indirect MeasuresIndirect Measures

Pretest/PosttestProjectsPortfoliosProfessional

subject area assessments

Self Reports: Surveys Interviews Focus Groups Questionnaires

Grades

Methods and Instruments

Collecting Data: Considerations

ValidityReliabilityQuantitativeQualitativeRubricsProtocolsPopulation SamplingTiming-When to collect data

Use of the Results of Evaluation

1. Demonstrate effectiveness 2. Make decisions for ongoing

improvement: Modify goals and processes Determine an

improvement/action plan Allocate funds Apportion resources

THE INITIAL STEP IN THE EVALUATION PROCESS

Logic Model

What is a Logic Model?

Describes your program visuallyPicture of the program’s: “Theory of

Change”: Events & activities that are expected to create a

change, resolve a problem-describes the desired results

Shows what leads to the desired results: Actions Results

Benefits of a Logic Model

Builds mutual understandingHelps initiate an evaluation:

Determine the wanted outcomes Decide what to measure and assess Establish at what point in time to conduct the

evaluation and collect the data

Summarizes the program Improves planning Identify gaps in services Guides use of resources and funds

Qualities of a Logic Model

Group ProcessMeaningfulReasonableRealistic/AttainableMeasurable

Components of a Logic Model

InputsOutputsOutcomes/ImpactAssumptionsExternal Factors

Where to Start?

AT THE END!!! THEN PROCEED BACKWARDS!!!

1. What are the desired outcomes? 2. Who will be the participants?3. What will be the services and

activities?4. What are the resources needed?

Inputs Outputs Outcomes -- Impact Activities Participation Short Medium Long

What we invest:

Funding: University funds Title III funds Act 101 funds

Staff: Tutors Peer Tutors Secretary Administrators

Training for staff-CRLA Certified

Online programs and access codesMini-Lesson plans and exercises

Collaboration with academic departments and administrators

What we do:

Tutoring Labs

Online tutoring programs

Review mini-lessons

Individual tutoring

Writing Portfolio assistance

Workshops

Summer Bridge

Peer Tutoring

Act 101 Program

Who we reach:

Students

Students’ Families

Faculty

Administrators

Alumni

This semester’s results:

Students increase their skills to college level

Students earn a B or above in their course

Students increase life skills and coping skills

Students adjust to college life

Next semester/next year’s results:

Students increase their GPA

Students persist and return the next semester/next year

Students become successful academically, socially, and emotionally at Lincoln

Lincoln’s retention rate increases

Results in 4-6 years and more:

Students successfully graduate Lincoln with skills for graduate school or the fast track in a corporation

Lincoln’s graduation rate increases

Lincoln’s reputation increases

More and better prepared students are attracted to Lincoln

Growth of Lincoln

Assumptions: External Factors:Following best practices in our services will help students increase their skills. LocationIncreased student skills will improve retention and graduation rates. EconomyIncreased graduation rates will help attract students and increase Lincoln’s reputation. State and federal fundingPerception of LRC by Faculty, Administrations, and Students

University of Wisconsin-Extension (2003). Enhancing program performance with logic models. Retrieved from [email protected]

Learning Resource Center Logic Model

Inputs Outputs Outcomes -- Impact

Activities Participation Short Medium LongWhat we invest:

Professional Writing Tutors

Mini review grammar lessons and exercises

Required tutoring laboratories

My Writing Lab software

Individual assistance for all students in writing

Conducive lab environment

Computer and Internet Access

What we do:

Tutoring Labs

Writing Labs that address grammar

Drop-in tutoring

Appointment tutoring

Writing Portfolio Assistance

Student development workshops

Who we reach:

Students

Students’ Families

Faculty

Administrators

Alumni

This semester’s results:

Students gain an understanding of the structure of an essay

Ability to construct a topic sentence and thesis statement with areas of support

General knowledge of proper grammar and punctuation

Faculty collaboration

Next semester/next year’s results:

Ability to construct a thesis driven essay

Students utilize the LRC on a regular basis

Expansion of students’ vocabulary

Ability to conduct research properly

Understanding of proper work citation

Results in 4-6 years and more:

Students are able to write proficiently across the curriculum

Satisfaction of writing portfolio requirements

Ability of students able to write a comprehensive and coherent response to graduate level exams

Assumptions: External Factors:Following best practices in our services will help students increase their skills. LocationIncreased student skills will improve retention and graduation rates. EconomyIncreased graduation rates will help attract students and increase Lincoln’s reputation. State and federal fundingPerception of LRC by Faculty, Administrations, and Students


Writing Lab Logic Model

AN INTERNAL AND EXTERNAL PERSPECTIVE OF YOUR PROGRAM

SWOT Analysis and Action Plan

SWOT AnalysisSWOT Analysis Action PlanAction Plan

StrengthsWeaknessesOpportunities

Threats

Build on strengthsTake advantage of

opportunitiesAddress

weaknessesAddress threats

SWOT Analysis and Action Plan

Strengths: What is done well?Evaluation of success indicators has resulted in collaboration on content goals and agreement to move to a mastery system addressing core skills of college English grammar.

Tutors developed several writing workshops that were presented in a broad range of academic disciplines.

Tutors make classroom visits at beginning of semester.

Tutors continue to offer extended hours for make-up labs and drop-ins for students seeking professional tutoring.

The Writing Lab has a professional staff that is dedicated to nurturing student success.

All of the tutors have earned CRLA Level I & II certification and continue training on a regular basis each semester.

Weaknesses: What could be improved?Factors for poor student attendance include the following potential causes:

1. Since students are not given credit for labs.

2. The LRC continues to have a credibility issue.

3. Students do not receive a letter grade for the lab.

4. There is no uniformity among professors teaching the course about how best to incorporate lab work in the overall grade for the course.

Students with low proficiency in reading and writing are often hesitant to participate in lab sessions.

One weekly 50-minute lab session is insufficient time.

Opportunities: What are the opportunities open?Design student development workshops and increase in-class presentations of workshops in order to improve students’ skills and knowledge of the writing process and expand the visibility of the LRC.

Creation of a mastery system addressing core skills of college grammar, to improve students’ proficiency in reading and writing.

Threats: What are the obstacles?Lack of competitive salaries or salary raises contributes to low tutor retention and high tutor turnover.

Some of the university’s faculty population holds a poor perception of the LRC which results in lack of support from individual faculty or entire departments.

Success students project a negative attitude towards the Writing Lab which diminishes students’ motivation and results in a decline in attendance during the semester.

Writing Lab SWOT Analysis

Improvement EvidenceThe Writing Lab will focus on grammar mastery. The student will be required to master 20 identified areas of the 26 total areas of the Study Plan in My Writing Lab at 85% to be eligible to test out of the lab.

To test out of lab, students in English 098 and English 099 will be required to score 85% on the post-test at midterm time. Students will be allowed to take the post test twice.

Lab tutors will focus on sentence grammar areas in My Writing Lab work.

Campbell, M. (1983). "Mastery Learning in the College Learning Center." Paper presented at the National Association for

Remedial/Developmental Studies Conference, Little Rock, AK, March 1983, 19 pp. (ED 247 592).

The study found that students need to understand the task at hand, the process, and the assessment involved in order for there to be successful learning.

Input from Lincoln University English Department Composition Committee.

Tutors will review sentence grammar facts twice in the semester.

In weeks 6 and 11, the tutors will review previously covered material.

Adult Literacy and Basic Skills Unit (1991). A self-directed approach to English grammar. Kingsbourne House, 229-231 High Holborn, London, England WC1V 7DA, United Kingdom.

The Writing Lab will increase its hands-on activities to help facilitate the mastery topic of punctuation in order to improve students’ writing aptitude.

Students will write sentences in lab.

Boylan, H. (2002). What works: Research-based best practices in developmental education. Boone, NC: Continuous Quality Improvement Network/National Center for Developmental

Education.

Accommodate diversity through varied instructional methods. Use technology with moderation.

Brophy, L. (1988). Educating teachers about managing classrooms and students. Teaching and Teacher Education, 4(1), 3.

This study found that increased time on task improved students’ skills.

Information from the student survey.

Writing Lab Action Plan

AN EVALUATION OF THE IMPACT OF YOUR PROGRAM

Student Learning Outcomes

Benefits of Student Learning Outcomes

Fulfills accreditation requirements

Fulfills funding requirementsEstablishes goalsTells us what success will look like!

SMART Outcomes

SpecificMeasureableAttainableResults-orientedTimed

Student Learning Outcomes

Assessment Method

Findings Modified Student Learning Outcomes

Action Plan

Students will increase their skills in the area of sentence grammar.

English 098 students will show a 50% improvement from the pre test to the post test in sentence grammar for those who attend six or more sessions.


Results of pre and post diagnostic tests.

For students who attended six or more sessions:

English 098 students show a 50% improvement in sentence grammar.

English 099 students show a 47% improvement in sentence grammar.

Students will increase their skills in the area of sentence grammar.



Students will improve sentence grammar by:

Focusing on mastering all sentence grammar areas in My Writing Lab.

Write sentences in lab. Receive tutor assistance with the online program.

Tutors giving students two review lessons during the semester to help reinforce grammar facts.

Writing Lab Student Learning Outcomes

References

Adult Literacy and Basic Skills Unit (1991). A self-directed approach to English grammar. Kingsbourne House, 229-231 High Holborn, London, England WC1V 7DA, United Kingdom.

Boyd, H. (2002, November 22). Use comparison groups to strengthen your evaluation. (Audio Conference). Retrieved from: http://www.uwex.edu/ces/pdande/resources/pdf/agenda.pdf

Boylan, H. (2002). What works: Research-based best practices in developmental education. Boone, NC: Continuous Quality Improvement Network/National Center for Developmental

Education.

Brophy, L. (1988). Educating teachers about managing classrooms and students. Teaching and Teacher Education, 4(1), 3.

Campbell, M. (1983). Mastery learning in the college learning center. Paper presented at the National Association for Remedial/Developmental Studies Conference, Little Rock, AK, March 1983, 19 pp. (ED 247 592).

References (con’t)

Council for the Advancement of Standards in Higher Education (2008). CAS Self-assessment guide for learning assistance programs. CAS: Washington, DC.

Council for the Advancement of Standards in Higher Education (2008). History of CAS.from http: //www.cas.edu

Dwyer, C. A., Millett, C. M., & Payne, d. G. (2006). A culture of evidence: Postsecondary assessment and learning outcomes. Princeton, NJ: Educational Testing Service.

Kellogg, W. K. (2004). Evaluation handbook. . W. K. Kellogg Foundation: Battle Creek, Michigan.

Kellogg, W. K. (2004). Logic model development guide. W. K. Kellogg Foundation: Battle Creek, Michigan.

Nichols, J. O. (2008). The administrative unit assessment handbook: Guidelines for planning and implementing. Measuring student support services & administrative outcomes. Orlando, FL: University of Central Florida. Retrieved from: http://oeas.ucf.edu.doc/adm_assess_handbook.pdf

Strategy: Create and implement the best strategy for your business, (2005). Boston, MA: Harvard Business School Press.

References (con’t)

SWOT analysis II: Looking inside for strengths and weaknesses, (2006). Boston, MA: Harvard Business School Press.

Taylor-Powell, E. & Henert, E. (2008). Developing a logic model: Teaching and training guide. Madison, WI: University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation.


Wargo, M. C. (2007). Handbook for program assessment in administrative/educational support units. Western Carolina University Cullowhee, NC. Retrieved from: http://www.wcu.edu/WebFiles/PDFs/AssessmentHandbook_AES.pdf

Weiss, C. H. (1972). Evaluation research: Methods for assessing program effectiveness. Prentice-Hall, Inc.: Englewood Cliffs, NJ.

logic model swot analysis and action plan student learning outcomes three ways to evaluate your...

Documents