prr 389 module 1: concepts, philosophy, process module 2: goals, objectives, perf. measures module...

PRR 389

• Module 1: Concepts, Philosophy, Process• Module 2: Goals, Objectives, Perf. Measures

• Module 3: Data Gathering & Analysis

• Module 4: Forecasts & Impacts

• Module 5: Communication/Reporting

Planning/Research/Evaluation

• Concepts: Definitions & Philosophy

• Purposes: Why plan, evaluate?

• Processes : Systematic Steps

• Types of Plans, Evaluations, Research

Definitions

• Planning = preparation for decision making

• Evaluation = process of judging the merit or worth of something

• Research = application of scientific methods to answer questions

• Pairs: Evaluation research, research planning, planning evaluation

Related terms

• Design

• Policy

• Management

• Data gathering …

• Decision making

• Analysis

Comprehensive planning

• Systems of facilities and programs

• Multiple providers and stakeholders

Why Plan?

• Prepare for/anticipate future• Establish goals, directions• Coordinate various activities & programs• Balance needs/wants with available resources• Establish systematic steps to achieve goals• Requirements for funding

Messes

• 1. Not well defined

• 2. Multiple ill-defined objectives

• 3. Multiple decision makers

• 4. Incomplete information.

• 5. Dynamic.

• 6. Multiple stakeholders

• 7. Uncertainty

Characteristics of Recreation & Tourism

• Nature of R/T– Diversity of activities

– Hard to define

– Freedom of choice

– A “frill”, worthy of serious study?

• Supply Characteristics– Services

– Fragmented providers/industries

– Public-private– non-profit mix

– Role of consumer in providing experience

– Externalities, public goods, common property resources

Characteristics (cont)

• Demand Characteristics– Demand for what?– Dispersed use– Substitution not understood– Multi-purpose trips– Peaking of demand– Prices/markets

• Methodological– Fuzzy concepts– Weak Measures/measurements– Planning- management-research gaps– Interdisciplinary, young as sciences go

Trends

• Government-private roles, privatization

• Federal, state, local roles

• Partnerships, cooperation vs competition

• Increasing accountability

• Convergence of public & private/business approaches

• Scale, pricing

• Shifting markets, activities

• Linkages with health, crime, aging, economic dev., diversity & other societal issues

Incremental vs Comprehensive Planning

• Lindblom, “science of muddling through” planning is ..– Incremental

– Remedial

– Serial

– Exploratory

– Fragmented

– Satisficing

– Iterative

Patton & Sawicki Tips for planning/policy analysis

• Focus quickly on central decision criteria

• Avoid tool box approach

• Learn to deal with uncertainty

• Say it with numbers

• Make analysis simple and transparent

• Check the facts

• Learn to advocate positions of others

• Give client analysis, not decisions

• Push boundaries of policy envelope

• No such thing as correct, rational, complete, comprehensive analysis

Steps in Planning Process

• Define the system• Establish goals and objectives• Gather information• Generate Alternatives• Evaluate Alternatives• Select & Implement• Monitor• Iterate

Types of plans

• By geographic scope/extent– Building, Facility, Site Master Plan (Design)– Community, Regional, State, National, System of Parks/Programs

• By Activity/Function– Housing, transportation, recreation tourism, land use, open space,

physical plan, comprehensive plan …– Program, maintenance, safety, security, risk management, budget

personnel, marketing…

• By Time– Feasibility, Short – Long Range, Strategic Plan, annual

performance plan, EIS.

Specific Types

• Community recreation plan

• SCORP

• Environmental impact statement

• Business Plan

• Feasibility study

• GMP, LMP, …

• Community or Regional tourism plan

Why Evaluate - the Academic list

• To assess merits of alternative programs• To discover whether & how well objectives are

being fulfilled• To determine the reasons for successes &

failures.• To uncover the principles underlying a

successful program.• To refine, revise, update or track a program

Why Evaluate- "The Real List"

1. Because we are required to

2. To make better decisions.

3. To learn from experience

4. To justify programs

5. To kill programs

Research Evaluation

Evaluation Research

Purposes of Research

• Answer management questions - applied research• For sake of knowing - pure or basic research• Answer research questions - methodological• Develop/test management alternatives -

developmental research

• Assess worth or merit of programs - evaluation research

Research Purposes - II

• Exploratory

• Descriptive

• Explanatory

• Predictive

Ways of Knowing or Establishing belief or truth

• Tradition• Authority• Repetition/Tenacity• Science

Science

• Body of Knowledge

• systematic• abstract• general• parsimonious

• Method of Inquiry

• logical– induction

– deduction

• self-corrective• empirical

Scientific Management

• Application of scientific principles to management and decision making

– systematic information gathering– empirical, objective, self-corrective

Process -- Steps

Research

•define problem

•objectives/hypotheses

• literature review

•research methods

•gather data/analysis

•conclusions

•further research

Evaluation

• describe program

• evaluation criteria

• program scoping

• evaluation methods

• gather data/analysis

• conclusions

• modify program?

Planning

• define system

• specify goals/objectives

• gather info

• generate alternatives

• evaluate alternatives

• choose & implement

• monitor, evaluate

Iterate

Types of Evaluationby Program Stage

• formative (conceptualization/design)]

• process (implementation)

• summative (outcomes, impacts, efficiency)

Types - By Approach

• Standards– norm-based– criterion-referenced

• Goals and objectives

• Impacts or effects

Types- by method

• Qualitative - quantitative

• Secondary data - Primary

• Survey - Experiment

• Internal vs external

• Who - Peers, participants, superiors, subordinates, scientists

Evaluation Criteria

• Effort - qnty and qlty of inputs

• Performance - qnty and qlty of outputs

• Adequacy - meet needs?

• Efficiency - benefits/costs

• Equity - distributional issues, fairness

Process Evaluation

• Identifies how and why program works– attributes – recipients– conditions– effects

• single or multiple

• intended or side effects

• timing & duration, long/short term

• cognitive, affective or behavioral

Types by program area/subarea

• Emphasis: Parks, Tourism, Interp, Program Mgmt, Commercial Rec, ...

• Program : lands, facilities, personnel, budget, marketing, PR, maintenance, design, policies, plans

Types of Research

• Basic - Applied

• Exploratory, Descriptive, Explanatory, Predictive

• In-house / out- house• by discipline - sociological, psych, economics

• by topic - boating, wilderness, legal, tourism, ...

• by method - survey, expmt, ...

Examples of Evaluation Studies

• Community needs assessment

• Feasibility study, SWOT analysis

• Program Audits, Market audit

• Cost/benefit, Cost effectiveness analysis

• Impact assessment

• PPBS, MBO, GPRA, CAPRA, LAC, ...

• Peer review

• Importance-performance analysis

AEA Principles for Evaluators

• Systematic Inquiry

• Competence

• Integrity/Honesty

• Respect for people

• Responsibility for public welfare

Scriven- main points

• Merit vs Worth• Grading vs Ranking• Facts and values• Big six: program, personnel,performance, policy,

proposal, product• Need vs market assessment• Holistic vs analytic• Indicators vs true measures of merit• Goal Achievement Eval vs Goal Free Evaluation

Goal free evaluation (GFE)

• Don’t worry about goals – look at all effects• Problems with goal achievement evaluation

– Whose goals, how measured?– Unintended/side-effects; interaction effects– Values left behind– Costs, alternatives/competing programs– Generalizability– Synthesis