Outcome-Based Management Program

Data Collection Requirements – Program Instructors

- 2 -- 2 -

Data Collection Background

The Outcome-Based Management (OBM) program will require improved and consistent data collection requirements to ensure that quality data is entered on a timely basis

Why are we updating the data collection requirements for clubs?

The OBM program will require more numerical data to understand the clubs’ impact on our members

Better data collection may streamline work associated with program reporting (i.e., TEEN Reach, monthly reports) and grant writing

Which data collection processes have updated requirements for Program Instructors?

Club Attendance Tracking

Selected Program Attendance Tracking*

Outcome Measurement Toolkit Administration

Selected Survey and Test Administration*

Who is responsible for meeting these updated requirements?

All staff will be held accountable for meeting these updated requirements through regular service center oversight and the annual review process

You may be meeting some of these requirements, but we need to meet them all to be successful!

*Requirements covered in this guide

- 3 -- 3 -

Data Collection Requirements

The updated data collection requirements encourage consistent data acquisition across clubs

Data Collection Process

Process Description

Usage Updated Requirements

Selected Program Attendance Tracking

Tracking member attendance at selected programs

Used with other metrics to see how program attendance impacts members

Every child must check in and out with the Program Instructor to participate in selected programs and activities

Club staff must enter program attendance within two days of program session

Batch scanning is acceptable for data entry

Selected Survey and Test Administration

Administering pre / post surveys and tests developed by BGCA / clubs on a regular basis

Used with program attendance to analyze changes in attitudes, behaviors, and knowledge

Selected pre / post surveys and tests must be administered to all members 2x per program at the beginning and end of program

Surveys / tests must be administered each time the program is run

All answers must be inputted into KidTrax

These requirements are important because operations will only review data entered in KidTrax

- 4 -- 4 -

Enforcement and Staff Impact

To make sure we meet these new requirements, the service center will monitor clubs’ data on a regular basis and take appropriate actions to ensure data is entered accurately and timely

• Reviewed by service center on an on-going basis

• Review will ensure data entered within two days of program session

Selected Program Attendance Tracking

• These requirements are expected to be met by all staff members and will be a part of the performance review process

• Clubs that are not in compliance will have a letter sent to the Club Director and Board President

• Staff disciplinary action will be taken for non-compliance in accordance to existing policies and practices

Selected Survey and Test Administration

• Reviewed by service center on a quarterly basis

• Review will ensure that survey and tests are being administered to all program participants

Enforcement Staff ImpactData Collection Process

- 5 -- 5 -

Requirements Processes and Details

- 6 -- 6 -

Selected Program Attendance Tracking – Suggested Process

Enter

Program

Member puts check next to name on attendance sheet

Program Instructor collects sheets and

enters data into KidTrax

Responsibility: Program Instructor*

Frequency: Up to 2 days after program session

Additional Data Entry Time (Staff): ~5-15 min per program session

Exit Program

Program Execution

*This responsibility will be included in Program Instructor goals / performance reviews

Program Instructor provides pre-populated

program attendance sheet to members

- 7 -- 7 -

Selected Program Attendance Tracking – Additional Details

Topic Details

Attendance Sheets Attendance sheets (part of the program planning kit) should be used for each program session and include every member’s name who has attended the program and rows for write-in members

– These attendance sheets should be used for multiple sessions and stored by the Program Director Program Instructor is responsible for ensuring all members are accounted for on the attendance sheets These attendance sheets should be stored for the entire school year

Scanners Handheld scanners will not be available for program attendance tracking at this time

Data Entry The Program Instructor is responsible for data entry within two days of the program session

– Directors / Managers are responsible for providing appropriate KidTrax training to Program Instructors

Batching of sign-in sheet information is permitted for attendance information

– See Batch Attendance instruction guide for further details

Best Practices Program Instructors should allot an extra 5-15 minutes per program session to input data Having a dedicated computer terminal makes inputting data quicker and easier Work w/ the Club Director / Program Director / Club Manager / Membership Clerks to ensure data is batched

and entered correctly

Expectations and Enforcement

Coordinators must enter data within two days of the program session Data from selected program dates will be reviewed by service center on an on-going basis This review will see if the majority of members were entered within the proper timeframe

Participating Programs

On-Going Programs SMART Programs Power Hour College Bound Keystone Club Torch Club

Quarterly Programs SMART Moves SMART Girls Street SMART Passport to Manhood Money Matters

Career Launch Youth for Unity Triple Play Skill Tech / Digital Suite

- 8 -- 8 -

Selected Survey and Test Administration – Guidelines

Topic Details

Survey / Test Administration

Pre / post surveys and tests for selected programs are available on KidTrax and must be completed by all participating members

Program Instructors may administer tests either electronically or by paper, but they are responsible for putting the information into KidTrax

– Surveys and tests must be linked back to an individual member for tracking purposes

Timeline Surveys and tests must be given 2 times per program (during first and last day of instruction) Allot 1-2 days to complete testing of members and data entry by staff

Best Practices Incentives may be given for member participation (e.g., pizza party, field trip, Power Points) See Survey Best Practices slide for additional suggestions

Expectations and Enforcement

Clubs are expected to survey all program attendees on the first and last day of instruction Service center will review the participation rates on a quarterly basis

Participating Programs

SMART Moves SMART Girls Street SMART

Other Notes Survey administration will be included in goals and performance reviews of the Program Instructor Results will be available in real-time for program improvement and staff discussions These requirements do not change any guidelines for other programs (e.g., 21st Century)

Responsibility: Program Instructor

Frequency: 2 times per program (during first and last day of instruction)

Additional Data Entry Time (Staff): ~5 mins per survey when survey or tests completed on paper

- 9 -- 9 -

Survey Best Practices

There are a number of best practices you can use for administering surveys and tests to members

Prepare all materials prior to the scheduled survey. If using hard copies, keep in mind kids often respond better to surveys printed on colored paper

Leave enough time to set up and arrange the room appropriately prior to the members’ arrival

Introduce yourself to the members as they arrive and give a brief description of your purpose for being there and administering the survey. Sample script:

– I am here to ask you to help make our clubs better. The survey your are about to take is being used to get your ideas about our club. It is not a test and there are no right or wrong answers — we want to know what your experiences and opinions are. Take your time and be sure to answer each question based on what you really think. If you don’t understand a question, please raise your hand or come find me and I can help you. The survey should take you only 8 to 10 minutes. (May differ depending on survey length).”

Request that members attempt to answer every question so their opinions and experiences can be included

Let members know you are here to help them if they have any questions

While members are completing the survey, quietly walk around the room (do not hover), answer questions and make sure things are running smoothly

When youth finish the survey, thank them for their time and provide incentives, if appropriate

Source: The Colorado Trust, ” Toolkit for Evaluating Positive Youth Development”