www.nationaldb.org program planning & improvement webinar: part iii -selection criteria - an...
Post on 11-Jan-2016
212 Views
Preview:
TRANSCRIPT
N atio n al Co n so rtiu mo n D eaf-Blin d n ess
TECHNIC
AL A
S S IST
ANCE
INF OR M ATION
www.nationaldb.org
Program Planning & Improvement
Webinar: Part III -Selection Criteria - An informational Service-
Presenters: Barbara Purvis & Ella Taylor
Moderator: Jon Harding
AGENDA FOR TODAY’S WEBINAR –
Disclaimer: The following information is taken directly from the published CFDA 84.326C competition with suggestions (only) on issues that new applicants may want to consider as they plan their projects and seek to provide more effective services to children and youth who are deaf-blind
Overview of the Selection Criteria & Format
Question/Answer period
GENERAL GRANT WRITING SUGGESTIONS
CRITICAL POINT – To assist the reviewers to score your proposal with the review forms that they are given, it is recommended that you:
Use the selection criteria headings as a Table ofContents
Use the same wording used in the RFP for the headings and sub-headings
PROJECT ABSTRACT
Provide a one-two page abstract of the proposed project that follows the description of that in the RFA. At a minimum, address:
1. Title of the program, the name of the absolute priority, and the CFDA number (CFDA 84.326C)
2. Purpose of project, disability addressed, age group, geography, severity
3. Proposed products (if any are to be developed)
4. Proposed outcomes and potential impact
5. Names and affiliations of key collaborators
POINTS TO CONSIDER - ABSTRACT
Abstract must include priority and named state project
The RFP does not count the abstract as a page within the Part III page limit
The abstract should state the significance of the project in one-three sentences
The abstract should clearly articulate the major goals (purpose) of the proposal
The abstract should address the intended impact of the proposed project
The abstract should provide the reader with a concise overview of the project
Selection Criteria to Evaluate Applications
Criteria are those for new grants required by the EDGAR selection criteria menu.
The maximum score for all of the criteria is 100 points.
The one-two page abstract should precede the application narrative
Part III – Application narrative should not exceed 70 pages
The narrative should include the following sections in the given order
PART III - PROJECT NARRATIVE
Selection Criteria and Format for CFDA 84.326C
(a) Significance (10 points)
(b) Quality of project services (25 points)
(c) Quality of project personnel (20 points)
(d) Adequacy of resources (10 points)
(e) Quality of the management plan (15 points)
(f) Quality of project evaluation (20 points)
(a) Significance (10)
The potential contribution to increase knowledge of early education or educational problems/issues and effective strategies
The likelihood of systems change or improvement
The building of capacity to provide, improve, or expand services
The utility of products
The importance of the results/potential impact
The Secretary considers…
(a) Significance (10)Convince the reader that you can:
Use your state’s 2006 and 2007 Census data as part of your needs assessment process
Verify the stated needs of the population, families, service providers, agencies (indicate your data sources)
Show a direct relationship between the needs assessment results and the needs being addressed
Consider summarizing the impact of your past project Show a direct relationship between the stated needs and the
proposed objectives Show how the objectives of the proposed project correspond to
the priorities Consider Dick Zeller’s findings from site review: These were the
common needs: Asses state needs Assess child needs Measure child outcomes Research based practices Evaluate project effectiveness
POINTS TO CONSIDER - SIGNIFICANCE
(i) Potential contribution of the proposed project to increase knowledge or understanding of educational problems, issues, or effective strategies.
Analyze your state’s census data – what needs are reflected here?
What critical needs did your Site Review recommend? What potential SPP/APR “determinants” will be targeted in
your state? What needs has the applicant identified from other state
data (Assistive technology, developmental disabilities, PTI data)?
Can the applicant show actual “child/youth/family” need data?
How do your proposed objectives and activities address the needs and contribute to improvement?
POINTS TO CONSIDER - SIGNIFICANCE
(ii) Likelihood that the Project will Result in System Change or Improvement; consider discussing the following:
Summarize data on effectiveness of system change in past 5 years in your state and your contribution to that improvement
Provide the overall proposed framework in a logic model Discuss how you propose to link your activities with
current systems of service (Part C; Part B Sec. 619; Part B; Voc Rehab; PTI Centers; University (IHE) Personnel Prep; Assistive Tech)
Discuss how your proposed activities are aligned with professional plans with other agencies in your state
Discuss how the proposed objectives are linked to current systems initiatives within your state
POINTS TO CONSIDER - SIGNIFICANCE
(iii) Extent that the Project is Likely to Build Local Capacity to Provide, Improve, or Expand Services That Address Needs
Are you proposing a model to build capacity Have needs changed based on current technology What capacity will you attempt to build How will you build capacity How will you know if you have built capacity
Do the objectives reflect how the proposed project will build local capacity to address the needs of the target population?
How does the applicant propose to collaborate with Parts B & C, SPDIG, Personnel Preparation programs, adult agencies in order to build capacity?
POINTS TO CONSIDER - SIGNIFICANCE(iv) Utility of the Products that will Result from the Project –
Products may include information, materials, process, or techniques
Will you show that products/services are Useful Relevant High quality Cost effective?
(v) Importance or magnitude of the Results/Outcomes How will other populations directly benefit? How might service providers and other families benefit? How will practices and services be improved as a result of the
proposed project? What is the impact?
(b) Quality of project services (25)
The quality and sufficiency of strategies for ensuring equal access and treatment for eligible participants related to:
Race Color National origin Gender Age, or Disability
The Secretary considers…
(b) Quality of project services (25)
Appropriateness to the needs of the recipients
Demonstration of support by those to be served
Use of knowledge from research and effective practices
Impact of the services on the intended recipients
Collaboration of appropriate partners
Use of efficient strategies, including those of technology and leveraging of resources
The Secretary also considers…
Quality of Project Services – (25)
NOTE: The Quality of Project Services should logically lead the reader from the stated needs and proposed objectives (outlined in the Significance section) to the activities necessary to carry out each objective. One or more “Logic Models” might be used to display the sequence and the “causal connections.”
POINTS TO CONSIDER – PROJECT SERVICES
1) Quality of the Services to be Provided
Does the proposal address each required priority? How well is the proposal designed? Is a strong
conceptual model included? Are the objectives and activities consistent with the
stated needs? How well do the project objectives and activities
correspond to the priorities?
POINTS TO CONSIDER – PROJECT SERVICES
2) Equal Access and Treatment for Eligible Project Participants
Describe the project’s provisions for providing equal access (GEPA)
Describe how your website will address accessibility issues Describe strategies that the project will use for the
recruitment of eligible project participants who are members of traditionally under represented groups (strategies should not refer only to children)?
Discuss how the project will address cultural and language issues in your state?
POINTS TO CONSIDER – PROJECT SERVICES
3) The Secretary Considers the Following Factors:
(i) The Extent to Which the Services to be Provided Are Appropriate To The Needs of the Intended Recipients
Is there a correspondence between targeted results & intensity of services?
Are the diverse and unique needs of the population addressed
Are the services to be provided consistent with IDEA’2004, GPRA, State Performance Plans & Annual Performance Report for Part C, Part B-619, & Part B indicators?
POINTS TO CONSIDER – PROJECT SERVICES
(ii) Extent to Which Entities Served by the Proposed Project Demonstrate Support
Are letters of support/intent from parents and consumers included?
Are letters from teachers, interveners, and service providers included?
Do letters from agencies address how the project can build capacity and facilitate improved outcomes?
Are letters of commitment from collaborative agencies included?
POINTS TO CONSIDER – PROJECT SERVICES
(iii) Extent to Which the Services Reflect Knowledge from Research and Effective Practices
Is research specific to technical assistance or professional development included?
Are the educational approaches planned based on sound theoretical models and research that indicates they will be successful for the population to be served?
Does the review of literature include research and effective practices that corresponds to the proposed activities?
Does the proposal address innovative strategies for TA? How does the project propose to provide on-site
“coaching” strategies?
POINTS TO CONSIDER – PROJECT SERVICES
(iv) Impact of Services on the Intended Recipients
Are project objectives clearly stated and measurable?
Does the proposal clearly state the results for which the applicant is responsible?
Does the proposal clearly indicate that proposed activities will result in significant impact.
Note: Since the RFA includes significant points for Evaluation, this sub-section might refer the reader to the Evaluation section for the discussion of specific objective measures of impact.
POINTS TO CONSIDER – PROJECT SERVICES
(v) Extent to Which the Services Involve Collaboration of Appropriate Partners for Maximizing the Effectiveness of Project Services
Are the specific agencies and collaborative partners clearly indicated and are their roles clearly stated?
Have these agencies committed in writing? Are the anticipated outcomes from any joint
collaborations clearly addressed? Are the goals and roles of the Advisory Council provided
(if one is proposed to address multiple perspectives for implementation and evaluation?)
POINTS TO CONSIDER – PROJECT SERVICES
(vi) Extent to Which the TA Services Involve the use of Efficient Strategies, including Technology, and Leverage Non-Project Resources
Are TA strategies based on research? How will distance learning be addressed? How is the project proposing to use technology? What specific technology needs to the children/youth
have? Are these current needs being addressed by the project? How will services impact educational practices? Does the project describe which effective strategies will
be used? Describe how your University & DOE/SEA support your
project?
(c) Quality of project personnel (20)
The quality of the personnel
Recruitment of individuals from under-represented groups
Qualifications, training and experience of key personnel
Qualifications, training and experience of consultants
Current vitae
The Secretary considers…
Quality of Project Personnel
NOTE: This section is worth a number of points for the
length. Staff on a project should have the experience and
skills: (a) to provide technical assistance to service
providers or families working with children; (b) to have the
knowledge and skills to address a range of disabilities
reflected in the population who are deaf-blind (c) to
demonstrate a working knowledge with the age-span of
children who are deaf-blind; (d) to deliver inservice
training and follow-up, and (e) to evaluate the
effectiveness of the project.
POINTS TO CONSIDER - PERSONNEL
1) Quality of Personnel Has the applicant included short paragraphs
summarizing the skills and experience of key staff? Does the staff’s experience reflect skills in managing
grants and in work specific to DB content, local capacity building and system change?
Has the applicant included a table (remember to double space this) or text of time each staff will commit to project? (See Person Loading Chart on page C-6)
Does the application include personnel summaries for key “donated” or “in-kind” staff?
POINTS TO CONSIDER - PERSONNEL
2) Extent to Which Applicant Encourages Applications Does the applicant describe active strategies for
encouraging applications from members of groups that have traditionally been under represented based on race, color, national origin, gender, age, or disability? Strategies must be included here, not just a sentence that you encourage applicants…how?
Does the applicant specify how a parent(s) may be included as staff?
What specific accommodations for personnel may need to be considered and budgeted?
POINTS TO CONSIDER - PERSONNEL
3) The Secretary also Considers the Following Factors:
(i) Qualifications of key personnel Describe major roles and responsibilities of
key personnel with FTE Does staff expertise match the major
objectives and work scope of the project? Are key staff on the project at sufficient FTE? How does staff experience/skill correspond to
objectives?
POINTS TO CONSIDER - PERSONNEL
(ii) Qualifications of the consultants or subcontractors
Do you have letters of commitment and vitae from consultants?
Has the applicant clearly outlined how consultants will be selected and utilized?
Are the relevant training and experience of the consultants discussed; or are their vita included?
(e) Adequacy of resources (10)
Support, facilities equipment and other resources of the applicant
Adequacy of the budget
Reasonableness of the costs in relations
The Secretary considers…
Adequacy of Resources
NOTE: Consider both human and non-human
resources. Consider how strong collaborations
can serve as a resource.
POINTS TO CONSIDER - RESOURCES
(i) Adequacy of Support Has the applicant discussed all resources? Does the agency have a history in carrying out projects? Has the applicant adequately described their facilities,
equipment, support services, and overall agency-resources?
How can the Advisory Counci be considered a resource? Can the applicant adequately carry out project activities
in terms of word processing, data storage and analysis, duplication, and travel?
What types of resources are available from the collaborating agencies?
POINTS TO CONSIDER - RESOURCES
(ii) Extent to Which the Budget is Adequate
How has the applicant addressed the budget in relation to the work scope?
Has the budget been addressed in relation to all activities?
Has the applicant convinced the readers how the allocated budget will be used to accomplish the project’s objectives?
Does the budget reflect costs for interpreters, access issues, etc.?
POINTS TO CONSIDER - RESOURCES
(iii) Extent to Which Costs are Reasonable
Has the applicant discussed how costs are reasonable in terms of the projected numbers of children, parents, service providers, administrators, paraprofessionals to be served?
Can project resources meet TA requests? Has the applicant indicated donated and in-kind dollars
and sources of these dollars? Clearly indicate how your SEA and agency (have and
will) provide additional resources to assist in the support of your proposed project.
(d) Quality of the management plan (15)
Adequacy of the plan to achieve objectives… On time Within the budget With clearly defined responsibilities In relation to specific timelines, and With milestones for accomplishing tasks
The Secretary considers…
Quality of the management plan
NOTE: Whereas a number of tables may be included
in the Management Section (and it is worth 15 points),
you may want to show a “snap shot of the specific
timelines” of one or two objectives and place the more
extensive timeline table in the appendix. However, do
give the reader a feel for what you are doing and what
it will look like so they are “tempted to look” in an
appendix. Put what is critical into the pages allowed.
Put what complements, reinforces and clarifies in the
appendix. However, do not over-do the appendix
section, more is not necessarily better.
POINTS TO CONSIDER – MANAGEMENT PLAN
(i) Adequacy of the management plan to achieve the objectives
How will the applicant use its resources and personnel to achieve the objectives, with clearly defined responsibilities?
Has the applicant developed an effective management plan with realistic timelines?
Are communication strategies included? Has the applicant included a personnel loading chart
by objective? (See page C-6) Has the applicant included timelines? Has the applicant provided measurable milestones for
accomplishing tasks? Is the project within the budget?
POINTS TO CONSIDER – MANAGEMENT PLAN
(ii) Adequacy of Procedures for Ensuring Feedback
How will staff assignments and assigned responsibilities be monitored?
What feedback systems are in place, such as the Goal Attainment Scale, GPRA indicators or other tools?
How are activities monitored? How are data used to make decisions?
POINTS TO CONSIDER – MANAGEMENT PLAN
(iii) Adequacy of Mechanisms for Insuring High Quality Products and Services
Does the project have strategies in place to determine the quality of the services? These may be needs assessments, surveys, focus groups, staff reviews, systematic reviews of the data for decision-making.
Does the proposal include a dissemination plan? How do larger staffs communicate and
supervise work activities?
POINTS TO CONSIDER – MANAGEMENT PLAN
(iv) Extent to Which Time Commitments of the Staff are Appropriate and Adequate
Does the applicant show the time commitments for staff across administrative and TA activities?
Does the applicant show the time commitments for all staff across other federal and non-federal projects (See C 5)?
Are the staff time commitments adequate? Do letters from agencies indicate their time
commitment?
POINTS TO CONSIDER – MANAGEMENT PLAN
(v) Insure a Diversity of Perspectives
How does the applicant include stakeholders, including parents and consumers, in all aspects of the project?
How does the project solicit cultural perspectives?
How are the needs of underserved families met? How does the project work with their PTIs to
address these issues?
(f) Quality of Evaluation (20)
The Secretary:
Considers the quality of the evaluation to be conducted in the proposed project
Considers the following factors:
POINTS TO CONSIDER – EVALUATION
(i) Extent to which the methods of evaluation are thorough, feasible, and appropriate to the goals, objectives and anticipated outcomes
Are both formative and summative evaluation measures included?
Are the proposed evaluations aligned with the level of TA intensity?
How are the intended outcomes being measured?
Are the proposed evaluation strategies reasonable in relation to project resources?
POINTS TO CONSIDER – EVALUATION
(ii) Extent to which the method of evaluation provide performance feedback and permit periodic assessment (such as the GAS, GPRA or other tools)
Have targeted outcomes been established across major activities? How are these incorporated into the annual Grant Performance Report?
Does data include GPRA indicators (high quality, relevant, useful)?
Are measurement systems in place? Are data consistently and regularly put into the
system? Are the data analyzed for progress each quarter?
POINTS TO CONSIDER – EVALUATION
(iii) The extent to which the methods of evaluation will provide timely guidance for quality assurance
Are the stated objectives and activities being implemented and evaluated as stated?
Are timelines being implemented as stated? Are data analyzed for the Grant Performance
Report? Are decisions being made based on data? How are challenges, barriers, issues being
documented?
POINTS TO CONSIDER – EVALUATION
(iv) Extent to which evaluation will provide guidance about effective strategies for replication or testing in other settings
How are TA strategies documented? How are baseline data collected? Are any model demonstration activities planned
in collaboration? Is there some change in performance (of
teachers, of interveners, of families) after the strategy was used?
How are “effective strategies” used to “scale up?” or to replicate in other LEAs or classrooms?
POINTS TO CONSIDER – EVALUATION
(v)The extent to which the methods provide for examining the effectiveness of project implementation
Are the timelines for activity initiation being met?
How is the Logic model used for formative evaluation?
Is fidelity of implementation of “coaching” strategies being measured or probed?
How are both the “efforts” and “effects” being examined for effectiveness?
POINTS TO CONSIDER – EVALUATION(vi) The extent to which the methods of evaluation
include the use of objective performance measures that are clearly related to the intended outcomes and the TA provided
Are objective measures (not subjective) being used to measure the performance of service providers and children/youth for critical TA activities?
Are probes included if projects lack resources for ongoing objective performance measures?
Are both effort and effect used as performance measures?
Are both qualitative and quantitative data being collected to the extent possible?
How is evaluation reflected in the logic models?
Protection of Human Subjects (IRB) What is research?
“If an activity follows a deliberate plan whose purpose is to develop or contribute to generalizable knowledge it is research.”
Definitions for DOE Supplemental Information for SF 424
Protection of Human Subjects (IRB)
“If an activity involves obtaining information about a living person by manipulation that person or that person’s environment, such as might occur when a new instructional technique is tested, or by communicating or interacting with the individual, as occurs with surveys and interviews, the definition of human subject is met.”
“If an activity involves obtaining private information about a living person in such a way that the information can be linked to that individual … the definition of human subject is met.”
Definitions for DOE Supplemental Information for SF 424
Exempt or Non-exempt?
Do you track individual student progress? IF yes, NON-EXEMPT
Answer the 7 questions under Non-exempt
IF no, do you track individual service provider or family progress?
IF yes, NON-EXEMPT Answer the 7 questions under Non-exempt
IF no to both, then you are Exempt. Provide a brief paragraph explaining the involvement of human subjects
NOTE: You do not need to get IRB approval before submitting, but you do need to address human subject protection in the Supplemental Information form
TIME FOR QUESTIONS
N atio n al Co n so rtiu mo n D eaf-Blin d n ess
TECHNIC
AL A
S S IST
ANCE
INF OR M ATION
www.nationaldb.org
THANK YOU!
Good Luck with Your Application!
top related