ies.ed.govies.ed.gov
Connecting Research,Connecting Research,Policy and PracticePolicy and Practice
Institute of Education SciencesGrant Writing Workshop
Elizabeth R. Albro, Ph.D.Acting Commissioner
National Center for Education Research
ies.ed.govies.ed.gov
Agenda
• Introduction to IES• Overview of IES Grant Programs• Education Research Grants (84.305A) and
Special Education Research Grants (84.324A)– Grant Topics– Grant Research Goals
ies.ed.govies.ed.gov
Agenda (cont.)
• Four Sections of the Research Narrative– Significance– Research Plan– Personnel– Resources
• Other IES Grant Programs• Application Submission and Review
ies.ed.govies.ed.gov
Organizational Structure
Office of the Director
National Board for Education
Sciences
National Center for Education Evaluation
National Center for Education Statistics
National Center for Education Research
National Center for
Special Education Research
Standards & Review Office
5
ies.ed.govies.ed.gov
Missions of the Research Centers
• NCER– supports rigorous research that addresses the
nation’s most pressing education needs, from early childhood to adult education
• NCSER– sponsors a rigorous and comprehensive program
of special education research designed to expand the knowledge and understanding of infants, toddlers and students with or at risk for disabilities from birth through high school
6
ies.ed.govies.ed.gov
Final Outcomes of Interest: Student Outcomes
Birth through Preschool• School readiness• Developmental outcomes for infants and toddlers with disabilities
Kindergarten through Grade 12• Academic outcomes in reading, writing, math, and science• Behaviors, interactions, and social skills that support learning in school
and successful transitions to post-school opportunities• High school graduation• Functional outcomes that improve educational results, transitions to
employment, independent living, and postsecondary education for students with disabilities
7
ies.ed.govies.ed.gov
Final Outcomes of Interest (cont.)
Postsecondary• Access, persistence, completion• Achievement in gateway math and science courses• Achievement in introductory composition courses
Adult Education• Reading, writing, and math for basic and secondary education
and English language learners
ies.ed.govies.ed.gov
Agenda
• Introduction to IES• Overview of IES Grant Programs• Education Research Grants (84.305A) and
Special Education Research Grants (84.324A)
• Four Sections of the Research Narrative• Other IES Grant Programs• Application Submission and Review
ies.ed.govies.ed.gov
Research & Research Training Grant Programs
• Education Research Grant Programs (84.305A)• Special Education Research Grant Programs (84.324A)
• Statistical & Research Methodology in Education (84.305D)• Evaluation of State & Local Education Programs & Policies (84.305E)• Accelerating the Academic Achievement of Students with Learning
Disabilities Research Initiative (A3) (84.324D) • Researcher-Practitioner Partnerships in Education Research (84.305H)
• Research Training Grant Programs in the Education Sciences(84.305B)
• Research Training Program in Special Education (84.324B)
ies.ed.govies.ed.gov
FY 2013 NCER (305) & NCSER (324)Grant Programs
Application Deadline
Letter of Intent Due
Date
Application PackagePosted
Start Dates
305A: Ed Research324A: Sp Ed Research305D: Stats/Methods
6/21/12 4/19/12 4/19/12 3/1/13 to
9/1/13
305A: Ed Research324A: Sp Ed Research305B: Research Training324B: Sp Ed Research Training324D: A3 Initiative305E: State/Local305H: Researcher-Practitioner Partnerships
9/20/12 7/19/12 7/19/12 7/1/13 to
9/1/13
Important Dates and Deadlines
11
ies.ed.govies.ed.gov
Information for Applying
http://ies.ed.gov/funding
Requests for Applications
Letter of Intent
IES Grants.gov Application Submission Guide
Application Package
ies.ed.govies.ed.gov
Requests for Applications (RFA)
• A separate RFA for each grant program
• Describes requirements for application
• RFAs are available on http://ies.ed.gov/funding
• To be informed about release of future RFAs, sign up for the IES Newsflash: http://ies.ed.gov/newsflash
ies.ed.govies.ed.gov
Letter of Intent (LOI)
• A short description of your intended application– PI, institution, collaborators– Budget (rough estimate)– Up to 1 page abstract describing work
• Not used in review process– Superseded by your application
• Submitted on http://iesreview.ed.gov
ies.ed.govies.ed.gov
IES Grants.gov Application Submission Guide
• Instructions for completing and submitting application package
• Available on http://ies.ed.gov/funding about April 19, 2012
ies.ed.govies.ed.gov
Application Packages for FY 2013
• Available at www.grants.gov– Help: [email protected] or 1-800-518-4726
• For June 21, 2012 deadline, packages available starting April 19, 2012
• For September 20, 2012 deadline, packages available starting July 19, 2012
• Packages specific for grant program and deadline
ies.ed.govies.ed.gov
Eligibility to Apply
• In general– Applicants have ability and capacity to conduct
scientifically valid research– Include, but are not limited to, nonprofit and for-profit
organizations and public and private agencies and institutions, such as colleges, universities, and school districts
• But eligibility is different for Training Programs, Evaluations of State & Local, and Researcher-Practitioner Partnerships
ies.ed.govies.ed.gov
How to Identify Appropriate Grant Programs
• Read the Request for Applications• Review announced topics and methodological
requirements• Look at abstracts of projects funded under a research
topic or program– http://ies.ed.gov/ncer/projects – http://ies.ed.gov/ncser/projects
• Contact appropriate Program Officer
ies.ed.govies.ed.gov
Agenda
• Introduction to IES• Overview of IES Grant Programs• Education Research Grants (84.305A) and
Special Education Research Grants (84.324A)
• Four Sections of the Research Narrative• Other IES Grant Programs• Application Submission and Review
ies.ed.govies.ed.gov
Grant Topics
• All applications to 84.305A and 84.324A must be directed to specific topic– Note: SF 424 Form, Item 4b (Agency Identifier Number)– Note: Top of Abstract and Research Narrative
ies.ed.govies.ed.gov
Education Research Topics (84.305A)
• Cognition and Student Learning• Early Learning Programs and Policies• Education Technology• Effective Teachers & Effective Teaching• English Learners• Improving Education Systems: Policies, Organization,
Management, and Leadership• Mathematics and Science Education• Postsecondary and Adult Education• Reading and Writing• Social and Behavioral Context for Academic Learning
ies.ed.govies.ed.gov
Exercise: Identify the Appropriate Education Research Topic
• Purpose: This study will examine the association between aspects of preschool quality and child health, behavioral and cognitive outcomes in community-based and school-based early care and education programs.
• Purpose: The purpose of this research is to test several possible ways to influence participation in college savings plans and subsequent savings behavior.
• Purpose: This study will provide a detailed examination of factors that predict gender differences in elementary school mathematics performance.
• Purpose: This project is designed around findings from a local needs assessment of teachers, which found: a) a need for more support for laboratory work; b) a need for greater access to subject matter experts; and c) a strong desire to plan together.
ies.ed.govies.ed.gov
Special Education Research Topics (84.324A)
• Autism Spectrum Disorders• Cognition and Student Learning in Special Education• Early Intervention and Early Learning in Special Education• Families of Children with Disabilities• Mathematics and Science Education• Professional Development for Teachers and Related Service Providers• Reading, Writing, and Language Development• Social and Behavioral Outcomes to Support Learning• Special Education Policy, Finance, and Systems• Technology for Special Education• Transition Outcomes for Secondary Students with Disabilities
ies.ed.govies.ed.gov
Exercise: Identify the Appropriate Special Education Research Topic
• Purpose: This research group will develop and preliminarily evaluate SELF: Social-Emotional Learning Foundations to promote emotional and behavioral self-regulation for children in kindergarten and first grade who are at risk for emotional and behavioral disorders.
• Purpose: This research will provide guidance for speech-language pathologists by examining how dosage, techniques, and context are associated with language outcomes.
• Purpose: There have been challenges with making computer-based test items accessible to students who are Braille readers. The purpose of this project is to add enhancements in testing accommodations for students who are blind or have low vision to an existing platform that delivers a Grade 8 reading assessment. This project will provide a fully functional computer-based test delivery platform designed to increase accessibility and meet the needs of these students.
ies.ed.govies.ed.gov
Topic Specific Issues for Education Research Program
• All require student outcomes (Effective Teachers, Systems)• Grade range varies by topic
– Most topics are for K-12 students only – Early Learning: For pre-K (ages 3-5) and their teachers
• Exception if project is to follow pre-K students into later grades applicant can choose most appropriate topic
– Education Technology: pre-K to adult except science (grade 12)– Cognition: pre-K to adult (voc ed, adult ed, remedial post-sec)– Postsecondary and Adult: for older students
• Postsecondary includes high school programs to get students into postsecondary
• Postsecondary limited to sub-baccalaureate and baccalaureate • Adult: adult basic education, adult secondary education, and English
Learner
ies.ed.govies.ed.gov
– Improving Education Systems: Policies, Organization, Management, and Leadership
• Anything designed to improve the overall functioning of a school, district, state, or national education system
– Programs– Finance– Leadership– Organization and Management
• Combined into 1 topic because interventions may include all of these approaches
Topic Specific Issues for Education Research Program II
ies.ed.govies.ed.gov
Topic Specific Issues for Education Research Program III
• Topics can Overlap– Effective Teachers & Effective Teaching
• Read/Write & Math/Science: determine if focus is on professional development or on curriculum/instructional approach
• Cognition: applying cognitive science to teacher practice• Early Learning topic is for pre-K teachers• English Learners: if for EL teachers can be either• Improving Ed Systems: teacher certification, recruitment, and retention
can go to either topic– Ed Technology with all programs
• Is focus/team on tech development or on substance?– English Learners with Read/Write & Math/Science
• Is EL the primary focus or a secondary focus?– Improving Ed Systems with all programs
• Except Early Learning Programs and Policies andPostsecondary and Adult Education
ies.ed.govies.ed.gov
Choosing among Overlapping Topics
• What literature are you citing?• To which topic is your area of expertise best
aligned?• If your focus is on a specific population of
students/teachers, go to that program/topic:– Is your focus on a specific type of student/teacher (e.g.,
students with autism or English language learners), or are you studying them as subgroup of your sample?
ies.ed.govies.ed.gov
Topic Specific Issues for Education Research Program IV
Pre-service programs•Only exploratory research can be done on teacher pre-service programs
– No development of pre-service programs, evaluation of them, or measures-development for them
•Can develop or evaluate pre-service components with in-service teachers•Support for leadership pre-service programs, if programs last 24 months or less
ies.ed.govies.ed.gov
Issues Specific to Special Education Program Topics
• Children with disabilities or at risk for developing disabilities– At risk based on individual assessment, not population
characteristics (e.g., low SES)– Be specific about which disabilities you are addressing
• Specify the inclusion/screening criteria• Applicants to following topics must address students
with disability only (not students at risk for disability)– Transition Outcomes for Secondary Students with
Disabilities– Autism Spectrum Disorders – Families of Children with Disabilities
ies.ed.govies.ed.gov
Issues Specific to Special Education Program Topics II
• Grade Coverage– Early Intervention: Birth to Age 5– Cognition: Birth to Grade 12– Technology: Birth to Grade 12– Autism: Pre-K to Grade 12– Transition: Secondary students (i.e., middle/high school) into
Postsecondary settings– Others: K to grade 12
• Overlaps– Autism and Other Topics
• Comprehensive interventions with multiple outcomes to Autism• One outcome goes to the relevant topic (e.g., Math/Science)
– Early Intervention and Other topics, if follow pre-K students to later grades
ies.ed.govies.ed.gov
Decide Which Topic Your Research Idea Would Fall Under
• Think about your research question(s)• Decide which topic it best fits under• If not sure
– Check RFA– Discuss with Program Officer
ies.ed.govies.ed.gov
Grant Research Goals
• All applications to 84.305A and 84.324A must be directed to specific research goal (1 of 5)– Note: SF 424 Form, Item 4b– Note: Top of Abstract & Research Narrative
• Goal describes type of research to be done
• So, every application is directed to specific topic/goal combination
ies.ed.govies.ed.gov
The 5 Research Goals
• Exploration• Development & Innovation• Efficacy & Replication • Effectiveness• Measurement
ies.ed.govies.ed.gov
Exploration Goal
• Explore associations between education outcomes and malleable factors
• Identify factors and conditions that may mediate or moderate the relations between malleable factors and student outcomes
ies.ed.govies.ed.gov
Malleable Factor
A factor that can be changed by the education system be it a student, teacher, or school characteristic, or an education program or policy
– Underlying processes that enhance or inhibit learning– Aspects of a school, district, or community associated with
beneficial education outcomes– Education interventions associated with beneficial
education outcomes
ies.ed.govies.ed.gov
Exploration Goal (cont.)
• Possible methodological approaches– Analyze secondary data– Collect primary data– Complete a meta-analysis
• Funding Limits– If secondary data analysis or meta-analysis only
• Maximum 2 years and $700,000
– If including primary data• Maximum 4 years and $1,600,000
ies.ed.govies.ed.gov
At the End of an Exploration Project
• A clear description of malleable factors found, and any accompanying mediators and/or moderators
• Evidence regarding associations of those factors with student outcomes
• A well-specified conceptual framework and/or theoretical framework linking identified factors and student outcomes
• Information about whether what you learned in this project could lead to:
– The development or refinement of an intervention– The rigorous evaluation of an intervention– The development of a conceptual framework to be used in the
development or revision of an assessment
ies.ed.govies.ed.gov
Would these Research Questions fit under Exploration?
• Do middle school girls score higher on English achievement tests than boys?
• Is hands-on science teaching associated with better grades for boys?
• Is increasing foster care payments linked to better academic outcomes of foster children?
• Does the Bluebird Reading Curriculum cause higher student achievement on reading tests?
• Do students with certain types of disabilities have shorter attention spans?
ies.ed.govies.ed.gov
Development & Innovation Goal
• Develop an innovative intervention (e.g., curriculum, instructional approach, program, or policy)
• OR improve existing education interventions
• AND collect data on its feasibility and usability in actual education settings
• AND collect pilot data on student outcomes
Development process must be
iterative!
ies.ed.govies.ed.gov
Feasibility of Implementation
• Implement intervention in authentic education delivery setting
• Small sample of users• Should be carried out in type of setting and with
types of users for which intervention is intended
ies.ed.govies.ed.gov
Development & Innovation (cont.)
• Collect pilot data on promise of intervention to achieve intended outcomes– Does not need to be causal study– Stronger with comparison group– Can be no more than 30% of grant budget– Obtain evidence to support grant for evaluation
• Maximum award: 4 years and $1,500,000
ies.ed.govies.ed.gov
At the End of a Development Project
• A fully developed version of intervention, along with– A well-specified theory of change and– Evidence that intended end users understand and can use
intervention• Data that demonstrate feasibility of implementation• Pilot data regarding promise, along with
– Fidelity measure(s) and– Evidence regarding fidelity of implementation.
43
ies.ed.govies.ed.gov
Would These Study Designs Fit Under Development & Innovation?
• Develop 9th grade biotechnology course over summer, implement from September to December, and measure student gains in knowledge.
• Give half the students iPads, monitor how they’re used, and compare test scores at end of year.
• New writing program – Develop with 10 teachers over 1 year – try components out in
class and revise accordingly– Feasibility test with the 10 teachers in Year 2– Compare writing scores of students of the 10 teachers to scores of students from 10 other teachers in Year 3
ies.ed.govies.ed.gov
Efficacy & Replication Goal
• Evaluate whether or not a fully developed intervention is efficacious under limited or ideal conditions
OR
• Gather follow-up data examining the longer term effects of an intervention with demonstrated efficacy
OR
• Replicate an efficacious intervention varying the original conditions
ies.ed.govies.ed.gov
Efficacy & Replication Goal II
• Ask what might be needed to implement intervention under routine practice
• Reduce appearance of conflict of interest for developer-evaluators
• Do not require confirmatory mediator analyses but recommend exploratory ones
• NCSER also accepts efficacy studies that propose single case experimental designs
ies.ed.govies.ed.gov
Efficacy & Replication Goal III
• Causal test of whether or not a fully developed intervention has a beneficial impact on student outcomes relative to a counterfactual in an authentic educational setting– Interventions already in wide use– Interventions not in wide use– Follow-up study
ies.ed.govies.ed.gov
Efficacy & Replication Goal IV
• Detailed description of intervention– Theory of change– Empirical evidence– Practical importance
• Designs that support causal inference are required– Random assignment to intervention and comparison
conditions preferred– Strong quasi-experiment designs can be proposed when
experiment not feasible– Single-subject methods/single-case designs permitted for
Special Ed– Check What Works Clearinghouse evidence standards (RCTs, RDD, Single Case, Attrition)
ies.ed.govies.ed.gov
Efficacy & Replication Goal V
• Address power of design to identify impacts• Address fidelity of implementation of treatment and
comparison groups• Address important moderators• Include details about measures & analysis plan• Avoid apparent conflicts of interest for evaluation
team
• Maximum award: 4 years and $3,500,000
ies.ed.govies.ed.gov
Efficacy & Replication Follow-up Study
• Follow students who received intervention into later grades, or
• Follow teachers (principals or schools) who received intervention after the project ends to see if sustained effect on practice and on student outcomes
• Maximum award: 3 years and $1,200,000
ies.ed.govies.ed.gov
At the End of an Efficacy Study
• Evidence of impact of clearly specified intervention on relevant student outcomes relative to comparison condition using research design that meets WWC standards
• Conclusions on and revisions to theory of change that guides intervention
51
ies.ed.govies.ed.gov
At the End of an Efficacy Study II
• If beneficial impact is found– Identify organizational supports, tools, and procedures
needed for future implementation
• If beneficial impact is not found– Determine whether and what type of further research
would be needed to revise the intervention
52
ies.ed.govies.ed.gov
Would these Study Designs fit under Efficacy & Replication?
• Randomly assign use of iPads to treatment and control classrooms
• Intervention will provide 3 weeks of teacher training, ongoing coaching, plus classroom materials
• Match 30 schools who adopted an anti-bullying program to 30 schools who did not based on % minorities and FSL and average test scores
• 4 districts agree to take part in a study that will randomly assign a math curriculum to 2 of them
ies.ed.govies.ed.gov
Effectiveness Goal
• Evaluate whether a fully developed intervention that has evidence of efficacy is effective when implemented under typical conditions through an independent evaluation
• Prior to submitting an effectiveness proposal, at least two efficacy studies of the intervention with beneficial and practical impacts on student outcomes must have been completed
54
ies.ed.govies.ed.gov
Effectiveness Goal
• IES expects researchers to– Implement intervention under routine practice– Include Evaluators independent of development/distribution– Describe strong efficacy evidence for intervention
• Does not expect wide generalizability from a single study– Expects multiple Effectiveness projects to this end– Sample size is not a key distinction from Efficacy
• Does not require confirmatory mediator analyses but encourages exploratory ones
• Cost of implementation is limited to 25% of budget
55
ies.ed.govies.ed.gov
Effectiveness Awards
• Limit of 25% of budget for implementation of the intervention
• Other requirements similar to Efficacy & Replication• Replications of Effectiveness studies are allowed with
different populations (e.g., students, schools)• Maximum award: 5 years and $5,000,000
ies.ed.govies.ed.gov
Effectiveness Follow-up Study
• Follow students who received intervention into later grades
• Maximum award: 3 years and $1,500,000
ies.ed.govies.ed.gov
At the End of an Effectiveness Study
• Evidence of impact of clearly specified intervention on relevant student outcomes implemented under routine conditions relative to comparison condition using research design that meets WWC standards
• Conclusions on and revisions to theory of change that guides intervention
58
ies.ed.govies.ed.gov
At the End of an Effectiveness Study II
• If beneficial impact is found– Identify organizational supports, tools, and procedures
needed for future implementation
• If beneficial impact is not found– Determine whether and what type of further research
would be needed to revise the intervention
59
ies.ed.govies.ed.gov
Would these Study Designs fit under Effectiveness?
• A researcher wants to test new in-service math teacher training program developed under a Development and Innovation grant in 60 randomly assigned classrooms.
• A district wants to compare two Algebra 1 curricula, and the companies agree to provide them at cost along with teacher coaching.
• A charter management company with evidence from 2 small efficacy studies receives funds from a millionaire to take over 40 schools. 80 schools apply, and the company will randomly select half if it receives IES funds to do an evaluation.
ies.ed.govies.ed.gov
Measurement Goal
• Development of new assessments or refinement of existing assessments, and validation of these assessments
OR
• Validation of existing assessments for specific purposes, contexts, and populations
61
ies.ed.govies.ed.gov
Measurement Goal
• Not for evaluating an assessment used as an intervention
• The measure is the primary product – Not intended to support the creation of a measure as part
of a larger study
• Maximum award: 4 years and $1,600,000
ies.ed.govies.ed.gov
Would these Study Designs fit under Measurement?
• Develop formative chemistry assessment to help students learn how to balance formulas
• Develop measure of teacher instruction in fractions and validate it against teacher logs and principal observations
• Develop measure of student attention and validate it against student grades as part of project to evaluate intervention to increase student time on task
ies.ed.govies.ed.gov
At the End of a Measurement Goal
• To develop/refine and validate an assessment– Description of assessment & its intended use– Description of processes used to develop/refine, including
field testing procedures & processes for item revision– Well-specified conceptual framework– Description of validation activities– Evidence on reliability & validity for specified purposes,
populations, and contexts
64
ies.ed.govies.ed.gov
At the End of a Measurement Goal
• To validate an existing assessment– Well-specified conceptual framework that provides
theoretical basis for assessment and its validation activities– Description of validation activities– Evidence on reliability & validity for specified purposes,
populations, and contexts
65
ies.ed.govies.ed.gov
The Goals Build on One Another
• Exploration may lead to– Development or modification of intervention– Efficacy evaluation of intervention
• Development & Innovation may lead to an Efficacy evaluation if intervention is feasible and pilot data is supportive
• Efficacy & Replication may lead to an Effectiveness evaluation if impact found
• Measurement can also feed into other goals
ies.ed.govies.ed.gov
Decide Which Goal Your Research Idea Would Fall Under
• Think about your research question(s)• Decide which goal it best fits under• If you are not sure:
– Check RFA– Discuss with Program Officer– If your idea straddles several goals, consider breaking it
into smaller pieces– Choose goal with best fit, not the one that offers the
most funding
ies.ed.govies.ed.gov
Dissemination Expected for all Goals
• Publications in peer-reviewed journals• Quick release of findings
– Working papers, presentations and posters, seminars• Products others can use
– Software, manuals, instruments, monographs• Teaching others to use findings/products
– Short courses, long courses, on-line tutorials• Long-term collaborations with practitioners• Submit final, peer-reviewed manuscripts to the
Educational Resources Information Center (ERIC)
ies.ed.govies.ed.gov
Agenda
• Introduction to IES• Overview of IES Grant Programs• Education Research Grants (84.305A) and
Special Education Research Grants (84.324A)
• Four Sections of the Research Narrative• Other IES Grant Programs• Application Submission and Review
ies.ed.govies.ed.gov
Connecting Research,Connecting Research,Policy and PracticePolicy and Practice
The Application’s Research NarrativeKey part of your application
4 Sections:Significance
Research PlanPersonnelResources
Each section scored and an overall score givenRequirements vary by program & goal
25 pages, single spaced
ies.ed.govies.ed.gov
Significance
• Describes the overall project– Your research question to be answered; intervention to
be developed or evaluated, or measure to be developed and/or validated
• Provides a compelling rationale for the project– Theoretical justification
• Logic Models, Change Models
– Empirical justification– Practical justification
ies.ed.govies.ed.gov
Significance
• Do not assume reviewers know significance of your work
• Do not quote back RFA on general importance of topic– e.g., RFA paragraph on lack of reading proficiency of 8th-
and 12th-graders based on NAEP data
• Do quote back RFA, if a specific topic is highlighted and your work will address that topic
ies.ed.govies.ed.gov
Significance: Exploration Goal
• Describe malleable factors, moderators, and mediators to be examined
• Justify their importance– Theoretical rationale– Empirical rationale– Practical importance
• How work will lead to useful next step– Development or modification of interventions to address the
identified malleable factors or underlying process to improve student outcomes
– Identification of interventions for more rigorous evaluation
• Overall importance
ies.ed.govies.ed.gov
Significance: Development Goal
• Context for proposed intervention– Why needed: what problem exits– What exists now (may be many alternatives already)
• Detailed description of intervention to be developed– Clearly identify components already developed, partially developed,
and to be developed (no jargon)– Don’t overextend (# grades, full vs. part year)
• Theory of change (theoretical support)• Empirical support• Practical importance:
– Meaningful impact, feasibility, affordability• Answer the question: Why will this intervention produce
better student outcomes than current practice?• Overall Importance
ies.ed.govies.ed.gov
Significance: Efficacy & Replication
• Detailed description of intervention– Show fully developed, implementation process, and ready to be
evaluated• Justification for evaluating the intervention
– Importance of practical problem it is to address– If in wide use, show it has not been rigorously evaluated– If not in wide use, show evidence of feasibility and promise to
address the practical problem• Theory of change: Why will this lead to expected
outcomes?– Theoretically and empirical rationale– Direct impact on student outcomes or through mediators
• Justify that it could lead to better outcomes than current practice
• Overall importance
ies.ed.govies.ed.gov
Significance: Effectiveness
• Detailed description of intervention• Justification for evaluating the intervention
– Evidence of meaningful impacts (Efficacy study)• Theory of change• Justify that it could lead to better outcomes than
current practice• Implementation under normal conditions• Independent evaluation• Evidence that implementation can reach high
enough fidelity to have meaningful impacts• Overall importance
ies.ed.govies.ed.gov
Significance: Measurement
• Description of assessment and how it will be used• Theoretical basis for constructs to be measured• Empirical evidence for constructs• Practical need for the assessment• Feasibility of use• Overall importance
ies.ed.govies.ed.gov
Significance: 2 Key Problem Areas
• Unclear description of malleable factor or intervention– May have many components and these may be applied at
different times• Graphic may help
– Unclear how to be implemented to ensure fidelity– Unclear why strong enough to expect an impact– Overly focused on actions not content
• Example: Provide 10 PD sessions, 4 hours apiece, but no detail on what occurs in sessions
ies.ed.govies.ed.gov
• Lack of a Theory of Change– Why malleable factor should be related to student outcome– Why intervention should improve outcomes versus current
practice– Why an assessment/instrument should measure a specific
construct– A well laid-out theory of change makes clear what is
expected to happen and in what order– Easy for reviewers to understand research plan – why
measure certain outcomes – Graphic can be helpful (e.g., a logic model)
Significance: 2 Key Problem Areas
ies.ed.govies.ed.gov
Four Sections of the Research Narrative
• Significance• Research Plan• Personnel• Resources
ies.ed.govies.ed.gov
Research Plan
• Describe the work you intend to do– How you will answer your research question, develop
your intervention, evaluate the intervention, or develop and/or validate your assessment
• Make certain Research Plan is aligned to Significance section– All research questions should have justification in
Significance
• Step-by-step process– Timeline to show when everything will be done
ies.ed.govies.ed.gov
Research Plans Differ by Research Goal
However, all should describe•Setting•Population and sample
– Sampling plan: inclusion and exclusion criteria– Sample size (power issue) and handling attrition– External validity
•Measures– Outcomes: proximal and distal; answer research questions– Other measures: fidelity, feasibility, operating as intended, feedback– Quantitative & qualitative– Reliability & validity– Relevance: sensitivity vs. broad interest– Multiple comparisons issue
ies.ed.govies.ed.gov
All Research Plans Should Include (cont.)
• Research Design (more detail on following slides)
• Analysis– Describe how it answers research questions– Show your model: show different types of models used– Address clustering– Describe how missing data will be handled– Check for equivalency at start of study and attrition bias throughout– Describe sensitivity tests of assumptions– Describe analysis of qualitative data and links to quantitative
analysis
ies.ed.govies.ed.gov
Research Design
• Start off with research questions
• Research design should answer research questions– Do not have design section written independently by
methodologist– If sections are written by different people, have everyone
read through whole application
• Issues common to designs across goals– Attrition and missing data– Obtaining access to and permission to collect/use data
ies.ed.govies.ed.gov
Research Design Varies by Goal
• Exploration– Primary data
• Sampling strategy• Data collection and coding processes
– Secondary data• Descriptive analysis• Statistical correlational analysis• Analyses attempting to address selection issues• Mediation analysis
• Development & Innovation– Focus should be on iterative development process– Feasibility study: use in authentic education setting– Pilot study: comparison to a similar group
ies.ed.govies.ed.gov
Research Design Varies by Goal
• Efficacy & Replication– Randomized Controlled Trial (RCT) preferred
• Unit of randomization and justification• Procedures for assignment
– Strong quasi-experiment - justify why RCT not possible• How it reduces or models selection bias• Discuss threats to internal validity – conclusions to be drawn
– Describe control/comparison group– Power analysis/MDES – show calculation and assumptions– Fidelity of implementation study in both T and C– Mediator and moderator analyses– Contamination issues: schools vs. classrooms
ies.ed.govies.ed.gov
Research Design Varies by Goal
• Effectiveness– Same as Efficacy & Replication except requires
implementation under routine conditions, independent evaluator, and a cost study
• Measurement– The plan to develop or refine the assessment
• Evidence of constructs• Interpretation of assessment results• Item development and selection• Procedures for administering and scoring
– Reliability and validity studies
ies.ed.govies.ed.gov
Four Sections of the Research Narrative
• Significance• Research Plan• Personnel• Resources
ies.ed.govies.ed.gov
Personnel Section
• Describe key personnel– Link each person and their expertise to their role in
project - show that every aspect of project has person with expertise to do it
• Methodologists: show expertise in particular method to be used• Substantive person for all issues addressed• Do not propose to hire a key person with X expertise• Project management skills
– Give time contribution for each - show that every aspect has enough time from expert
• Orient CVs so specific to project– 4 pages plus 1 page for other sources of support
ies.ed.govies.ed.gov
Personnel Requirements
• Publication record and projected publications from this grant are considered
• Developers should discuss past success getting developed interventions evaluated
• If previous IES grant, discuss results• Evaluations require attention to objectivity should a
developer or persons with financial interest be involved– Efficacy projects: address how objectivity maintained– Effectiveness: Independent evaluation: developer can
provide routine implementation support
ies.ed.govies.ed.gov
Personnel Strategies for PI
• Senior Researcher– Show adequate time to be PI– Make credentials clear: not all reviewers may know
• Junior Researcher as PI– Show adequate expertise not only to do work but to
manage project– Reviewers may be more comfortable if you have senior
person(s) on project to turn to for advice
ies.ed.govies.ed.gov
Resources
• Show institutions involved have capacity to support the work– Do not use university boilerplate
• Show that all organizations involved understand and agree to their roles– What will each institution, including schools,
contribute to project– Show strong commitment of schools and districts– Have alternatives in case of attrition
ies.ed.govies.ed.gov
Resources (cont.)
• Appendix C should back this up – Detailed Letters of Support from research institutions,
States, districts, schools
• Data issues– Document permission to use and access to confidential
data (letters in Appendix C)– Show familiarity with data and show that it can be
used to do proposed work– If merging datasets, show that it can be done
ies.ed.govies.ed.gov
Appendices
• Appendix A (15 page limit)
– Figures, charts, and tables– Examples of measures– 3 pages to address past reviewer comments or to argue
that a proposal is a new submission• Appendix B (10 page limit)
– Examples of materials used in intervention or assessment
• Appendix C (no page limit)
– Letters of agreement (districts, schools, data providers, other partners, consultants)
ies.ed.govies.ed.gov
Budget and Budget Narrative
• Provide clear budget and budget narrative for overall project and each sub-award
• IES Grants.gov Application Submission Guide describes budget categories
• Check RFA for specific budget requirements for Research Goals and Grant Programs
• Ensure agreement among Research Narrative, Budget, and Budget Narrative
ies.ed.govies.ed.gov
Agenda
• Introduction to IES• Overview of IES Grant Programs• Education Research Grants (84.305A) and
Special Education Research Grants (84.324A)
• Four Sections of the Research Narrative• Other IES Grant Programs• Application Submission and Review
ies.ed.govies.ed.gov
Connecting Research,Connecting Research,Policy and PracticePolicy and Practice
Grant-writing for Additional FY 2013 Research Programs
ies.ed.govies.ed.gov
Other Research Grant Programs
• Do not use topic/goal structure, but are informed by that structure
• Stats/Methods and Evaluation of State & Local use a similar Research Narrative
• All except the Stats/Methods program are due no later than September 20, 2012
• Stats/Methods applications are due June 21, 2012
ies.ed.govies.ed.gov
Statistical & Research Methodology in Education (84.305D)
• Research projects intended to expand and improve methodological and statistical tools available for education researchers
• These tools will be used to improve design of research studies, analysis of research data, and interpretation of research findings
100
ies.ed.govies.ed.gov
Evaluation of State & Local Education Programs & Policies (84.305E)
• Support for rigorous evaluations of education programs or policies that are paid for and implemented by State or local education agencies
• Evaluations are to determine both overall impact of programs/policies and impact across a variety of conditions
101
ies.ed.govies.ed.gov
Accelerating the Academic Achievement of Students with Learning Disabilities Research Initiative (A3)(84.324D)
• To develop and evaluate interventions (e.g., instructional approaches, curricula, technology) to accelerate reading and math achievement of students with or at risk for reading and math disabilities in grades 3 through 8
• Will create tightly linked network of researchers across variety of disciplines who will work collaboratively to address the problem
102
ies.ed.govies.ed.gov
A3 Project Narrative
• Significance• Research Plan
– Development of a set of interventions– Evaluation of the efficacy of an evaluation– Supplementary exploratory research (optional)– Timeline
• Personnel• Management Plan• Resources• Project Narrative is limited to 35 single-spaced pages
103
ies.ed.govies.ed.gov
Researcher-Practitioner Partnerships in Education Research (84.305H)
• Intended to support partnering of researchers with State and local education agencies in development of joint research projects
104
ies.ed.govies.ed.gov
Partnership Project Narrative
• Significance of the Research-Practitioner Partnership– Strength of the partnership– Research aims of the partnership
• Research Plan– Developing the partnership– Developing the research plan– Developing the application for further Institute funding
• Personnel• Resources• Project Narrative limited to 25 single-spaced pages
105
ies.ed.govies.ed.gov
Research Training Programs in the Education Sciences (84.305B)
• Postdoctoral Research Training Program• Researcher and Policymaker Training
Program in the Education Sciences
106
ies.ed.govies.ed.gov
Postdoctoral Research Training Program
• For institutions to establish postdoctoral training programs to train researchers in skills necessary to conduct the type of research that the Institute funds
107
ies.ed.govies.ed.gov
Researcher and Policymaker Training Program in the Education Sciences
• Researcher Training– to provide opportunities for current education researchers
to maintain and upgrade their methodological skills
• Policymaker Training– to share evidence from rigorous education research with
education practitioners and policymakers working on a specific program or policy
108
ies.ed.govies.ed.gov
Training Program Narrative
• Significance of Training Program• Research Training Plan• Personnel• Resources
• Training Program Narratives are limited to 15 single-spaced pages.
109
ies.ed.govies.ed.gov
Research Training Program in Special Education (84.324B)
• Early Career Development and Mentoring – Provides new investigators support to further develop
methodological, content, and grant writing expertise needed to develop a strong line of research that includes federal funding
– Requires identification of mentor at time of application– Requires a research plan and career development plan
110
ies.ed.govies.ed.gov
Early Career Program Narrative
• Significance of the Research and Career Development Plans
• Research and Career Development Plans• Personnel• Resources
• Early Career Program Narratives are limited to 25 single-spaced pages.
111
ies.ed.govies.ed.gov
Award Parameters (84.305B-H & 84.324B-D)
Program Maximum Number of Years
Maximum Award(direct + indirect)
305B Postdoc Training 5 years $687,000
305B Researcher & Policymaker Training
3 years $1,000,000
324B Early Career 4 years $400,000
324D A3 5 years $10,000,000
305D Stats/Methods 3 years $900,000
305E State/Local305H Partnerships
5 years2 years
$5,000,000$400,000112
ies.ed.govies.ed.gov
Agenda
• Introduction to IES• Overview of IES Grant Programs• Education Research Grants (84.305A) and
Special Education Research Grants (84.324A)
• Four Sections of the Research Narrative• Other IES Grant Programs• Application Submission and Review
ies.ed.govies.ed.gov
Grant Submission
• Make sure your institution is registered on Grants.gov
• Complete your online forms and upload PDFs• Authorized representative completes the process• Submit by 4:30:00pm DC time on deadline –
earlier is safer• If problems uploading
– Contact Help Line 1-800-518-4726 and get a case number
114
ies.ed.govies.ed.gov
Verification of Submission
• You should receive three e-mails – Grants.gov will say they have received your
submission and assign you a number that starts with GRANT
– Grants.gov will say your application is validated or rejected due to errors. If the latter, resubmit until validated.
– Department of Education will assign you a grant number starting with R305 or R324
115
ies.ed.govies.ed.gov
Application Review (Standards & Review Office)
• Compliance screening for format requirements• Responsiveness screening to program/goal requirements• Assigned to review panel
– 2-3 reviewers (substantive and methodological)– If scored high enough, application is reviewed by full panel
• Many panelists will be generalists to your topic• There will be an expert in every procedure you use
– Overall score plus scores on Significance, Research Plan, Personnel, and Resources
– So far, all applications with overall score of Outstanding and Excellent have been funded
• Resubmissions encouraged– Talk to Program Officer and address reviewer comments
116
ies.ed.govies.ed.gov
Peer Review Process Information
http://ies.ed.gov/director/sro/peer_review/index.asp
ies.ed.govies.ed.gov
Notification
• All applicants will receive e-mail notification of the status of their application
• All applicants receive copies of reviewer comments
• If you are not granted an award the first time, plan on resubmitting and talk to your Program Officer
ies.ed.govies.ed.gov
Additional Webinars Available
http://ies.ed.gov/funding/webinars/index.asp
•Application Process•Grant Writing Workshops for each goal•Grant Writing Workshops for early career researchers and minority serving institutions•Overviews of research training programs•Overviews of NCSER funding opportunities