national intelligence university outcomes and assessment toolkit · 2019-02-14 · list and...
TRANSCRIPT
i
National Intelligence University Outcomes and Assessment Toolkit
Tools for development of effective curriculum, assessment and evaluation
November 2018
ii
iii
Introduction
“An institution of higher education is a community dedicated to students, to the pursuit and dissemination
of knowledge, to the study and clarification of values, and to the advancement of the society it serves.
The Middle States Commission on Higher Education (MSCHE), through accreditation, mandates that its
member institutions meet rigorous and comprehensive standards, which are addressed in the context of
the mission of each institution and within the culture of ethical practices and institutional integrity
expected of accredited institutions.1”
“Middle States accreditation is an expression of confidence in an institution’s mission and goals, its
performance, and its resources. An institution is accredited when the educational community has verified
that its goals are achieved through self-regulation and peer review. The extent to which each educational
institution accepts and fulfills the responsibilities inherent in the process of accreditation is a measure of
its commitment to striving for achieving excellence in endeavors.2”
The National Intelligence University (NIU) has evolved over the years to redefine its mission and
curriculum, along with changes to its name. Accredited institutions are expected to demonstrate
compliance and conduct their activities in a manner consistent with the standards and requirements
established by MSCHE. These efforts focus on the student learning experience, institutional assessment
and assessment of student learning, and continuous self-review and improvement.
This document provides university faculty, administration, and staff with the tools needed to understand
the standards and assessment processes at NIU that help ensure we meet accreditation standards. The
toolkit is updated regularly with:
Information needed by both staff and faculty to ensure the quality of education
New requirements from NIU's accrediting bodies and from the internal knowledge and
experience gained during the previous review cycle
Faculty tools, templates, aides, and other resources required (or strongly recommended) by
the Office of the Provost.
More specifically, this document is a toolkit to help faculty develop quality programs and student learning
outcomes, syllabi that enable student understanding of expectations, and effective tools for student
assessment.
Questions about the information in this document can be directed to the Institutional Effectiveness (IE)
Department at:
NIPR: [email protected]
JWICS: [email protected]
1 Middle States Commission on Higher Education (MSCHE), “Standards for Accreditation and Requirements of Affiliation”, Thirteenth Edition, pg. v. 2 Middle States Commission on Higher Education (MSCHE), “Standards for Accreditation and Requirements of Affiliation”, Thirteenth Edition, pg. 1.
iv
Additional information about IE and university assessments can be found on the NIU SharePoint site:
https://iconline.coe.ic.gov/sites/NIU_MGT/niu1provost/InstutionalEffectiveness/SitePages/Home.aspx
v
Table of Contents
Institutional Effectiveness Department ............................................................................................... 1
Policies ............................................................................................................................................. 2
NIU’s Institutional Assessment Plan ................................................................................................... 5
Assessment Strategies and Tools ....................................................................................................... 7
Faculty Curriculum Vitae Requirements .............................................................................................. 9
Academic Review Roles .................................................................................................................... 10
Development of Learning Outcomes ................................................................................................. 13
Bloom’s Taxonomy .......................................................................................................................... 15
Course Syllabi .................................................................................................................................. 18
Academic Program Review ............................................................................................................... 23
NIU’s Roadmap to Evidence ............................................................................................................. 24
Appendix A: Assessment Acronyms & Glossary ................................................................................. 25
Appendix B: Direct vs. Indirect Measures of Assessment ................................................................... 27
Appendix C: Student Surveys ............................................................................................................ 28
Appendix D: Using CoursEval to Access Survey Results ...................................................................... 32
Appendix E: Evaluation vs. Assessment ............................................................................................. 34
Appendix F: Rubrics ......................................................................................................................... 35
Appendix G: Faculty Peer-to-Peer Review Form .............................................................................. 39
Appendix H: Course Syllabus Template ............................................................................................. 40
Appendix I: Outcomes Assessment Form .......................................................................................... 43
Appendix J: Pre-Public Release Review Process ................................................................................. 45
vi
1
NIU Institutional Effectiveness Department
The Institutional Effectiveness (IE) Department serves in an advisory role, under the Provost, as a resource
providing recommendations for assessment, data gathering and analysis, program review, and support of
accreditation efforts. IE is committed to being proactive in providing quality services to support the
university and decision makers with accurate and timely information needed to effectively lead the
university.
VISION
Build a university-wide culture of assessment and continuous improvement.
MISSION
Enhance university effectiveness, maintain accreditation, and advise leadership through
institutional evaluation and assessment.
GOALS
1. Coordinate and monitor compliance activities for institutional accreditation.
2. Produce accurate, concise, and timely data and analysis to support institutional
planning, leadership decision-making, and university publications.
3. Foster conversion of institutional assessment data into strategic action.
4. Measure achievement of institutional, program, and course learning outcomes.
2
Policies
Defense Intelligence Agency (DIA) is the executive agent for NIU, and therefore NIU falls under governance
of DIA policies, directives and instructions. NIU only develops policies to cover specific areas that are not
covered by DIA issuances.
NIU Policies can be found on the NIU SharePoint site on JWICS at:
https://iconline.coe.ic.gov/sites/NIU_MGT/NIU%20Policies/Forms/AllItems.aspx
DIA issuances can be found on JWICS at:
https://dia.coe.ic.gov/sites/Issuances/Lists/DIA%20Instructions/AllItems.aspx
Some of the most commonly used DIA policies are:
DIA Directives
DIAD 1000.100 DIA Editors Board 30 Jun 2015
DIAD 1010.100A Health Promotion and Wellness Program 24 Jul 2012
DIAD 1020.100 DIA Equal Employment Opportunity and Diversity Program
3 Jun 2016
DIAD 1100.100 Disclosure of DIA Organizational and Personnel Information
04 May 2015
DIAD 1426.100 Remedial, Disciplinary, Adverse, and Performance-Based
Actions 2 Feb 2017
DIAD 1500.001 DIA Civilian Fitness 03 Aug 2017
DIAD 2200.100 Combatting Trafficking in Persons 16 Jun 2016
DIAD 3200.100 Human Subjects Research and Adherence to Ethical Standards in DIA Supported Research
21 Oct 2016
DIAD 3305.100 Joint Reserve Intelligence Program 12 May 2014
DIAD 5100.100 National Intelligence University Board of Visitors 17 Mar 2014
DIAD 5200.100 Workplace Violence Prevention and Response 18 Jun 2014
DIAD 5200.200 Investigative and Special Investigative Inquiries 29 Dec 2015
DIAD 5240.200 Intelligence Oversight Program 18 Nov 2013
DIAD 5400.200 Freedom of Information Act Program 29 Jun 2018
DIAD 5410.600 DIA Policy and Procedures on Civilian Military and Contract Employee Contact with Congress
28 May 2014
DIAD 5700.100 Access Identification Badges 3 Jul 2014
3
DIAD 5801.100 Personnel Security Program 27 Oct 2015
DIAD 6055.100 Occupational Safety Health and Environmental Protection Program
2 May 2014
DIAD 6495.100 Sexual Assault Prevention and Response Program 1 Aug 2016
DIA Instructions
DIAI 1015.001 Drug-Free Workplace Program 1 Aug 2016
DIAI 1020.002 Reasonable Accommodation of Qualified Individuals with Disabilities
19 May 2015
DIAI 1025.002 Personnel Training 11 May 2015
DIAI 1025.003 Establishment and Support to Defense Intelligence Agency-Affiliated Private Organizations
24 Aug 2016
DIAI 1100.003 Outside Employment Reporting Process 6 Oct 2016
DIA 1300.001 Military Reserve Personnel Administration 19 Apr 2017
DIAI 1348.001 Military Personnel Awards 2 Nov 2015
DIAI 1350.001 Military Personnel Administration 20 Nov 2015
DIAI 1400.002 Civilian Compensation and Work Schedules 24 Apr 2015
DIAI 1400.008 Employment and Placement 03 Jun 2015
DIAI 1400.010 Federal Benefits 1 Aug 2016
DIAI 1404.001 Civilian Performance Management 1 Aug 2016
DIAI 1404.010 Civilian Career Assignment Programs 2 Sept 2016
DIAI 1416.001 Performance-Based Bonuses 22 Dec 2016
DIAI 1416.002 Injury Compensation Program 23 Jul 2013
DIAI 1422.002 Time and Labor Reporting 29 Oct 2013
DIAI 1424.001A Civilian Leave Programs 7 Jul 2017
DIAI 1426.002 Employee Grievance System 14 Dec 2015
DIAI 1432.001 Civilian Awards Program 1 Aug 2016
DIAI 1500.002 Telework Program 15 Jun 2017
DIAI 1700.001 Recording Person Administrative and Career Information in eZHR
20 Jul 2016
DIAI 4515.001 Temporary Duty Travel Request and Reimbursement Procedures
3 Aug 2016
DIAI 5000.035 DIA Acquisition Regulation Supplement and Instruction (DARSI)
9 Apr 2015
DIAI 5010.003 Internal Control Program 9 Feb 2015
DIAI 5145.001 Alternative Dispute Resolution Program 5 Oct 2014
DIAI 5330.002A Use of Copyrighted Materials 3 Mar 2015
4
DIAI 5340.001 Administrative Investigations 1 Mar 2017
DIAI 5400.001 Privacy and Civil Liberties Program 05 Feb 2015
DIAI 5400.005 Prepublication Review of Information Prepared for Public Release
8 Sep 2016
DIAI 5520.001 Agency Support Agreements 1 Jun 2016
DIAI 5700.001 Visitor Access Control 21 Aug 2014
DIAI 5800.001 Personnel Security Procedures 27 Oct 2015
DIAI 6010.001 Ergonomics Program 28 Apr 2014
DIAI 6050.003 Accident Investigation and Reporting 8 Mar 2017
DIAI 6050.004 Environmental Compliance Program for Facilities and Sites 02 May 2014
DIAI 6055.002 Occupational Health 22 Apr 2016
DIAI 7040.001 DIA Conference Approval Process 24 Jan 2017
DIAI 7040.003 Government Travel Charge Card and Travel Advances 29 Dec 2014
DIAI 7040.008 Coin Policy 23 Oct 2017
DIAI 7050.002 Whistleblower Protection 24 Jun 2013
DIAI 8460.002 Portable Electronic Devices 03 Mar 2015
DIAI 8500.003 Systems Access 23 Mar 2014
NIU must also comply with certain policies from other organizations within the Department of Defense
(DoD) and Office of the Director of National Intelligence (ODNI). The ones that most impact NIU are:
ODNI Issuances
Strategic Guidance for the National Intelligence University 2 Jan 2015
ICD 203 Analytic Standards 02 Jan 2015
ICD 204 National Intelligence Priorities Framework 02 Jan 2015
ICD 205 Analytic Outreach 28 Aug 2013
ICD 660 IC Civilian Joint Duty Program 11 Feb 2013
ICPG 660.1 IC Civilian Joint Duty Program Implementation Guidance 24 Jul 2015
CJCS Issuances
CJCSI 1800.01E Officer Professional Military Education Policy 29 May 2015 with Modifications
DoD Issuances
DoDD 5105.21 Defense Intelligence Agency 18 Mar 2008
DoDI 3305.01 National Intelligence University 24 Jan 2017
5
NIU’s Institutional Assessment Plan Each year IE executes a university-wide assessment plan of the university. The goal of the assessment
plan is to:
1) Assess student learning by ensuring students are acquiring and demonstrating the knowledge,
skills, and abilities taught in the classroom.
Evidence: student papers, tests, quizzes, presentations, theses, etc.
2) Assess university-wide effectiveness by measuring the quality of the NIU academic programs
and the effectiveness of university processes and policies.
Evidence: academic program review, surveys, focus groups, and interviews of NIU faculty, staff,
and students.
3) Collect direct and indirect evidence (See Appendix B: Direct vs. Indirect Measures of Assessment)
from a variety of sources that demonstrates student learning and the overall effectiveness of
NIU programs and faculty.
Evidence: The NIU Roadmap to Evidence repository
IE rolls out the Assessment Activity Plan at the beginning of each academic year. The plan includes the
list and schedule of the standard assessments, surveys, and reports that will launch for the year.
SURVEYSFall
Quarter
Winter
Quarter
Spring
Quarter
Summer
Quarter
Orientation
Research Fair
Research Workshop
Pre-JPME Acculturation Survey
Weekly President's Lecture Series
End of Course
End of Program - Degree
Post-JPME Acculturation Survey
External Stakeholder Alumni (every 2-3 years)
External Stakeholder IC Leader (every 2-3 years)
Executive/Reserve Distinguished Lecture Series
ASSESSMENTSFall
Quarter
Winter
Quarter
Spring
Quarter
Summer
Quarter
Academic Program Review - Planning
Academic Program Review - Execution
6
Reports are distributed within 2-8 weeks after the close of the assessment. IE initially distributes the
reports as follows:
Reports Recipient
Annual Report and FactBook ODNI, USDI and DIA Senior Leaders, BOV, University students, faculty and staff, and stakeholders
Orientation Report NIU Leadership, Deans, Directors, Department Heads
Research Fair and Research Workshop Summary
Reports
NIU Leadership and Research Fair and Workshop
Coordinators
End-of-Course Reports/ Executive Summary/Program
Level Summary Reports (including Bachelor’s,
Master’s, Core and Thesis)
Faculty (Course/Section Reports only)
NIU Leadership, Deans, and Academic Program and
Academic Center Directors
End-of-Program Executive Summary NIU Leadership, Deans and Directors
External Stakeholder IC Leader Report NIU Leadership and Office of Engagement
External Stakeholder Alumni Report NIU Leadership and Office of Engagement
NIU Facts and Figures Public (NIU Website)
President’s Lecture Series (PLS) Summary Report NIU Leadership, PLS Coordinator
Monthly Executive/Reserve Distinguished Lecture Series
NIU Leadership, DLS Coordinator
JPME Pre- and Post-Acculturation Surveys JPME Program Director
MSCHE Annual Institutional Update Posted to MSCHE Website
Figure 2: Report Distribution List
REPORTSFall
Quarter
Winter
Quarter
Spring
Quarter
Summer
Quarter
Annual Report - Prior Year
Annual Report - Current Year
At a Glance
Common Data Set
Quick Fact Report
FactBook
Graduation and Retention Report
Orientation Report
End of Course Summary Reports
End of Program Reports
NIU Historical Data Report
Research Fair Report
Research Workshop Report
Specialized Reports
Figure 1: Sample Assessment Activity Plan
7
Assessment Strategies and Tools
IE Assessment of Institutional Effectiveness
IE utilizes a variety of assessment methods to collect data and evidence, and to identify and analyze
performance outcomes at all levels of the university. Some of these methods include:
Online Surveys: IE deploys numerous surveys throughout the academic year to collect data from
students, faculty, and/or staff about their opinions and experiences regarding university
programs, courses, services, processes, practices, and governance. (See Appendix C: Student
Surveys for information about the student surveys.)
Focus Groups: A form of qualitative research in which a target or sample group of students,
faculty, and/or staff are asked about their perceptions, opinions, beliefs, and attitudes towards a
specific subject in order to understand and/or identify criteria that may help solve or resolve a
problem (or opportunity).
Academic Program Review: A review process designed to look across NIU at both academic and
non-academic programs and review them for quality, demand, cost effectiveness, and centrality
to mission.
Faculty Assessment of Student Learning
Assessment of student performance is critical in determining the effectiveness and success of NIU. NIU
evaluations are continual and interactive, and involve students, faculty, and University leadership.
Evaluations of NIU classes and curricula are carefully constructed to ensure clear linkage among desired
learning outcomes, changes to course syllabi, joint educational goals, and the NIU mission.
NIU faculty use a combination of strategies and tools to evaluate and assess3 student learning. Some of
these strategies and tools include:
Observation checklist: A list of questions an observer will be looking to answer in relation to
course or program outcomes when doing a specific observation of a classroom or student
academic exercise (problem solving, verbal communication, collaboration, etc.)
Anecdotal notes: A record of specific observations of individual student behaviors, skills, and
attitudes as they relate to the outcomes in the course or program. Such notes provide cumulative
information on student learning and direction for further instruction4.
Portfolios: A purposeful collection of student work samples, self-assessments, and faculty
assessment/feedback that reflect student progress.
3 See Appendix E: Evaluation vs. Assessment to learn the differences between evaluation and assessment goals.
4 Learnalberta.ca/content/mewa/htlml/assessment/anecdotalnotes.html “Assessment Strategies and Tools: Anecdotal Notes.
8
Questions and answers: Verbal or written tests of knowledge that can be delivered in multiple
forms — exams, quizzes, papers, assignments, in-class activities or discussion, presentations.
Grading scales: Tools that state specific criteria for levels of learning and/or meeting
course/program outcomes.
Rubrics: A tool that consists of a fixed measurement scale and detailed description of the
characteristics for each level of performance that is used to evaluate a student’s product or
performance. The descriptions focus on quality (and not quantity) of the product or performance
being evaluated. Bloom’s Taxonomy is the standard used at NIU for developing criteria to
describe acceptable levels of performance. (See Appendix F: Rubrics for more information.)
At NIU, the core curriculum represents a framework of knowledge that the University believes the
students must have to understand intelligence at the strategic and operational levels and to study more
advanced topics.
9
Faculty Curriculum Vitae Requirements
In addition to managing the university’s assessment plan, IE coordinates and monitors compliance activities for institutional accreditation. One of the key activities involves maintaining a file of current curriculum vitae (CVs) for all faculty. This is essential to the support of MSCHE Standard III5, criteria 2, stating:
Student learning experiences that are designed, delivered, and assessed by faculty (full-time or part-time) and/or other appropriate professionals who are:
o Rigorous and effective in teaching, assessment of student learning, scholarly inquiry, and service, as appropriate to the institution’s mission, goals and policies;
o Qualified for the positions they hold and the work they do;
o Sufficient in number;
o Provided with and utilize sufficient opportunities, resources, and support for professional growth and innovation;
o Reviewed regularly and equitably based on written, disseminated, clear, and fair criteria, expectations, policies and procedures.
Faculty must submit updated CVs at the beginning of each academic year. IE sends out a data call for CVs at the start of the academic year. Adjunct, part-time or new faculty starting at other times of the year, will submit CVs upon arrival.
There is no prescribed format for CVs, however, items that should be considered for inclusion are:
Name and contact information
Education: degrees, school names, thesis/dissertation titles, Professional licenses/certifications
Professional/Academic Honors and Awards
Professional Experience
Extracurricular and Volunteer Experience (Relevant to the position)
Professional Affiliations and Activities (Specific to Academic Career)
Research Experience
Publications and Presentations
Interests and Qualifications
5 Middle States Commission on Higher Education (MSCHE), “Standards for Accreditation and Requirements of Affiliation”, Thirteenth Edition, pg. 7.
10
Academic Review Roles
Although IE facilitates a majority of the institutional assessment efforts at NIU, NIU faculty have several
opportunities to actively participate in the ongoing review of all NIU programs through various review
processes and forums.
Quality Circle Review
Through the Quality Circle process, faculty examine their own observations, peer reviews, student
critiques, and inputs from the Deans, Program Directors, and Director, IE, of the courses they teach. The
Quality Circle lead and faculty colleagues decide on modifications for the course description, mission
statement, topical outline, and desired student learning outcomes. Minor modifications to electives can
be handled between the Quality Circle and Program Directors. Major modifications must be presented to
the Dean(s) and briefed to the entire faculty of the College and/or School that owns the program
curriculum. Program Directors from the Master of Science of Strategic Intelligence (MSSI) and Master of
Science and Technology Intelligence (MSTI) programs supervise all of the Quality Circles to ensure that
course design, revision, and updates occur regularly.
Curriculum Working Groups (CWGs). CWGs and Concentration Chairs represent critical components of
the NIU review process. The faculty’s findings are necessary for a credible assessment of the University’s
mission success.
The College of Strategic Intelligence CWG (CSI/CWG) is chaired by the MSSI Program Director and includes
faculty members elected from the CSI faculty.
The CSI/Graduate Core Working Group (GCWG) is responsible for managing the MSSI and BSI program
curricula and ensuring all undergraduate and graduate courses and certificates nominated for inclusion in
the MSSI and Bachelor of Science in Intelligence (BSI) curricula meet academic minimums for course
credits and MSCHE accreditation standards. NIU/GCWG is chaired by Graduate Program Directors from
the College and School, and ensures the quality and relevance of the graduate core curriculum.
Concentration Chairs. Within the Anthony G. Oettinger School of Science and Technology Intelligence
(S&TI), faculty selected as Concentration Chairs are responsible for managing curriculum concentrations.
Findings and recommendations are made by these Chairs with the appropriate oversight from the S&TI
Dean or any other members of the University leadership. Concentration Chairs are selected by the Dean
of the School, and may also serve as Faculty Supervisors.
Academic Policy and Standards Committee (APSC)
Co-chaired by the Graduate Program Directors from the College and School, APSC is an administrative
committee responsible for reviewing academic policy, admissions criteria, and institutional standards, and
11
makes policy recommendations to the Deans. APSC also reviews student complaints, appeals, and grade
protests, and conducts investigations into alleged academic misconduct.
Faculty Senate
Another important part of NIU program review is the Faculty Senate. The Faculty Senate advises the
President, Provost, Deans, and Academic Program Directors on matters affecting the academic welfare of
the University. The Senate promotes effective faculty participation and communication in the governance
of the University as a whole. Participation as an NIU Faculty Senator is an inherently governmental
function, so it is limited to government employees.
The NIU Chair of the Faculty Senate is elected by the senators, serves as a member of the President’s
Council, and attends the semiannual Board of Visitors (BOV) meetings as a guest. Lastly, the Deans solicit
faculty inputs and prepare agendas for periodic faculty meetings. The faculty meeting is an important part
of University communications and review, and includes discussion on curriculum changes, course
additions, instruction, research, outreach, human subjects review, institutional assessment, institutional
effectiveness, and student issues. Unresolved issues are considered for further discussion with
Concentration Chairs, CWGs, or the APSC.
University Leadership
The recommendations and inputs from the various review processes within the University are discussed
in many leadership fora: Provost’s Deans and Directors meetings and other leadership meetings. When
issues beyond the normal realm of University activities arise, the Deans, Provost, IE Director, and
President may create special committees to review such issues and make recommendations. The Joint
Professional Military Education (JPME) Quality Circle serves as one example. Formed to work in
conjunction with the Quality Circles for courses containing Joint Learning Areas (JLAs), the JPME Quality
Circle ensures that the JLAs are presented effectively in the course material, and has evolved into an
important, ongoing body.
The various NIU review processes culminate at every other week President’s Council Meeting that is
attended by the President, Provost, Chief of Staff, Vice Presidents, CSI and S&TI Deans, Program Directors
and Front Executive Staff.
At this meeting, the University President leads a dialogue on all issues of importance to the University,
including those outside the University environment that could affect the University’s mission.
Decisions on curriculum issues and University policy are finalized, and typically take effect upon the
issuance of the next catalog or appropriate handbook (policy or student). If a policy change takes effect
before the issuance of the relevant publication, or immediately, the President directs University
leadership to provide immediate and clear communication throughout the University.
12
Peer-to Peer Review
An important part of ensuring the quality of instruction is regular review of the faculty, their expertise and
teaching abilities. While there are formal reviews conducted by the Deans, Associate Deans or leadership,
another type of review is the peer-to-peer reviews. These can be conducted in a number of ways, from
new faculty being reviewed by the mentor or other faculty to help them with their teaching styles or as
part of teaching a new course; to regular reviews of faculty by supervisors or program directors to
seasoned faulty being observed by less experienced faculty to learn from observations. A peer-to-peer
review form has been developed to help observers to evaluate faculty and their teaching. (See Appendix
G: Faculty Peer-to-Peer Review Form.)
13
Development of Learning Outcomes
Learning outcomes describe what learning students should have acquired or should be able to demonstrate at the end of the class or program of study. Learning outcomes emphasize the knowledge, skills and abilities that are going to be taught, rather than the form of instruction. Some guidelines when writing Student Learning Outcomes:
Include one action verb per outcome and include an observable student behavior.
Begin with the end in mind (what the student should accomplish.)
Student learning outcomes should be measurable.
Students should be assessed, based on the Learning Outcomes.
Note: The action verb chosen for the learning outcome statements should focus on what is going to be assessed and the tools, instruments and metrics that will be used to assess the extent of the intended learning. Bloom’s Taxonomy provides an excellent source of action verbs and is NIU’s standard for classifying learning outcomes.
The most industry-wide accepted method of writing outcomes is the:
ABCD Method
o A – Audience (Learners, who will be doing the performance) o B – Behavior (Performance, Skill or Knowledge learner will be able to do) o C – Condition (Under what circumstances) o D – Degree (How well must they perform)
Suggested examples:
o Cognitive (problem solving/synthesis level) – “C: Given two cartoon characters, A: the student B: will be able to list five major personality traits of each of the two characters, combine these traits into a composite character, and develop a short storyboard for D: that illustrates three to five of the major personality traits of the composite character.”
o Cognitive (application level) – “C: Given a sentence written in the past and present tense, A: the student B: will be able to re-write the sentence in future tense D: with no errors in tense or tense contradictions (i.e. I will see her yesterday.).
o Cognitive (comprehension level) – “C: Given examples of constructivist and non-constructivist activities in a college classroom, A: the student B: will be able to accurately identify the constructivist examples and explain why each example is or isn’t a constructivist activity D: in 20 words or less.”
o Psychomotor – “C: Given a standard balance beam raised to a standard height, A: the student B: will be able to walk the entire length of the balance beam D: steadily, without falling off, within a six-second time span.”
o Affective – “C: Given the opportunity to work in a team with several people of different races A: the student B: will demonstrate a positive increase in attitude towards non-discrimination of race, D: as measured by a checklist utilized by non-team members.”
14
When using the ABCD Method, the outcomes should meet the SMART criteria:
o Specific – Focuses on specific category of student learning o Measurable – Data can be collected o Attainable – Attainable given amount of experience knowledge o Relevant – Aligned with program learning outcomes o Time-bound – Any time restrictions
When writing learning outcomes , you should answer the following questions:
What do you want students to accomplish?
What knowledge, skills and abilities should the student demonstrate?
How will the students be able to demonstrate what they have learned? (What steps will they take to accomplish the outcome? What activities will they do? How will they acquire the learning? Under what conditions will the learning occur?)
How do the program and organizational outcomes fit with the program?
(What evidence do you have to demonstrate that learning took place? What criteria will be used to evaluate the evidence? Who will conduct the evaluation?)
Verbs to Avoid
In order to measure the intended learning outcomes, it is best to avoid verbs that represent concepts that are difficult or impossible to measure. Examples of some of these are:
Appreciate
Be aware of
Be familiar with
Believe
Comprehend
Know
Learn
Understand
15
Bloom’s Taxonomy The Taxonomy of Educational Objectives is a framework for classifying statements of what we expect or
intend students to learn as a result of instruction. The original hierarchical model was created in 1956,
and named after Dr. Benjamin Bloom, who chaired a committee of educators (Bloom, Engelhart, Furst,
Hill & Krathwohl) that devised the taxonomy. It identified three domains of education activities or
learning:
Cognitive – mental skills (knowledge)
Affective – growth in feelings or emotional areas (attitude or self)
Psychomotor – manual or physical skills
The cognitive domain involves knowledge and development of intellectual skills. The cognitive domain is
then further broken down into six major categories that are used to classify educational learning
outcomes into levels of complexity or difficulty. The taxonomy was revised in 2001 by Lorin Anderson and
David Krathwohl and is known as the Revised Taxonomy. At NIU, the taxonomy is used to structure
curriculum and program learning outcomes, assessments and activities.
The Revised Taxonomy identifies the following levels of cognitive learning6:
Remembering – Retrieving, recognizing, and recalling relevant knowledge from long-term
memory
Understanding – Constructing meaning from oral, written, and graphic messages through
interpreting, exemplifying, classifying, summarizing, inferring, comparing and explaining
Applying – Using information in new ways; carrying out or using a procedure or process through
executing or implementing
Analyzing – Breaking material into constituent parts; determining how the parts relate to one
another and to an overall structure or purpose through differentiating organizing, and attributing
Evaluating – Making judgements based on criteria and standards through checking and critiquing;
defending concepts and ideas
Creating – Putting elements together to form a coherent or functional whole; reorganizing
elements into a new pattern or structure through generating, planning or producing
6 International Assembly for Collegiate Business Education 2014-2016, “Bloom’s Taxonomy of Educational Objectives and Writing Intended Learning Outcomes Statements.
16
Reasons for Using Bloom’s Taxonomy7
Accurately measuring student’s abilities requires an understanding of different levels of cognition
that are critical for learning.
Developing intended student learning outcomes according to Bloom’s Taxonomy helps students
understand what is expected of them.
Using Bloom’s Taxonomy to develop intended student learning outcomes helps professors to plan
and deliver appropriate instruction.
Developing intended student learning using Bloom’s Taxonomy helps faculty to design and
implement appropriate assessments, tasks, measures, and instruments.
Having intended student learning outcomes based on Bloom’s Taxonomy helps to ensure that
instruction and assessment are appropriately aligned with the intended outcomes.
Learning outcomes for upper level undergraduate and graduate programs should use the more complex
skills on the right of the chart below.
7 International Assembly for Collegiate Business Education 2014-2016, “Bloom’s Taxonomy of Educational Objectives and Writing Intended Learning Outcomes Statements.
17
Action Words for Bloom’s Taxonomy8
Knowledge Understand Apply Analyze Evaluate Create
define explain solve analyze reframe design
identify describe apply compare criticize compose describe interpret illustrate classify evaluate create label paraphrase modify contrast order plan list summarize use distinguish appraise combine name classify calculate infer judge formulate state compare change separate support invent match differentiate choose explain compare hypothesize recognize discuss demonstrate select decide substitute select distinguish discover categorize discriminate write examine extend experiment connect recommend compile locate predict relate differentiate summarize construct memorize associate show discriminate assess develop quote contrast sketch divide choose generalize recall convert complete order convince integrate reproduce demonstrate construct point out defend modify tabulate estimate dramatize prioritize estimate organize tell express interpret subdivide find errors prepare copy identify manipulate survey grade produce discover indicate paint advertise measure rearrange duplicate infer prepare appraise predict rewrite enumerate relate produce break down rank role-play listen restate report calculate score adapt observe select teach conclude select anticipate omit translate act correlate test arrange read ask administer criticize argue assemble recite cite articulate deduce conclude choose record discover chart devise consider collaborate repeat generalize collect diagram critique collect retell give examples compute dissect debate devise visualize group determine estimate distinguish express
illustrate develop evaluate editorialize facilitate judge employ experiment justify imagine observe establish focus persuade infer order examine illustrate rate intervene report explain organize weigh justify represent interview outline make research judge plan manage
review list question negotiate rewrite operate test originate show practice propose trace predict reorganize transform record report schedule revise simulate schematize transfer simulate write solve speculate structure support test validate
8 Center for University Teaching, Learning, and Assessment, http://uwf.edu/cutla/SLO/ActionWords.pdf.
18
Course Syllabi
The following guidance recommends best practices for syllabus construction as well as the guidelines for
including course learning outcomes on course syllabi. Use the NIU Syllabus Template (See Appendix H:
Course Syllabus Template) to build your syllabus.
Overview
Courses at NIU must satisfy the common student learning outcomes of their respective program. Each
faculty member determines the specifics of course implementation, provided the established
competencies and learning outcomes for the course are addressed.
Instructors often have differing teaching styles for a particular course, but all must have the same course
description (and this must match the one published in the University Catalog), the same contribution to
mission statement, the same overall lesson topics, and the same desired student learning outcomes on
their syllabi.
Instructors may choose their own texts (in addition to at least one text that may be used by all the classes),
and define their own expectations, deliverables, teaching methodology, and grading practices.
Syllabus Guidance
The syllabus provides an overall layout for a course. University policy establishes all courses shall have a
syllabus and the syllabus shall be uploaded into Blackboard and made available to students at or near the
start date of the course. The syllabus shall include, but not be limited to the following:
Basic course and instructor contact information
Course description and contribution to mission
Course goals and student learning outcomes or learning goals
Joint Learning Outcomes (if applicable)
Course format, procedures, and/or policies
Course requirements
The instructor's grading criteria or policies
Course schedule or outline
In addition to the required components, NIU recommends the following university statements as an
addendum to the course syllabus:
Academic Integrity
Attendance
Incomplete and Withdrawal grades
Inclusivity
Surveys
19
I. Basic Information: Provide basic information about the course, such as course numbers and titles,
instructor contact information, and class time and location.
II. Course Description/Contribution to Mission: The description on the syllabus must be consistent with
the description listed in the current NIU catalog (http://ni-u.edu/wp/niu-academic-catalog/). Additional
information may be included after the bulletin description.
Contribution to Mission: Map course learning outcomes (CLOs) to program learning outcomes (PLOs).
Why does this course exist? How does it fit in with the rest of the field/area’s curriculum?
III. Learning Outcomes: For all students, clear learning outcomes for the course provide a roadmap of the
material to be covered over the semester. Clearly articulated learning outcomes are also important for
faculty members to ensure that assignments and learning activities align with the course outcomes.
Learning outcomes also provide a mechanism to ensure that evaluation methods and activities align with
the content that is presented.
IV. Joint Learning Outcomes: For all classes that have Joint Learning Outcomes (JLOs), those outcomes
must also be included in the syllabi. It is important for both students and faculty to understand what JLOs
will be covered during the course.
V. Course Format and Procedures: This is where specifications for attendance, participation, how class
will be carried out, instructor expectations, etc. should be spelled out. In addition, if the course has
multiple formats (like lecture & recitation, lab and discussion, group learning projects and/or
presentations) these should be explained clearly. The syllabus should provide instructions to students on
how they are expected to meet the learning outcomes for the course. For example: Will most outcomes
be met through examination? Through participation? or through projects?
1. Class attendance and participation:
2. You will meet the outcomes listed above through a combination of the following activities in this
course:
Attend
Complete
Participate
3. Late Work: (Example: Be sure to pay close attention to deadlines—there will be no makeup
assignments or quizzes, or late work accepted without a serious and compelling reason and
instructor approval.)
20
VI. Course Requirements: Whatever tasks and assignments are included in your course, they should be
aligned with the specified learning outcomes (final learning state, skills, knowledge, attitudes and values
the students leave the course with) you have defined and specified earlier.
Provide full text citations of all required materials required resources.
1. Course readings:
a. Required text: [Title. Author]
b. Background readings, (what and where can they be accessed)
2. Course materials: lab manuals, technology, supplies, and any other materials required or
recommended for the student to complete the course requirements.
3. Use of the course Blackboard or other web site?
4. Resources?
VII. Evaluation Procedures/Grading Criteria: The syllabus should indicate how students are evaluated,
including tests, quizzes, papers, assignments, weight of the assignments, etc. and clearly identify how the
course grades are determined.
Criteria for grading includes the grading scale used for the course. If points are earned, be sure the total
number of points is correct and all points are accounted for in the grading scale.
Grading Scale Examples
Final Exam 40% of final grade D = 60 to < 70%
Total Points 100 points F = < 60%
VIII. Course Schedule/Outline/Calendar of Events: Provide students with a tentative projected outline of
significant events that occur throughout the semester, including assignments, projects, examinations,
field trips, guest speakers, etc. (May change to accommodate guest presenters & student needs)
For example:
Topics Readings to be discussed Assignment Week 1 Topics/Major Concepts covered [Text] Chapter #, additional readings
from course packet, handouts Week 2 Experiential and collaborative learning -
Guest Lecturer [Text] Chapter #, additional readings from course packet, handouts
Week 3 Exam #1 Week 4 Experiential and collaborative learning -
Guest Lecturer [Text] Chapter #, additional readings from course packet, handouts
Etc. Uses of technology Guest Lecturer
[Text] Chapter #, additional readings from course packet, handouts
21
IX. Academic Integrity: Each student in this course is expected to abide by the NIU Code of Academic
Integrity. Any work submitted by a student in this course for academic credit will be the student's own
work. [Optional: For this course, collaboration is allowed in the following instances: list instances.]
You are encouraged to study together and to discuss information and concepts covered in lecture
and the sections with other students.
You are expected to credit properly and accurately the source of materials directly cited or
indirectly used (i.e., paraphrased) in any oral or written work. All students’ work shall be their
own, unless otherwise properly noted.
You may not use entire papers or substantive selections of a paper from one course to complete
work for another course or courses.
A grade of F will normally be assigned for any work proven to be undertaken or performed in violation of
academic integrity. All instances of alleged violations of academic integrity will be handled in accordance
with published NIU policies.
X. Attendance: Students are expected to attend all scheduled class sessions. Students missing more than
one session face, at the discretion of the faculty member, penalties ranging from the lowering of the final
grade to failure in the course. A student who misses three or more sessions and does not withdraw faces
removal from the course and a failing grade.
XI. Incomplete ‘I’ and Withdrawal ‘W’ Grades: A faculty member may assign an incomplete (I) grade to a
student whose work is satisfactory but is unable to meet all course requirements for extenuating
circumstances. It is the student’s responsibility to discuss with the faculty member the possibility of
receiving an ‘I’ grade. All requirements must be completed by the 9th week of the following quarter, or
the 7th week of an eight-lesson quarter, and the faculty member must turn in the final grade by the 10th
week of the following quarter. If a final grade is not submitted by this deadline, the ‘I’ grade is converted
to an F. The Dean may extend the deadline in exceptional cases. As long as the ‘I’ remains on the transcript,
it is treated as unsatisfactory academic performance.
Withdrawing from a course: Students may withdraw from a course until the midpoint of that course, such
as the end of the 5th session of a 10-session course, or the end of the 4th session of an 8-session course.
A notation of W is assigned to the transcript of a student who withdraws from a course before the
withdrawal deadline. Withdrawal from a course after the midpoint of that course is allowed only for non-
academic reasons and requires permission of the faculty member and the approval of the Dean. Students
who are approved to withdraw after the midpoint of a course are assigned a grade notation of WP
(withdraw passing) or WF (withdraw failing) by the faculty member, depending on the student’s academic
standing in the course at the time. The grade notation of WP carries no credit or academic penalty. A
grade of WF is treated as an F when calculating the grade-point average and triggers academic warning.
Students withdrawing at any time must complete the necessary documentation through the Enrollment
Services Office. Students who stop attending classes without an official withdrawal or the Dean’s approval
receive a grade of F for the course.
22
XII. End of Course Survey: Students are expected to complete a course survey near the end of this course.
Course surveys are an essential component of the university’s educational assessment process. Failure to
complete the end-of-course survey may result in the withholding of all student grades by the Dean. Follow
university instruction on accessing the university survey site (myCoursEval) and completing course
surveys. Contact [email protected] if you have any questions regarding course surveys.
XIII. Inclusivity Statement: We understand that our faculty and students represent a rich variety of
backgrounds and perspectives. NIU is committed to providing an atmosphere for learning that respects
diversity. While working together to build this community we ask all members to:
share their unique experiences, values and beliefs
appreciate the opportunity that we have to learn from each other in this community
value each other’s opinions and communicate in a respectful manner
keep confidential discussions that the community has of a personal (or professional) nature
use this opportunity together to discuss ways in which we can create an inclusive environment in
this course and across the Intelligence and National Security communities.
23
Academic Program Review
NIU conducts academic program review (APR) on a four-year cycle in which existing academic programs
and services are reviewed for their quality, demand, cost-effectiveness, and centrality to mission. Each
unit takes a comprehensive look at its opportunities and challenges in regard to the NIU strategic plan and
university standards of accreditation. The results of the review are used by the unit or University to
develop and implement plans to sustain, improve, or change programs and services. Detailed instructions
are found in the NIU Academic Program Review Process Guide.
Figure 3: The APR Schedule (AYs 2018-2021)
Purpose
Self-assessment for planning
Improvement of programs and/or services
Enhancement of student learning
Support of Middle States Commission on Higher Education (MSCHE) requirements
In cooperation with NIU’s culture of assessment and continuous improvement
Review of Programs/Units
APR provides each unit the opportunity to pursue a path of inquiry, discovery, and assessment. The
emphasis of the review is on forward planning, informed by analysis of recent data trends with units
identifying a detailed plan to sustain excellence, correct deficiencies, or enhance the educational mission
of NIU. The unit is not expected to address issues during the review period, but will develop plans to do
so in a timely manner and update progress in follow-up years in-between the next review cycle.
Annual Update
The Unit is expected to take actions to address the findings of the program review and report on actions
taken by submitting a progress report every year after the conclusion of the review. Reviews are
scheduled every 4 years. IE is responsible for scheduling the annual reports and next review.
Year 1:
Outreach
Admissions & Registration
Year 2:
STI and Finance
Year 1 Update
Year 3:
CSI
Year 1-2 Updates
Year 4:
Research & IE
Year 1-3 Updates
24
NIU’s Roadmap to Evidence
The Outcomes Assessment Form
It is important to measure the outcomes that are derived from the various means of assessment, e.g.,
changes to curriculum based on end-of-course surveys, changes to courses, changes to required readings,
or types of student assessments, etc. While taking notes during curriculum working groups, quality circles,
etc. may capture the data, it can be difficult to then find the information needed when trying to provide
the evidence for accreditation purposes.
The Outcomes Assessment Form (Appendix I: Outcomes Assessment Form) was developed to capture
information about the type of meeting, purpose of the meeting, alignment with strategic, MSCHE, and
JPME goals and outcomes, evidence, discussion and analysis conducted, and then the decisions, outcomes
or recommendations that come out of that meeting. There is also a space for any follow-up activities that
needs to be conducted.
This form is recommended to be used for meetings of groups evaluating curriculum, goals, outcomes, and
other significant events impacting the way NIU conducts business.
Documenting Evidence of Student Learning Outcomes and Continuous Improvement
NIU’s Roadmap to Evidence (RTE) is an archive of documented evidence demonstrating NIU is meeting its
accreditation standards and requirements. The evidence is collected from leadership planning and
decision making, policy, student feedback, academic program review, faculty working groups (outcomes
assessment form and support documents (meeting notes, briefs, and other updates), and faculty
evaluation and assessment of student learning.
The RTE is maintained by IE. RTE evidence is unclassified and processed through the prepublication
process (Appendix J: Pre-public Release Review Process) for release to NIU’s MSCHE and JPME
accreditation reviewers.
IE maintains three versions of the RTE:
A physical database of all archived evidence that exists on the IE unclassified shared drive. This
database is regularly updated with the most current evidence.
A virtual, password protected library of the most current evidence prepared for the latest MSCHE
review. This online RTE (http://ni-u.edu/RTE/) is normally updated in support of MSCHE reporting
(self-study; periodic review) and visits.
A physical database of all JPME archived evidence that exists on the IE unclassified shared drive.
This database is normally updated before a J7 team visit.
25
Appendix A: Assessment Acronyms and Glossary
APR Academic Program Review
APSC Academic Policy and Standards Committee
BOV Board of Visitors
BSI Bachelors of Science in Intelligence
CLOs Course Learning Outcomes
CSI College of Strategic Intelligence
CSIR Center for Strategic Intelligence Research
CVs Curriculum vitae
CWGs Curriculum Working Groups
DIA Defense Intelligence Agency
DIAD DIA Directives
DIAI DIA Instructions
DoD Department of Defense
GCWG Graduate Core Working Group
IE Institutional Effectiveness Department
ILOs Institutional Learning Outcomes
JLAs Joint Learning Areas
JLOs Joint Learning Outcomes
JPME Joint Professional Military Education
MSCHE Middle States Commission on Higher Education
MSSI Master of Science of Strategic Intelligence
MSTI Master of Science and Technology Intelligence
NIU National Intelligence University
ODNI Office of the Director of National Intelligence
PAJE Process for Assessment of Joint Education
PLOs Program Learning Outcomes
RTE Roadmap to Evidence
S&TI Science and Technology Intelligence
Assessment: The ongoing process of measuring effectiveness for the purpose of improvement (to increase
quality).
Assessment Activity Plan: NIU’s schedule of standard assessments, surveys, and reports that will launch
during the academic year. This plan is developed by the IE Department.
Evaluation: The process of observing or measuring, at a point in time, against an established target or
standard for the purpose of judging value or quality. (The primary focus is on scores or grades.)
Learning Outcome: Statement of which a student would know and/or be able to perform as a result of a
learning experience (completion of a course; program; certificate).
26
Student Learning Outcome Assessment: The process of using direct and indirect measures to evaluate
student learning, and using fact-based evidence to change and improve programs. The primary focus is
on learning, teaching, and outcomes. Findings may result in changes in learning, teaching, or non-
academic support and services. Examples: Curriculum Working Group and Quality Circle assessments.
Measures: Means used to benchmark performance and to determine if an outcome or goal/objective is
achieved.
Direct Measure: Measure that evaluates the actual performance or achievement of an identified outcome
(administrative or student learning). A direct measure is preferable to an indirect measure because it is
tangible and visible. Examples: performance output (count; product), student output (paper; project;
presentation), the application of a rubric to assess an identified outcome.
Indirect Measure: Measure that signifies performance or achievement through the evaluation of
something else (i.e. opinion; perception; satisfaction.) An indirect measure may help substantiate a direct
measure in that an indirect measure along with a direct measure is expected to provide a more
comprehensive view of student learning (or process or service impact). Examples: survey, interview, peer
review, graduation rate, course grade*.
Academic Program Review (APR): A systematic, comprehensive review of a unit’s quality, demand, cost
effectiveness, and centrality to mission that results in a self-assessment report and action plan, APR
committee review report, and briefing to senior leadership.
Unit/Program Action Plan: Plan based on the self-assessment conducted in an academic program review
that describes the direction and goals (including benchmarks and timelines) for the next 4-year review
period.
Roadmap to Evidence (RTE): NIU's online repository of direct and indirect evidence of student learning
and compliance with Middle States Commission on Higher Education and Joint Professional Military
Education accreditation standards and requirements.
*Reasons why course grades are considered an indirect measure: 1) a course grade represents a
combination of course learning outcomes, where performance is averaged into a final grade 2) course
grades often include corrections not related to learning outcomes, such as extra credit or penalties for
excused absences.
27
Appendix B: Direct vs. Indirect Measures of Assessment
28
Appendix C: Student Surveys
About Surveys
Surveys are an essential component of NIU’s institutional assessment plan that allows NIU to collect indirect evidence of student learning. NIU surveys provide students and faculty with an opportunity to tell us what is going well and what should be improved. This input provides the data necessary to analyze, maintain, and improve the University’s institutional effectiveness, academic programs, and student services. NIU uses the CoursEval survey tool for online survey processing. See Appendix D: Using CoursEval to Access Survey Results for more information about this tool. The IE manages university assessment processes. If you have questions or concerns regarding NIU surveys and survey reports. Contact IE at: [email protected]. NIU Survey Schedule Weekly Surveys
President’s Lecture Series (PLS) Survey: The PLS Survey captures student satisfaction with the lecture and lecturer and collects additional information used to improve the quality of the lecture series. This survey runs weekly for the duration of the Fall, Winter and Spring quarters.
Quarterly Surveys
End-of-Course (EOC) Survey: The EOC Survey provides students with the opportunity to share their course learning experience. NIU uses the data and information to help understand how well a course was presented, how it enabled student learning, and how the course may be improved to better meet learning outcomes. This survey runs quarterly at/near the close of each course.
Annual Surveys
Orientation Survey: The Orientation Survey captures student satisfaction with student on-boarding and orientation. NIU uses the data and information collected to improve admissions, orientation, overall university processes and information services. The survey is open during the NIU Fall orientation and closes at the end of August.
Research Fair Survey: The Research Fair Survey captures student satisfaction and interest with the NIU Research Fair. NIU uses the data and information collected to improve the planning and execution of future Research Fair/Topics. This survey is open annually during the Research Fair.
Methods & Research Workshop Surveys: The Research Workshop Surveys are designed to capture student interests at the Workshops. NIU uses the data and information collected to enhance workshop planning and experiences. This survey is open annually during the Research Workshops.
29
End-of-Program (EOP) Survey: The EOP Survey provides degree-seeking students with an opportunity to share their views about their university experience and their satisfaction with their degree program and their views about the various services offered by the University. This survey is open annually from May through July.
Biennial Surveys
External Stakeholder Surveys: External Stakeholder Surveys collect data and information from IC leaders, University alumni, and incoming students that help ensure that NIU is fulfilling its mission. The surveys also provide insight into what the Institution is doing well and opportunities for enhancement.
Additional Surveys: Leader and Alumni Surveys are run every 2 to 3 years. The Incoming Student Survey runs annually as part of the Orientation Survey.
Other Surveys
Faculty Orientation Survey: The Faculty Orientation Survey captures new faculty experiences and satisfaction with the onboarding and orientation processes. This surveys runs as needed throughout the year following the end of a faculty orientation session.
Faculty Access to Survey Results Survey results are available for Faculty review after surveys close. Faculty members receive an e-mail notification announcing that reports are available that includes login information for the survey site. E-mail notifications are sent to the faculty member’s e-mail address on file in the Registrar’s Jenzabar System (the initial unclassified or personal e-mail address provided during faculty setup). If there is no e-mail on file, the Survey Management Team may update the system using your dodiis.mil account. If you do not receive an e-mail notification, the Survey Management Team does not have an e-mail address on file and/or the Team could not find a dodiis.mil account for you. NOTE: Please contact the Survey Management Team at: [email protected] if you want your survey results to go to a different e-mail address. Survey Results Access:
Faculty: Individual End-of-Course (EOC) survey results are available to the faculty member assigned as the instructor for the course.
Lead/Point of Contact for the Research Fair, Research Workshop, etc.: The respective survey results are available to the lead/point of contact.
30
Faculty Department Chairs: EOC survey results for faculty members assigned to a department are available to the Faculty Department Chair. Some reports will automatically show in your Report queue. Others may need to be manually e-mailed to you by the Survey Management Team.
Program & Center Directors: EOC survey results for program courses are available to the respective Program Director. Academic Center Directors have access to all courses taught at their center, using the course section number.
Deans & Associate Deans: EOC survey results for degree courses and results from shared degree courses (i.e. graduate core courses) are available to all Deans and Assistant Deans. Deans also have access to the EOP survey results and all EOC and EOP summary results reported to the NIU Provost and President.
Outreach Office: PLS survey results are available to the Vice President and Deputy Vice President of Outreach and the PLS Manager.
NIU Leadership: A summary of EOC and EOP survey results are available to the NIU President, Provost and Chief of Staff. A summary of EOP survey results are reported in the annual NIU FACTBOOK.
Survey Results Utilization:
• May be reviewed by NIU Leadership, Deans, and Program Managers/Chairs who make course, program, teaching, and faculty decisions.
• May be incorporated into the Institution’s accreditation report of other critical assessment
reports, and made available for future data requests or publishing. Reports are edited to maintain anonymity and ensure they are unclassified.
• Are stored and used as evidence of NIU’s assessment processes in accordance with Middle States
Commission on Higher Education (MSCHE) and Joint Professional Military Education (JPME) assessment standards.
Survey Reports:
IE provides specialized reports as requested. Requests should be submitted on the data request form. Requests should specify the information requested: course(s), academic year(s), and quarter(s). They should also include specific format requested (i.e., average scores, percentages, etc). All report requests must be signed or approved by the respective Dean or Associate Dean. The following form should be utilized.
31
Survey Data Request Form
The below information must be included in the email request OR a scanned copy of the
completed form must be sent to the Surveys Team at: [email protected]
I. Description of Request including course number(s): _________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
II. Timeframe of Data Requested. Please include a date range if applicable.
AY ____
Q1 ___ Q3 ___
Q2 ___ Q4 ___
III. Numerical Data and/or Comments Requested?
Data ___
Comments ___
IV. If Numerical Data is requested, please select the way you would like it presented.
Percentages ___
Numbers ___
V. Requestor Name ___________________________
VI. Approval for release of data. Signature of Dean or Associate Dean required.
_________________________
VII. Date needed ______________________
Please allow a minimum of a week to complete data requests. Someone from the Surveys
team will be in touch if they require any clarification regarding the request
All requests and any questions should be emailed to [email protected]
IE received date ______________________
32
Appendix D: Using CoursEval to Access Survey Results NIU uses the CoursEval survey tool for online survey processing.
Viewing Your CoursEval Results:
1. Log into the NIU Survey Site (mycourseval) at:
https://p3.courseval.net/etw/ets/et.asp?nxappid=HW1&nxmid=start&li=t
2. Enter your username and password and click Sign In. The survey site will open the “mycourseval” page.
3. Select the Reports icon from the mycourseval menu.
o Click on Recent Reports to see your course survey results.
o Click on Evaluation Reports or Individual Reports to see historical reports.
4. Select the course results you would like to view from the courses listed and use the options available on the screen. The options vary depending on which ‘Reports’ menu option you choose to view or print.
Viewing Departments, Degree, Program, or Center Results: (Deans and Directors):
1. Log into the NIU Survey Site (mycourseval).
2. Select Admin from the mycourseval menu.
3. Choose the course results you would like to view from the programs listed under Department.
4. Click the # Reports for that Department.
5. Select the course results you would like to view and click on the Report icon to the right of the row.
NOTE: If a null symbol appears on top of the Report icon, there are no results available.
Viewing Historical Program Course Results:
1. Click Reports on the menu at the top of the screen.
2. Click Evaluation Reports.
3. Use the menu filters at the top of the screen to locate desired reports.
NOTE: Academic Year 2015 (and half of Academic Year 2014) survey results are not in CoursEval. Please contact [email protected] to have these surveys e-mailed to you.
33
Printing Survey Results:
From the IE browser:
a. Click on View Report to open the .pdf.
b. Right click anywhere on the .pdf and select Print from the list of options.
From the Mozilla Firefox browser:
a. Click on View Report to open the .pdf.
b. Click on Open Menu icon located at the top far right of the screen far right (This icon looks square with 3 lines inside of it).
c. Click Print from the list of options.
Things to Know about Student Surveys
Survey responses are anonymous. The Survey Management Team can view who has or has not completed a course survey, but NOT individual student responses. Composite results are reported to faculty for each course. Course results are also aggregated at the program level and presented to NIU Leadership.
Student survey invitations are sent to their personal (or unclassified) e-mail address provided during registration and admission. Students must contact [email protected] if they do not receive their survey invitation(s) or would prefer the Survey Management Team to use a different e-mail address than the one provided during registration.
End-of-Course surveys are launched around the conclusion of the course. Students are required to complete a survey for each course in which he/she is enrolled. Failure to complete surveys may result in the withholding of all grades by University Deans.
End-of-Program surveys are launched at the beginning of the summer quarter and close prior to graduation. All degree-seeking students are invited to participate in the survey regardless of graduation year. Survey participation is mandatory in the academic year in which the student is graduating. Survey completion is required before a student/graduate can check out of the University.
President's Lecture Series (PLS) attendance is required for full-time students. Students are strongly encouraged to take the PLS Survey even if they have not attended the lecture. We understand there are various reasons why a student may not attend a lecture. There is an indicator at the top of the survey students can check to indicate they did not attend the lecture. Checking this indicator and submitting the survey removes the survey from the student’s survey queue.
34
Appendix E: Evaluation vs. Assessment
Evaluation: The process of making a judgment about the quality of student work for the purposes of
determining grades and identifying level of performance. Evaluation is part of the assessment process, by
systematically gathering, analyzing and interpreting the data/evidence.
Assessment: An ongoing process aimed at understanding and improving student learning. Assessment
helps faculty understand how well their students understand course/program topics, lesson and
outcomes.
The Assessment Process:
Steps involved in the assessment process include:
Establishing students learning outcomes for the course
Systematically gathering, analyzing, and interpreting evidence on how well student’s learning
matches:
o Faculty expectations for what students will learn, and
o Stated expectations/learning outcomes
Faculty members should use this evidence of student learning to:
o Provide feedback to students about their learning
o Adjust teaching methods to ensure greater student learning
Examples of evaluating student learning against student learning outcomes:
Exams
Quizzes
Papers
Homework Assignments
In-class Activities
Class Discussion
Briefs
Examples of assessment activities for assessing expected learning outcomes.
Non-graded quizzes
Reaction papers
Application papers
o What is the most important concept from this class?
o Identify their questions related to lesson
End of course surveys
35
Appendix F: Rubrics
Rubrics are a tool for assessment of performance. They provide a scoring guide used to evaluate the
quality of students’ work and can be used for a variety of assignments. Rubrics should contain evaluation
criteria, definitions for those criteria, and a scoring strategy. A rubric can help define expectations of
quality around a task and help ensure consistent criteria for grading. They can also save time in grading.
Scoring rubrics include dimensions on which performance is rated, such as:
Criteria – traits that serve as the basis for judging student response
Descriptors – define or clarify the meaning of each criteria
Levels – scale of values to rate each criteria
Standards – spells out performance required for each level
Rubric Examples
Assignment Rubric
Course Content Exceptional
4
90-100%
Good
3
80-89%
Fair/Adequate
2
70-79%
Limited
1
60-69%
Poor/Unacceptable
0
0-59%
Points
Availa
ble
Points
Awarded
Attention to instructions
and
assignment
Follows and exceeds expectations noted in
instructions
Follows instructions
Follows some but not all instructions
Gestures toward instructions but
demonstrates little
comprehension or
competency
Disregards instructions
Creative
Thought/
Problem-
Solving
Demonstrates a high
degree of originality,
insight, and/or
problem-solving skill
Shows some
originality,
insight, and/or
problem-solving
skill
Shows minimally
acceptable
originality, insight,
and/or problem-
solving skill
Demonstrates very
little creative
thought or insight;
consists mostly of
second-hand ideas
Shows no
original thought;
all second-hand
ideas
Thesis
statement
Thesis statement is
specific, significant,
arguable, and easy to
find
Thesis statement
arguable and
easy to find
Thesis statement
minimally
acceptable but
somewhat general
or obvious
Thesis statement
simplistic or
incomprehensible
No thesis
statement
Development
and Support
Thoroughly and
insightfully explores,
explains, and supports each idea
Develops and
supports key
points
Inconsistently
develops and
supports ideas
Inadequately or
ineffectively
explains and defends ideas
Does not make a
meaningful
attempt to explain or
support ideas
Source use Demonstrates
comprehension of all
source material; integrated sources
appropriately and
effectively
Most source
material is
appropriate, but some is not fully
explained or
integrated into
the paper
Source material is
not consistently
appropriate or integrate into the
paper
Source material is
rarely appropriate
and/or rarely integrated into the
paper
No use of source
material
Clarity Difficult material is
made clear and
presented in academic
language
Most ideas are
presented
clearly, but
sometimes too
simplistically
Wordy; some points
require rereading to
understand fully
Unclear and
difficult to
understand
Largely
incomprehensible
36
Thesis Rubric
1 –Did not meet requirements 2-Substandard 3-Standard 4 - Proficient
Not Meeting
Requirements
Substandard Standard Proficient Maximum
Possible
Points
Research
question
Research question or
problem that is trivial,
weak, unoriginal, not
related to the context of
the IC, intelligence or
national security issues.
Research question that is
not significant, is often
highly derivative, is only
tangentially related to
the context of the IC,
intelligence or national
security issues; is narrow
in scope
Research question or
problem that tends to be
small and traditional; is
the next step in a
research program (good
normal science), is
clearly related the
context of the IC,
intelligence or national
security issues
Novel question or
addresses an important
question or problem;
clearly states the
problem and why it is
important. Question is
perfectly aligned with
the context of the IC,
intelligence or national
security issues
4
Thesis
Development
Lacks originality,
insight, thoughtfulness
and clarity; often loses
coherence; may ramble;
has a weak, inconsistent,
self-contradictory,
unconvincing, or invalid
argument. Is not
structured around a
research question.
Workmanlike, and
generally coherent;
rarely insightful;
argument may be
coherent or
comprehensive but not
both. Is only somewhat
structured around a
research question
Clear, coherent, concise,
creative, thoughtful;
sometimes insightful;
has a comprehensive and
coherent argument. Is
structured around a
research question.
Original insightful,
persuasive,
sophisticated, and
thoughtful; exhibits
mature, independent
thinking; argument is
focused, logical,
rigorous, and sustained;
connects components in
a seamless way. Is
structured around a
research question.
4
Research
Methodology
Relies on inappropriate
or incorrect methods;
data are flawed, wrong,
or false
Uses standard methods;
data are sometimes
incomplete
Demonstrates technical
competence; uses
appropriate (standard)
theory, methods, and
techniques;
Original, unique, or
otherwise highly
effective research
design; uses or develops
new tools, methods, or
approaches; has rich data
from multiple sources
4
Data
Information is presented
but it is mostly not
related to answering the
research question or
there is no data to back
up the research question
answer
Not enough evidence has
been provided to answer
the research question.
Information within
thesis is only partially
related to the research
question.
Has sufficient
information to support
the main assertion of the
research question, but
contains some
extraneous material or
leaves some potential
questions unexplored.
Has a great deal of
information from
concrete or statistically
valid sources. Research
question is fully
answered, with no
unexplored issues or
connections. All of
material is related to
research question.
4
37
Points awarded for the research proposal out of 28 points available: ________
Analytic,
Critical, &
Creative
Thinking
Wrong, inappropriate,
incoherent, or confused
analysis; includes results
that are obvious, already
know, unexplained, or
misinterpreted; has
unsupported or
exaggerated
interpretation. Exhibits
little in the way of
critical or creative
thinking.
Unsophisticated
analysis; does not
explore all possibilities;
misses connections; has
predictable results.
Exhibits gaps in critical
thinking and little
creativity.
Obtains solid, expected
results or answers;
misses opportunities to
completely explore
interesting issues and
connections. Exhibits
solid critical thinking,
and/or some degree of
creativity. Multiple ideas
are at least somewhat
synthesized.
Comprehensive analysis,
complete, sophisticated
(findings based on
analysis and follows
logically from research,
thesis includes
contradictory evidence);
results are significant;
conclusion ties the work
together. Exhibits solid
critical thinking and
demonstrates creativity.
Synthesizes multiple
concepts and ideas.
4
Theory
Knowledge
Does not understand or
misses relevant
literature; does not
understand theory well;
theory is missing or
wrong; does not
understand basic
concepts, processes, or
conventions of the
discipline
Displays a narrow
understanding of the
field; reviews literature
adequately; knows the
literature but is not
critical of it or does not
discuss what is
important; demonstrates
understanding of theory
at a simple level, and
theory is minimally to
competently applied to
the problem
Shows understanding
and mastery of the
subject matter.
Displays a deep
understanding of a
massive amount of
complicated literature;
exhibits command and
authority over the
material; is theoretically
sophisticated and shows
a deep understanding of
theory. Author
challenges the literature.
4
Impact
The outcome or further
applications of the
research is of no interest
to intelligence
community; does not
contribute to the field
The outcome or further
applications of the
research may be of
interest to a narrow
segment of the
community; makes a
small contribution to the
field
The outcome or further
applications of the
research are of interest to
a moderate segment of
the community; unlikely
to change the way
people think; makes a
modest contribution to
the field but does not
open it up, might
moderately change the
behavior or attitudes of
professionals in the
community
The outcome or further
applications of the
research are of interest to
a larger community and
changes the way people
think; pushes the
discipline’s boundaries
and opens new areas for
research; can
extensively change the
behavior or attitudes of
professionals in the
community
4
38
Writing Rubric
1 –Did not meet requirements 2-Substandard 3-Standard 4 - Proficient Not Meeting
Requirements
Substandard Standard Proficient Maximum
Possible
Points
Style Content not appropriate for
audience
Content is inappropriate
for audience
Content is appropriate for
audience
Content is appropriate for
audience
4
Style is inappropriate for
audience
Style is inappropriate for
audience
Style is appropriate for
audience
Style is appropriate for
audience
Content does not satisfy
assignment
requirements/purpose
Some content irrelevant to
assignment
requirements/purpose
Content meets assignment
requirements/purpose
Content meets assignment
requirements/purpose
No theory/perspective
utilized
Does not acknowledge
theory/perspective that
shapes argument or
analysis
Articulates some elements
of their theory/perspective
Addresses how their
theory/perspective
influences their analysis or
argument
Uses inappropriate
language for audience
Uses some inappropriate
language for audience
Appropriate language for
audience
Appropriate language for
audience
Structure and
Organization
Has no thesis statement Weak thesis statement Effective thesis statement Effective thesis statement 4
No organization around
thesis statement
Some organization around
thesis statement
Mostly organized around
thesis statement
Fully organized around
thesis statement
No organization around
thesis statement
Paragraphs not fully
developed, coherent
Paragraphs mostly well-
developed, coherent
Fully developed, coherent
paragraphs
Paragraphs not developed,
incoherent
Minimal connection
between ideas/concepts
within paragraphs
Minor problems with flow
of ideas/concepts within
paragraphs
Organized into logical
flow of ideas/concepts
within paragraphs
Lack of conclusion/ not
consistent w/ thesis/body
Minimal connection
between thesis/body and
conclusion
Meaningful conclusion Meaningful conclusion
No section divisions Appropriate section
divisions
Appropriate organization
within sections
Appropriate organization
within sections
Inappropriate content in
sections
Somewhat jumbled content
in sections
Appropriate content in
each section
Appropriate content in
each section
No discussion/no
conclusion
Undeveloped
discussion/conclusion
Somewhat undeveloped
discussion / conclusion
Developed discussion and
conclusion
Content,
Argument,
and Evidence
Thesis statement not
backed by coherent
arguments
Thesis statement backed
by inadequate arguments
Thesis statement backed
by adequate arguments
Thesis statement backed
by ample and effective
arguments
4
Argument(s) contain(s) no
evidence
Arguments contain
inadequate sources of
evidence
Arguments contain
adequate sources of
evidence
Arguments contain ample
and effective evidence.
No/inadequate number of
citations
Sources lack
credibility/relevance
Sources are mostly
credible and relevant
Sources are credible and
relevant
No support/support is
irrelevant
Support not relevant to
topic
Support is mostly fully
integrated
Support is fully integrated
No/inadequate number of
citations
Improper format for
citations
Minor problems with
format for citations
Consistent format for
citations
Images/Maps not related to
the paper
Images/Maps not
integrated well and do not
add to the paper
Images/Maps integrated
well and add to the paper
Images/Maps integrated
well and add to the paper
Mechanics
Writing errors make
effective communication
impossible
(over 19 writing or
grammar errors)
Writing errors interfere
with effective
communication
(13 and 18 writing or
grammar errors)
Some writing errors – do
not interfere with effective
communication
(7 and 12 writing or
grammar errors)
Very few errors/no errors
in punctuation, grammar,
spelling or usage (between
1 and 6 writing or
grammar errors)
4
Writing unclear and
imprecise
Writing unclear and
imprecise
Writing is generally clear
and precise
Writing is clear and
concise
Writing ineffective Writing ineffective Writing is mostly effective Writing is skillfully and
effectively crafted
Does not meet
requirements for format
and length
Writing mostly meets
assignment requirements
for format and length
Meets assignment
requirements for format
and length
Meets assignment
requirements for format
and length
39
Appendix G: Faculty Peer-to-Peer Review Form
National Intelligence University
Peer Review of Classroom Instruction
Observations of teaching STYLE/DELIVERY of content and interaction with students:
Observations of STRENGTHS and best practices of the instructor:
SUGGESTIONS for the instructor and/or the seminar:
Please send a copy of the completed form to the reviewed instructor and to Institutional Effectiveness.
NIU Form MCE-2 Approved for use: 20151117
INSTRUCTOR: Reviewer:
Course:
Date: Time:
40
Appendix H: Course Syllabus Template
NATIONAL INTELLIGENCE UNIVERSITY
SYLLABUS TEMPLATE
I. Basic Information:
Course: [Course prefix, catalog number, and title]
Quarter/Year: [Quarter, AY]
Instructor: [Name]
Contact: [Office, e-mail, phone]
Office Hours: [scheduled + by appointment?]
Class Time/Location: [Day, time, classroom]
Textbook: [Title, Author, Edition]
II. Course Description/Contribution to Mission:
Course Description: (must be consistent with the current NIU catalog)
Contribution to Mission: (must be consistent with the current NIU catalog)
III. Course Goals and Learning Outcomes:
Goals: (must be consistent with the current NIU catalog)
Primary Learning Outcomes: (must be consistent with the current NIU catalog)
Joint Learning Outcomes: (must be consistent with the OPMEP Instruction)
41
IV. Course Format and Procedures:
Class attendance and participation:
You will meet the outcomes listed above through a combination of the following activities in this course:
o Attend
o Complete
o Participate
Late Work:
V. Course Requirements:
Course readings:
Course materials?
Use of the course Blackboard or other web site?
Resources?
VI. Evaluation Procedures/Grading Criteria:
VII. Course Schedule/Outline/Calendar of Events:
VIII. Academic Integrity (optional):
Each student in this course is expected to abide by the NIU Code of Academic Integrity. Any work
submitted by a student in this course for academic credit will be the student's own work. [Optional: For
this course, collaboration is allowed in the following instances: list instances.]
You are encouraged to study together and to discuss information and concepts covered in lecture and
the sections with other students.
You are expected to credit properly and accurately the source of materials directly cited or indirectly
used (i.e., paraphrased) in any oral or written work. All students’ work shall be their own, unless
otherwise properly noted.
You may not use entire papers or substantive selections of a paper from one course to complete work
for another course or courses.
42
A grade of F will normally be assigned for any work proven to be undertaken or performed in violation of
academic integrity. All instances of alleged violations of academic integrity will be handled in accordance
with published NIU policies.
IX. Attendance (optional):
Students are expected to attend all scheduled class sessions. Students missing more than one session
face, at the discretion of the faculty member, penalties ranging from the lowering of the final grade to
failure in the course. A student who misses three or more sessions and does not withdraw faces removal
from the course and a failing grade.
IX. End of Course Survey (optional but recommended):
Students are expected to complete a course survey near the end of this course. Course surveys are an
essential component of the universities educational assessment process. Failure to complete your end-
of-course survey may result in the withholding of all student grades by the Dean. Follow university
instruction on accessing the university survey site (myCoursEval) and completing course surveys.
Contact [email protected] if you have any questions regarding course surveys.
X. Inclusivity Statement (optional):
We understand that our faculty and students represent a rich variety of backgrounds and perspectives.
NIU is committed to providing an atmosphere for learning that respects diversity. While working
together to build this community we ask all members to:
share their unique experiences, values and beliefs
appreciate the opportunity that we have to learn from each other in this community
value each other’s opinions and communicate in a respectful manner
keep confidential discussions that the community has of a personal (or professional) nature
use this opportunity together to discuss ways in which we can create an inclusive environment
in this course and across the Intelligence and National Security communities
43
Appendix I: Outcomes Assessment Form
The Outcomes Assessment Form was developed to capture information about the type of meeting,
purpose of the meeting, alignment with strategic, MSCHE, and JPME goals and outcomes, evidence,
discussion and analysis conducted, and then the decisions, outcomes or recommendations that come out
of that meeting. There is also a space for any follow-up activities that needs to be conducted.
This form is recommended to be used for meetings of groups evaluating curriculum, goals, outcomes, and
other significant events impacting the way NIU conducts business.
By utilizing this form, the documentation of decisions and changes will be clearly spelled out, and the
evidence used to come to that conclusion.
44
Outcomes Assessment Form
Date:
Assessment Unit/ Team / Working Group
Project/Assessment
Purpose / Objective / Goal:
Alignment: (MSCHE & JPME
Standards/Strategic Goals & Obj.)
Analysis
Data / Evidence:
Discussion / Analysis:
Results
Decisions / Outcomes / Recommendations:
Next Steps / Implementation Plan (Post-Review Follow-up):
45
Appendix J: Pre-Public Release Review Process
There are specific procedures for assuring compliance with the agency policy for Prepublication Review of Information Prepared for Public Release. These procedures identify two key processes: those of NIU and those of the Office of Corporate Communications (OCC).
References: DIAI 5400.005, Prepublication Review of Information Prepared for Public Release, 19 November 2013.
Definitions:
OFFICIAL: Material prepared and being released as part of one’s official duties as a DIA employee or contractor.
NON-OFFICIAL or UNOFFICIAL: Material prepared by a DIA employee or contractor as a private individual and who is not acting in an official capacity for DIA, DoD, or the U.S. Government.
Application:
Pre-public release review applies to all DIA/NIU personnel, civilian, military, and contractors, regardless of rank.
All unclassified products that are intended for public release, whether the information is intended to be an official or non-official product, must be submitted for prepublication review. This includes publications, speeches, library contributions, web blogs, etc.
If you’ve been asked to speak to the public (conferences, panels, alumni events, etc.) about DIA or your mission area, you are required to have OCC approval prior to accepting the invitation.
Responsibilities:
Directorates and Departments must submit all unclassified official and unofficial information and products prepared and intended for public release to OCC Prepublication Review after review and approval by component’s management chain of command and security.
Individuals are responsible for filling out the Prepublication Review Form and obtaining permission of directorates and special offices which have equities in the material.
o Official products intended for public release must be coordinated with those directorates and special offices which have equities in the material prior to submitting the product to NIU leadership for Prepublication Review.
o DIA personnel who are requesting review of material prepared in a non-official capacity shall include a statement that the material submitted for review is derived from unclassified
46
information and is, to the best of the submitter’s knowledge, unclassified and appropriate for public disclosure.
o Draft material prepared in a non-official capacity must be submitted for review at each stage of development prior to disclosing such information to anyone, such as a publisher, editor, co-author, or other member of the public, who does not have the requisite clearance and “need-to-know.”
All employees must ensure that OCC Prepublication Review has been completed prior to disclosing such information to anyone who is not authorized by DIA to have access to the material.
Process:
Individual requesting public release of information, must submit both the document which you are requesting release and the completed Prepublication Review Request form on JWICS. The form can be found at:
https://www.dia.ic.gov/homepage/occ/policy/prepub_review_form_Sep2015.pdf
Contractors must also clear their submissions through the COR.
Prepublication Review Requests will be submitted via JWICS to the GG-15 Division Chief (or Higher) chain of command for processing and review.
o Requests from faculty or students will go to the Dean of the appropriate School or thesis advisor’s School for NIU review.
o Requests from research faulty will go to the Director of the Center for Strategic Intelligence Research.
o Accreditation, Assessment or IE requests will be submitted to the Director for IE.
o Other requests will go to the Director of Office of Research.
The Division Chief will determine if the information is Official or Unofficial, and will submit the proposed publication to Security for review and Foreign Disclosure review (FDO) as appropriate.
Upon approval of security and FDO (if required), the Division Chief will submit the request to OCC for processing.
Academic Issues:
In the interest of academic freedom and the advancement of national defense-related concepts in the DoD academic environment, students and faculty members of NIU may prepare academic papers and manuscripts for open publication. They may express their views in such materials as long as those views do not disclose classified or OPSEC critical information or jeopardize DoD interests and the author accurately portrays official policy, even if the author takes issue with that policy.
Papers or other material prepared in response to academic requirements do not have to be submitted for review when they are not intended for release outside the academic institution.
47
Papers or other material must be submitted for review and clearance if it is intended for public release or made available in libraries to which the public has access.
DIA personnel teaching at a non-DoD schools must submit their written instructional material for review and clearance if it includes classified or “for official use only” information. It is incumbent on the individual to ensure any classroom discussion or anecdotes of one’s experiences in or knowledge of the Intelligence Community or DoD do not reveal classified or otherwise sensitive information.