nerc trainingnerc training task analysis - emmos · nerc trainingnerc training task analysis ems...

NERC TrainingNERC TrainingTask Analysis

EMS User Group MeetingSeptember 21 2010September 21, 2010

Chicago, IL

Presented By

Robin PodmoreIncSys, Issaquah, WA

(with key slides from John Taylor, NERC)

Outline• PER-005 Requirements• Task Analysis and Learning Objectives• Cognitive Task Analysis• Task Statements• Task Analysis Methods• Task Performance Objectives

2009 Copyright Incremental Systems Corporation

Task Analysis ADDIEyPER-005 Tasks“ C“R1.1. Each Reliability Coordinator, Balancing

Authority and Transmission Operator shall create a list of BES company specific reliabilitycreate a list of BES company-specific reliability-related tasks performed by its System Operators.”Operators.

“R1.2. Each Reliability Coordinator, Balancing Authority and Transmission Operator shall y pdesign and develop learning objectives and training materials based on the task list created in R1.1.”

Task Analysis ADDIEyPER_005 Requirements R1.1 and R1.2Both R1.1 and R1.2 can be addressed at the same timeDevelop (modify) task statements in a format that gives

you both task statements and learning objectives for training

No need to do both as separate projects and allocation ofNo need to do both as separate projects and allocation of resources

Follow a suggested format for task statements and you will have learning objectives when you finish the task statements

Task Analysis Applications

• Instructional Design• Designing Human Computer Interactions


From Task Analysis to Learning Objectives

ADDIE

Objectives

Cognitive Task Analysisg y• What are operators thinking about?• What are they paying attention to?• What strategies are used to make decisions?• What are they trying to accomplish?• What do they know about how a process works?


Data Collection Methods

• Interviews• Self-Reports• Job Shadowing• Questionnaire• Incident Analysis• Observation of Simulator Drills


Cognitive Task AnalysisPrimary Aspects

• Knowledge elicitation• Data analysis• Knowledge representation


Task Statements ADDIE

What format should be used for developing taskWhat format should be used for developing task statements?1. When is it done (Trigger, event, or condition)?( gg , , )2. Who takes action?3. What action is taken?4. How do we know the task is complete (criteria for

performance)?

Task Statements ADDIE

Given (initiating event, condition), person taking action, do this (action, response), this well (standard of performance, quality/effect of output).

E t/C ditiTask Performer

Given a transmission system overload, the li bilit di t ill l ti

Event/Condition Action

reliability coordinator will use real-time contingency analysis study results to determine the impact of removing a line from service on thethe impact of removing a line from service on the loading of other transmission lines

Outcome, criteria for successful performance

Task Statements and Terminal Objectives

ADDIE

ObjectivesUsually more than one task in a procedure.When building a task list for a position, structure

task statements as shown above.Each task statement is a Terminal Objective (high

level learning objective)Id tif th k l d kill biliti d d tIdentify the knowledge, skills, abilities needed to

perform task successfully (Enabling Objectives).Id tif t l f iliti i f ti t tIdentify tools, facilities, information, support, etc.

required to perform the task as required (to meet the criteria for successful performance)the criteria for successful performance)

Enabling Objectives ADDIEg jStatements of Knowledge, Skill, or Ability required to meet

Terminal ObjectiveTerminal ObjectiveTerminal Objective: Given a transmission system overload,

use real-time contingency analysis study results to determine the impact of removing a line from service on the loading of other transmission lines 1 Enabling Objective: Conduct a real-time contingency1. Enabling Objective: Conduct a real-time contingency

analysis in the EMS. (performance objective)2. Enabling Objective: Analyze contingency analysis results.

(k l d bj ti )(knowledge objective)3. Enabling Objective: Decide whether or not to remove a

line from service to relieve the overload. (knowledge objective)

Tasks and Learning ObjectivesADDIEg jProcedure Competency Training Components

(B h i )(Behavior)Task A Capability Terminal Objective

Step 1 K,S,A Enabling Objective 1Step 2 K,S,A Enabling Objective 2Step 3 K,S,A Enabling Objective 3

Tasks and Learning ObjectivesADDIEg jProcedureTask 1.Condition – When this happensActions – Do these thingsCriteria - Required outcome of actions

TerminalObjectiv

Action AAction B

e

K,S,AAction CEnabling

Objectives

, ,

K,S,A

s K,S,A

Analysis – Tasks From ProceduresADDIE

y

Usually more than one task in a procedure.When building a task list for a position, structure task

statements as shown above.Each task statement is a Terminal Objective (high levelEach task statement is a Terminal Objective (high level

learning objective)Each Terminal Objective should have several Enabling j g

Objectives under it.Enabling Objectives should indicate knowledge, skill or

ability to complete taskability to complete taskEnabling Objectives can be classified as Knowledge

Objectives or Performance Objectivesj j

Task Taxonomy ADDIEyPAY ATTENTION to the verb in task statements,

t i l bj ti d bli bj titerminal objectives, and enabling objectivesState the task in terms of correct taxonomy levelKnowledge objectives – Blooms TaxonomyPerformance objectives – Dave’s Taxonomy

Knowledge Taxonomy ADDIEg y

www.uvm.edu/~cdisorda/blooms.html

Bloom’s Knowledge TaxonomyADDIEg yEvaluation: appraise, argue, assess, attach, choose compare, defend

estimate judge predict rate core select support value evaluateestimate, judge, predict, rate, core, select, support, value, evaluate.Synthesis: arrange, assemble, collect, compose, construct, create, design,

develop, formulate, manage, organize, plan, prepare, propose, set up, write. A l i l i l l t t i t t iti iAnalysis: analyze, appraise, calculate, categorize, compare, contrast, criticize,

differentiate, discriminate, distinguish, examine, experiment, question, test. Application: apply, choose, demonstrate, dramatize, employ, illustrate,

interpret operate practice schedule sketch solve use writeinterpret, operate, practice, schedule, sketch, solve, use, write.Comprehension: classify, describe, discuss, explain, express, identify,

indicate, locate, recognize, report, restate, review, select, translate, Knowledge: arrange define duplicate label list memorize name orderKnowledge: arrange, define, duplicate, label, list, memorize, name, order,

recognize, relate, recall, repeat, reproduce state

Cognitive Composition ADDIE

Axiom

New Principle New ..

Principle Principle

Relationship

Concept Concept

D t D t D t D t

Concept

Relationship Relationship

Concept

Data Data Data DataRelationship Relationship

Psychomotor (Performance) Taxonomy ADDIE

y ( ) y

Performance Levels –Psychomotor Domain

ADDIE

Psychomotor Domain

Performance Levels – Psychomotor Domain

ADDIE

DomainOne Criteria for

Performance - Correctly

Two Criteria for Performance – Correctly and Fast

Task/Performance Objective ADDIE

PRECISION Level of a Performance Objective

Event/Condition Task Performer

Given a transmission system overload, the reliability

Event/Condition

coordinator will correctly run a real-time contingency analysis for the line to be removed

ActionO tcome criteria for s ccessf l

One criteria – correctly Psychomotor level of PRECISION

Outcome, criteria for successful performance – only one criteria

One criteria – correctly, Psychomotor level of PRECISION

Task/Performance Objective ADDIE

ARTICULATION Level of a Performance Objective


Given a transmission system overload, the reliability


coordinator will correctly run a real-time contingency analysis for the line to be removed

ithi 15 i t i dwithin a 15 minute window. Action

15 i t i d dCorrectly – one criteria 15 minute window – second criteria (time)

T it i tl d ithi 15 i tTwo criteria – correctly and within 15 minutes, Psychomotor level of ARTICULATION

ADDIE

DesignDesign

Training On Tasks ADDIE

gWhat tasks will the training address?Do the task descriptions contain:1. Performer – performs the task?2 T i h i h k f d h2. Trigger – when is the task performed or what event or

condition requires action?3 Action – what action is required by the performer?3. Action what action is required by the performer?4. Criteria – what outcome is required; when is the task

complete?Is the task performance criteria measurable or

observable?

Learning Objectives ADDIE

What are the learning objectives?A th l i bj ti th tAre the learning objectives on the same taxonomy

level as the related task performance criteria?What prerequisite knowledge skills abilities ofWhat prerequisite knowledge, skills, abilities of

participants are required?Are the learning objectives knowledge objectivesAre the learning objectives knowledge objectives,

performance objectives, or a mix of both?

Knowledge Objectives - Learning AssessmentADDIE

Cognitive Level – Bloom / Dave

Type Learning Assessment

Evaluation / Creating Judge Value/Effectiveness Of Solutions – Judging, Non-linear Process Developmentp

Synthesis / Problem Solving Decision Making – Critical Decision Making Assessment

Analysis / Abstract Rule Using Separate Concept Into ComponentAnalysis / Abstract Rule Using Separate Concept Into Component Parts – Apply Complicated Formula or Abstract Rule

Application / Concrete Rule Rule Using Apply Simple Formula OrApplication / Concrete Rule Using

Rule Using - Apply Simple Formula Or Rule

Comprehension / Classifying Essay Explaining ConceptKnowledge / Discriminated Recall

True/False, Multiple Choice, Matching, Listing, Completion

Performance Objectives – LearningAssessment

ADDIE

Psychomotor Taxonomy Dave

Type Learning AssessmentTaxonomy – Dave / SimpsonNaturalization / Adaptation -

Two or more skills combined, sequenced, and performed consistently and with ease The performance isAdaptation -

Originationconsistently and with ease. The performance is automatic with little physical or mental exertion.

Articulation / Complex Overt

Two or more skills combined, sequenced, and performed consistently (Do It right and quickly without help)Complex Overt

Response consistently (Do It right and quickly without help)

Precision / Mechanistic

Accuracy, proportion and exactness exist in the skill performance without the presence of the original source (Do It right without Help)

Manipulation / Guided Response

Guided via instruction to perform a skill.

Imitation / Set Observing and copying someone else

Learning ObjectivesADDIE

g j• Are the objectives and content on a level

suitable for the target audience?• Do the learning objectives contain the same

elements as the related task description?

Learning AssessmentADDIE

g• Is the learning assessment appropriate for the

content level?content level?• What is the type and content of the learning

assessment?assessment?• Will retesting be offered?• Are re tests significantly different from original• Are re-tests significantly different from original

test?• What is the pass level (based on job• What is the pass level (based on job

requirements)?

PrerequisitesADDIE

q• What knowledge, skills and abilities required for

t ?entry?• What is the targeted audience for the targeted

t k ?tasks?• Are trainees responsible and accountable for

tasks?tasks?• Does management support the training?

I ti f th t t di d di t d f• Is time of the target audience dedicated for training?

Structure ADDIE

• How will the training be structured in terms of t t d ?content and sequence?

• What are available sources for course content?• How will the training program be evaluated?• What resources will be needed to develop and

d li th t i i ?deliver the training?• What recordkeeping will be needed?

Structure ADDIE

• What is the task hierarchy (sequence and/or relative importance?relative importance?

• How will the training be structured in terms of content and sequence?content and sequence?

• What are available sources for course content?• What resources will be needed to develop and• What resources will be needed to develop and

deliver the training?• How will the training program be evaluated?• How will the training program be evaluated?

Design Outcomes ADDIE

• Entry behaviors – prerequisite knowledge, skills, abilitiesabilities

• Learning objectives – knowledge and performance objectives from task statementsperformance objectives from task statements

• Learning assessment – matched to the level of the learning objectivesthe learning objectives

• Learning steps (performance steps) – task hierarchye a c y

• Structure and sequence - program outline

DesignADDIE

gIdentify Learning Objectives

Bloom’s Taxonomy & RevisionLearning StrategiesLearning Strategies

Identify Instructional StrategyIdentify AssessmentsIdentify Assessments

Learning Objectives ADDIE

• Learning Objectives are the nucleus of training.• Learning Objectives provide a framework for g j p

the development of instruction. • Learning Objectives are contract between

student and teacher.student and teacher. • An objective is an explicit statement describing

a skill, knowledge or attitude that produce anaction that an operator is expected to performaction that an operator is expected to perform and includes a description of:• What the operator will be expected to do – in

behavioral termsbehavioral terms• The circumstances under which the operator will do

itHo ell the operator ill do it• How well the operator will do it

Unclear Objectives Revised Objectives

Figure 2: Examples of unclear and revised objectives.ADDIE

Students will know the dates of important events in U.S. History

Each student will recall the 10 major events in the Civil War

j j

History.

Students will know cases of mental disorders.

War.

Each student will classifyobserved or described

f t l di d

Students will understand the relevant and irrelevant

cases of mental disorders

Each student will distinguish between relevant and irrelevant

numbers in a mathematical word problem.

grelevant and irrelevant numbers in a mathematical word problem.

Students will know the best way to solve the word

Each student will judgewhich of the two methods is the best way to solve the y

problem.y

word problem.

NERC Generic Job List• Transmission• Generation• Interchange• Reliability Coordination and Markets


Transmission Duties• Control Center Emergencies• Forced Transmission Outage Response• Forced Transmission Outage Response• Knowledge and data maintenance• Logs, Reports, Notifications• Load Shedding and Restoration• Monitor Protective systems• Manage System Disturbances• Manage System Disturbances• Maintaining System Security• Monitor Telecommunications System• Planning and Prevention• Remedial Action Scheme Management


Voltage Control Tasksg• Coordinate operations of voltage control equipment with

interconnected utilities in accordance with operating agreements orinterconnected utilities in accordance with operating agreements or regional requirements

• Coordinate with generating plant owners/operators to adjust unit output voltage as required in accordance with operatingoutput voltage as required, in accordance with operating agreements

• Connect or remove reactive devices to stay ahead of system voltage during daily load cycleduring daily load cycle

• Monitor dynamic reactive reserves to ensure that they are adequate to maintain voltage stabilityM it th t t d il bilit f t lt l t• Monitor the status and availability of generator voltage regulators and respond as required to deficiencies


Voltage Control Tasks• Monitor the voltage and coordinate the reactive dispatch of

transmission facilities and the interconnections with neighboring systemssystems

• Monitor VAR flows to ensure that they are within acceptable limits on an actual and contingency basis and determine need for corrective actioncorrective action

• Monitor voltage at key buses and identify actual or potential violationsR d t t l ti lt d i ti i d• Respond to actual or contingency voltage deviations in accordance with company procedures/priorities and regional policies or guidelinesR t d i ti t t bl l l• Restore dynamic reactive reserves to acceptable levels as soon as possible after use

• Utilize transmission line removal as a voltage control tool only if system studies indicate that system reliability will not be degraded below acceptable levels


Learning Objectiveg jExample: It is noon on a hot summer week day. The Crawford –

Oakdale 1 & 2 lines have tripped out of service due brush fires.Oakdale 1 & 2 lines have tripped out of service due brush fires. Brush fires are also reported under the Baker to Amus line. Take the necessary preventative actions and prepare a remedial action plan to make the system secure. y


System Assessment - Cues• Monitor system voltages at following stations …… • Monitor dynamic reactive reserves at following generating stations …..• Monitor reactive support being obtained from neighboring systems at following tie-

lines …..• Monitor reactive support being delivered to neighboring systems at following tie lines

…..…..• Identify voltage profile under current system conditions• Identify voltage violations under contingent system conditions• Identify the worst single contingencyIdentify the worst single contingency • Monitor transmission flows in comparison with SIL levels• Monitor status of ULTCs including control mode and tap position• Ensure all lines are in-service where ever possible• Ensure all lines are in-service where ever possible.• Ensure all generator voltage regulators are in service• Ensure all Static VAR controllers are in service• Ensure taps are set to avoid circulating VAR flows• Ensure taps are set to avoid circulating VAR flows


Apply Correct Mental Models• MVAr production / consumption of transmission lines depending upon MW loading

level in comparison with surge impedance loading• MVAr response of capacitors as square of voltage• MVAr response of capacitors as square of voltage• MVAr charging characteristics of lines with different nominal voltages• MVAr charging characteristics of cables with different nominal voltages

Areas of the system that have weak voltage support• Areas of the system that have weak voltage support• Areas of system that have strong voltage support• Transmission paths that are prone to voltage collapse

T f h MVA f hi h id t l id l id t i d• Transformers push MVArs from high side to low side as low side taps are raised.• As low side taps are raised with strong high side bus, low side voltage will increase

and high side bus will remain constant.• As low side taps are raised with weak high side bus low side voltage may increase /• As low side taps are raised with weak high side bus, low side voltage may increase /

decrease and high side bus will also decrease.• Confidence to resist increasing transformer taps to improve low side voltage when

system is on verge of voltage collapse.


Apply Correct Mental Models• Generation re-dispatch options to reduce flows on paths prone to voltage collapse• Increased sharpness of the “knee of the curve” when capacitors are in service• Impact of sub-transmission and distribution regulators in raising taps and moving

VARs from high side to low side when system operator fails to get ahead of the voltage.

• Low impedance paths that are subject to high MVAr flows when regulated voltagesLow impedance paths that are subject to high MVAr flows when regulated voltages are not matched.

• Tendency for runaway transformer taps ULTCs are in automatic mode with weak high side buses.

• Parallel units regulating a common bus voltage• Tendency for capacitors to create extra high voltages when lines are out of service

causing extra weak buses.I t f t ti MW fl d lt• Impact of worst contingency on MW flows and voltages

• Circulating VAR flows in parallel transformers• Circulating VAR flows in transmission loops with transformers


Evaluate and Apply Correct Control ActionAction

• Coordinate with generating plant owners/operators to adjust unit output voltage as requiredrequired.

• Ensure profile of system voltages is maintained uniformly to avoid un-necessary VAR flows.

• Connect capacitive devices to stay ahead of system voltage during morning load p y y g g gpickup.

• Remove reactive devices that were used to absorb MVArs under light load conditions• Monitor dynamic reactive reserves at generating units to ensure that they are

adequate to maintain voltage stability• Monitor the status and availability of generator voltage regulators.• Ensure the MW flows are maintained within limits to prevent voltage collapse• Development of remedial action plan for handling harmful contingencies• Development of preventative action plan for handling harmful contingencies


nerc trainingnerc training task analysis - emmos · nerc trainingnerc training task analysis ems...

Documents