Assessing Controller Workload

2011 AGA Operations Conference2011 AGA Operations ConferenceCharles Alday

AGA Program DescriptionAGA Program Description

• The CRM rule requires that a companyThe CRM rule requires that a company monitor the general level of controller activity on an annual basis. This presentation will pdescribe a method for assessing tasks and mental workload.

Rule, FAQs, Inspection ProtocolsRule, FAQs, Inspection Protocols

• “monitor the content and volume of generalmonitor the content and volume of general activity being directed to and required of each controller…that will assure controllers have sufficient time to analyze and react to incoming alarms.” g

• PHMSA representatives have remarked that “general activity” refers to all tasks for which a controller is responsible.

AGA White PapersAGA White Papers

• The SCADA system provides alarms whichThe SCADA system provides alarms, which are preset and represent deviations outside of expected parameters. p p

• The alarm screens are monitored constantly, and critical alarms require immediate and qcontinuous actions.

• Non-critical alarms require action as qdetermined by the controller.

AGA White PapersAGA White Papers• Review the content and volume of activity

b i di t d t h G C t ll tbeing directed to each Gas Controller to assure sufficient opportunity to analyze and react to incoming alarmsreact to incoming alarms

• Volume of activity being directed to Controllers This can include:Controllers. This can include:– Phone calls– ReportsReports– Visitors– Load forecasting and supply

FAQs and Inspection ProtocolsFAQs and Inspection Protocols

• These draft documents contain more• These draft documents contain more detailed guidance that are not

d t b i d BUT????supposed to be required… BUT????

Rule, FAQs, Inspection ProtocolsRule, FAQs, Inspection Protocols• “General activity” means any activity that is

required of the controller This includes but isrequired of the controller. This includes, but is not limited to, pipeline operations, handling SCADA alarms, conducting shift change, , g g ,greeting and responding to visitors, administrative tasks, impromptu requests,

l h ll f h i i i htelephone calls, faxes, or other activities such as monitoring weather and news reports, checking security and video surveillance systems usingsecurity and video surveillance systems, using the internet, and interacting with colleagues, supervisors, and managers.supervisors, and managers.

Rule, FAQs, Inspection ProtocolsRule, FAQs, Inspection Protocols• Process must have a sufficient degree of

formality and documentation.• Describe the level of activity for each console,

i l di (i f t l ith lti lincluding (in cases of control rooms with multiple consoles) which console has the most activity and which has the leastand which has the least.

Do These “Fit” Your Company?Do These Fit Your Company?

“My Head’s About to Come Off”My Head s About to Come Off

What is Workload?What is Workload?

Workload isPhysical & =

Demands of all Tasks & y

Mental Requirements

Assessing Task WorkloadAssessing Task Workload1. Identify controller tasks during normal

operations.2. Assess the “difficulty” of tasks.33. Count the number of occurrences, over a

specific amount of time.a. Use data that is available in your system.

4. Measure the amount of time per task.5 P j t th t ti th t5. Project that time over a month, a quarter, a

year.

Assessing Task WorkloadAssessing Task Workload• Information Access (how difficult is it to get

information to perform task?)information to perform task?)• Mental Loading (how great is the “mental

workload” to complete the task?)p )• Action Loading (how complex are the physical

actions required?)C i ti L di (h l d• Communication Loading (how complex and demanding are communication requirements?)

• Stress (how great is the psychological stressStress (how great is the psychological stress due to time demands, abnormal conditions, or exposure to hazardous conditions?)

“Ten Most Common Tasks”Ten Most Common Tasks1. Line Operations2. Line Adjustments3. Log Sheet/Schedule4 Sampling Calibrate Proving Testing4. Sampling, Calibrate, Proving, Testing5. Abnormal Operations6. Monitoring6. Monitoring7. Phone Calls8. Communications9. Administrative Tasks10. Miscellaneous

Monitoring?Monitoring?

• Let me tell you a storyLet me tell you a story– “These controllers are spending 49% of their time

just sitting there watching screens.”• “Therefore…”

Monitoring?Monitoring?

• “Monitoring for the controllers is a continuousMonitoring for the controllers is a continuous and cognitively demanding task performed while also performing mathematical p gcalculations, paperwork, phone calls, and deliveries.”

• This does not include the abnormal conditions to which the controller must immediately attend.

Mental WorkloadMental Workload

• Some of you won’t like this methodSome of you won t like this method.• It’s not purely quantitative.• It’s subjective!It s subjective!• It involves asking individual controllers to rate

their workloadtheir workload.• Then those ratings are analyzed and

compiled quantitativelycompiled quantitatively.• This is combined with the task workload.

Mental WorkloadMental WorkloadThe NASA TLX is a multi-dimensional rating

d th t id ll kl dprocedure that provides an overall workload score based on a weighted average of ratings on six subscales:on six subscales:– Mental demand– Physical demandPhysical demand– Temporal demand– Effort– Performance– Frustration level

NASA TLX DimensionsNASA TLX Dimensions

• Mental Demand: The amount of mentalMental Demand: The amount of mental and/or perceptual activity (e.g., thinking, deciding, calculating, remembering, looking, g, g, g, g,searching, etc.) that is required by the task or tasks. – Were the tasks easy or demanding, simple or

complex, precise or forgiving?

NASA TLX DimensionsNASA TLX Dimensions

• Physical Demand: The amount of physicalPhysical Demand: The amount of physical activity that is required by the task or tasks (e.g., writing, punching the calculator buttons, ( g , g, p g ,walking, moving chairs, moving the mouse, pushing, pulling, turning, controlling, p g p g g gactivating, etc.). – Was the task easy or demanding, slack or

strenuous?

NASA TLX DimensionsNASA TLX Dimensions

• Time Demand: The amount of time pressureTime Demand: The amount of time pressure due to the rate or pace at which the tasks happened. pp– Was the pace slow and leisurely or rapid and

frantic? – How pushed for time was the controller? – Was the controller “under the gun”?

NASA TLX DimensionsNASA TLX Dimensions

• Effort: How hard the controller had to workEffort: How hard the controller had to work (mentally and physically) to accomplish their level of performance.p– Did the controller have to expend a great deal of

effort to complete the task?

NASA TLX DimensionsNASA TLX Dimensions

• Performance: How well the controllerPerformance: How well the controller performed the task. – How satisfied was the controller was with his or

her performance?

NASA TLX DimensionsNASA TLX Dimensions

• Frustration: How insecure or secureFrustration: How insecure or secure, discouraged or gratified, irritated or content, stressed or relaxed, and annoyed or , ycomplacent the controller felt.– Was the controller “on his last nerve”? – How worried was she?

Example of ResultsExample of ResultsDimension System 1 System 2Performance 285 233

Time Demand 254 212Time Demand 254 212

Mental Demand 231 245

Eff 129 1Effort 129 175

Frustration 70 104

Physical Demand 35 72

Workload AssessmentWorkload Assessment• Phase 1: The goal is to understand workload in

the normal control and monitoring of the pipelinethe normal control and monitoring of the pipeline and refinery processes. Workload will be assessed in the actual operating environment ofassessed in the actual operating environment of the control center during all shifts.

• Task analysis will be conducted first in order to yget a full list of the controller tasks. The task analysis would be based on these procedures

d d bl d d id d b thand records, assembled and provided by the Company

Workload AssessmentWorkload Assessment• Phase 2: The goal will be to determine the five

most common tasks from Phase 1 and introducemost common tasks from Phase 1 and introduce abnormal and emergency events into the system and measure controller reaction time. This willand measure controller reaction time. This will be conducted in a controlled and planned environment of the training simulator where infrequent abnormal events can be programmed to occur. If a simulator is not available, tabletop

i ill b d l d f b l dscenarios will be developed from abnormal and emergency procedures.

Workload AssessmentWorkload Assessment• Another consideration that affects workload is

the workspace and environment and the number of hours worked. The method for this study would include an assessment with appropriatewould include an assessment with appropriate evaluation methods of environmental, ergonomic and spatial factors These factorsergonomic, and spatial factors. These factors affect controller workload, situation awareness, vigilance, and attention. g

AGA Program DescriptionAGA Program Description

• The CRM rule requires that a companyThe CRM rule requires that a company monitor the general level of controller activity on an annual basis. This presentation will pdescribe a method for assessing tasks and mental workload.