safire: situational awareness for firefighters firefighter outreach, testing, validation goal:...

SAFIRE: Situational Awareness for Firefighters

Firefighter Outreach, Testing, Validation

Goal: “Maintain critical contacts with the firefighter and first responder community in order to guide, test, and validate our technology development efforts.”


Firefighter Outreach, Testing, Validation

Strategies: Firefighter forums Joint exercises Distinguished speaker lectures Technology prototyping Data / Information sharing After Action Reviews Studies / Experiments


Firefighter Outreach, Testing, Validation Overview Joint Drills / Technology Testing Exercises - Impact Analysis

Live Burn & CO Sensing Study: OCFA, LA County Fire (23 FEB 09) HazMat, casualties, First Response drill and SAFIRE Deployment (16 SEP 08) HazMat drill (with multiple casualties) and SA Study (12 MAY 09) Tabletop exercise IC Usability Experiment (15 MAY 09)


Firefighter Outreach, Testing, Validation Overview CERT sponsored Firefighter forums

Quarterly meetings Roundtables, distinguished speakers, FF presentations

One-on-one discussions, meeting, visits LA County Fire: CO sensing

Joint technology development Technology Prototyping Feedback on SAFIRE technologies

SAFIRE: Situational Awareness for Firefighters 5

Technology Evaluation, Analysis, Validation Assessing effectiveness of developed technologies

in improving FF safety is key component of projectStage 1:

Technology Evaluations

Stage 2:

Impact Analyses

Stage 3:

Validation

Performance Metrics:

•Reduction in casualties•Improved response times•Better allocation of resources•Improved coordination

Evaluation Platforms:

•Simulation Models•Tabletop Exercises

Methods:

•Data from past incidents•Tabletop exercises with first-responders

Stage 1:

Technology Evaluations

Stage 2:

Impact Analyses

Stage 3:

Validation

Performance Metrics:

•Reduction in casualties•Improved response times•Better allocation of resources•Improved coordination

Evaluation Platforms:

•Simulation Models•Tabletop Exercises

Methods:

•Data from past incidents•Tabletop exercises with first-responders


Technology Evaluation and Analysis Responsphere Drills

First HazMat drill (17 JUL 08) Second HazMat drill (12 MAY 09)

Live Burn exercise (23 FEB 09) Immediate feedback on technology

After-action reviews Occur after each exercise

Impact Analysis - utilization experiment, tabletop exercise (15 MAY 09)

Reports on testing outcomes Data sharing

CO Data available for Jeff Burgess (Arizona)


Technology Validation: SAFIRE Studies SAFIRE / FICB Usability Study – 15 MAY 09

Goal: “To test the usability of SAFIRE technology for Incident Commanders and receive input on the technology.”

Methodology: Mixed-methods (quantitative and qualitative data) design SAFIRE evaluated by panel of IC experts at FF forum – Tabletop Exercise Drill Scenario stopped at 5 “freeze points” to assess 1) Usability, 2) Impact on decision-making



Freeze points were identified as critical junction / decision points:1. IC Setup2. Spill location Efforts3. Information Traffic Flows4. Search and Rescue5. Localization of firefighter personnel

PRE-FICB INTERVENTION “Given the capabilities of the FICB technology, would you use it at this freeze point?” YES / NO Optional Elaboration: If yes, how? If no, why? POST-FICB INTERVENTION “Would having this capability and information from FICB assist your decision on what to do next?” YES / NO Optional Elaboration: If yes, how? If no, why?



Results:1.Usability and decision-making impact significantly correlated with SAFIRE technology among ICs.2.Qualitative feedback overwhelmingly positive. Also, many suggestions for improvement.


Technology Validation: SAFIRE Studies Situational Awareness (SAFIRE – FICB) – 12 MAY 09

Goal: “To test the impact of SAFIRE technology on Situational Awareness during the Responsphere HazMat drill.”

Methodology: Traditional Experimental Design Control Group: Incident Commander without FICB Experimental Group: Incident Commander with FICB Drill Scenario stopped at 6 “freeze points” to assess Situational Awareness



Freeze points were identified as critical junction / decision points:1. Initial Baseline (no FICB)2. IC Setup3. Spill location Efforts4. Information Traffic Flows5. Search and Rescue6. Localization of firefighter personnel

At each freeze point, ICs were surveyed on their perception, comprehension, and projection (Endsley model of SA).



Results:1.Both ICs attained 100% comprehension. FICB reached this point earlier.2.FICB IC consistently higher in SA scores.3.SA with FICB: 49%4.SA without FICB: 32% (was indirectly influenced by FICB by virtue of FF actions)


Technology Validation: SAFIRE Studies Carboxyhemoglobin and CO Sensors – 23 FEB 09

Goal: “To test the efficacy and accuracy of commercial SpCO sensors and calibrate SAFIRE-created CO sensors in a live burn, field-deployment scenario.”

REvent Timetable:

9:57 -10:21 - Instrumentation/ calibration10:21- 10:47 – Burn10:47 – 11:46 – Break11:46- 12:01 – 2nd Burn

Results:1.FF Movement (e.g., hand, arm motion) caused spikes in SpCO readings and instrumentation read errors by misaligning the sensor strip. 2.Environmental factors (heat, smoke, etc.) caused no malfunctions.3.6,929 data points obtained in order to calibrate SAFIRE CO sensors.

safire: situational awareness for firefighters firefighter outreach, testing, validation goal:...

Documents