Integrated Factory Acceptance Test (IFAT) as Security Best Practice

04/21/23 FoxGuard Solutions 1

Larry Alls, Security Engineering ManagerFoxGuard Solutions

Good Afternoon

Brief History of Threats Security Myths Layered Defense Implementing a layered defense in the Industrial Controls

System (ICS) network Factory Acceptance Testing IFAT

– Questions to ask– Helpful hints– Lessons learned– Outcome– Benefits

04/21/23 FoxGuard Solutions 2

Evolution of Security Challenges

04/21/23 FoxGuard Solutions 3

GLOBALInfrastructure

Impact

REGIONALNetworks

MULTIPLENetworks

INDIVIDUALNetworks

INDIVIDUALComputer

Target and Scope of Damage

First Gen Boot

viruses

Weeks Second Gen

Macro viruses

Denial ofService

DaysThird Gen Distribute

d Denial ofService

Blended threats

Minutes

Next Gen Flash

threats Massive

“bot”-driven DDoS

Damaging payload worms

Seconds

1980s 1990s Today Future

Evolution of Threats and Exploits

04/21/23 FoxGuard Solutions 4

Packet Forging/Spoofing

Password Guessing

Self Replicating Code (WORM)

Password Cracking

Vulnerability Scanning

Audit DisablementBack Door

Exploits

Session Hijacking

Sniffers Stealth Diagnostics

High

Low

Pulsing Zombies

Self Installing Root Kits

Time

Dynamic Capabilities

Intelligent Bots

Com

ple

xit

y Expertise

Required

Mitnick or Wozniak

Script Kiddies

~90s Today

Think about it…

Implementing security on control systems at power plants is becoming more and more critical for the reliability of our electric sector.

Why is that? – Because NERC says so? – Because of terrorist threats?

What does this mean to the plant and the plant operators? How do we take the IT best practice of layered defense and apply

it to a control system environment? What is the impact of installing security on a control system? How does it affect the plant, the vendor, and the integrator? 

04/21/23 FoxGuard Solutions 5

Common Security Myths

Only specific users have access to my systems and I know who they are

We air-gap the ICS network so it’s not exploitable

Our firewall is bulletproof

What’s the worst that can happen?

04/21/23 FoxGuard Solutions 6

Worst-Case?

04/21/23 FoxGuard Solutions 7

Repeat After Me!

Disregard Security and your network:

Is vulnerable

Is exploitable

And someone will access it

04/21/23 FoxGuard Solutions 8

Why?

04/21/23 FoxGuard Solutions 9

Control systems use IT systems and networking technologies– NIST Special publication 800-82 is riddled with

information about the addition of IT technologies and how they pose threats to the ICS system, and what needs to be done to mitigate these threats.

Control systems may have implemented IT based solutions, but they have not kept up with IT technology. – ICS was designed to last 15 – 20 years– Lifecycle for typical IT system is 3 - 5 years– Combined with the security myths and the ever

growing IT threats, it’s time to act

Implementing Securityin the ICS

Challenging due to different vendors

Can you integrate these solutions into a single solution

Vendors don’t usually integrate their systems with one another

Some power providers are toying with the idea of managing their security from a single management layer, but are finding it challenging because of the different vendor solutions

This type of solution calls for some network designing and extensive testing prior to deployment

04/21/23 FoxGuard Solutions 10

Factory Acceptance Testing

TEST, TEST, and TEST AGAIN!!!

The answer for integrating anything into the ICS has always been a Factory Acceptance Test (FAT)

Implementing security is no exception

Integrated Security Factory Acceptance Test (IFAT)– Vendors, customer and integrator come together prior to installation

to “work out” site specific issues and test every facet of the security install

– These issues would normally have to be dealt with during the outage – This process saves the plant considerable time during the outage as it

relates to the cyber-security installation – They can then concentrate on other upgrades that are being

performed knowing that the added security is not going to impact start-up

04/21/23 FoxGuard Solutions 11

Questions to Ask

What vendors will be integrated into this plan?

Are they willing to work with the other vendors in a neutral environment?

To what extent will they cooperate?

Who will integrate this solution?

Who will write the test plans and oversee the IFAT?

What facilities are needed to accommodate the vendors?

What onsite security will be required by each vendor?

How can we maintain secure data transactions?

How can NDAs be handled between vendors?

04/21/23 FoxGuard Solutions 12

Top 5 Things to Remember

1. Communicate early, honestly, and thoroughly

2. Manage expectations on all sides

3. Not all the vendors will participate equally

4. Expect surprises that were not anticipated

5. Have clear definitions for Success and Failure

04/21/23 FoxGuard Solutions 13

Lessons Learned

Get complete requirements from all vendors and set up well in advance

Run at least two mock IFATs prior to having the real IFAT

Have clear applicable test plans and procedures

Keep personnel limited

Allow ample time for complete testing

04/21/23 FoxGuard Solutions 14

Closing the IFAT

Intangible product: Confidence– Confidence that the system to be delivered meets

expectations. This confidence is built from a long process consisting of several major milestones, one of which is the IFAT; another being the successful installation and execution of “real” science on the system.

Tangible product: The certification of a formal agreement– A signed agreement detailing what passed, what failed, and

the remediation plan for each failure/deficiency. If the remediation plan cannot be fully addressed at the IFAT, then a deadline for presenting this plan to the customer should be set. If another IFAT is required, this should be part of the remediation plan. In the worst case, the remediation plan may include how the system will be corrected on site, after installation at the customer facility.

04/21/23 FoxGuard Solutions 15

Who Benefits?

04/21/23 FoxGuard Solutions 16

The Vendor• This approach validates all the hard work that the vendor has put into

its system• Reduced loss / cost due to false expectations• Improved customer relations / confidence

The Customer• Confidence in the systems• Minimal impact during installation• Reduced implementation costs • Reduced costs due to non-compliance

The Integrator• Expectations of delivery are clear• Increased success rate of implementations• Reduced losses due to false expectations

Questions?

Larry Alls, Security Engineering ManagerFoxGuard Solutions

lalls@foxguardsolutions.com

FoxGuard Solutions provides cyber security, including HMI patching and updates, to industrial control systems.

www.foxguardsolutions.com

04/21/23 FoxGuard Solutions 17