improving the operational effectiveness of dhs’s cyber ... · • bill gates’ “trustworthy...
TRANSCRIPT
![Page 1: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/1.jpg)
ImprovingTheOperationalEffectivenessofDHS’SCyberSecurityEvaluationTool(CSET)
Researchers:GilbertoCastroDannySeo
FacultyAdvisor:HenryJ.Sienkiewicz
![Page 2: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/2.jpg)
So why? Address the gap in ICS/SCADA/IoT
cyber security assessments
![Page 3: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/3.jpg)
Background• Researchprojects:
– Summer2018:Initialassessment– useofCSETtoidentifyvulnerabilities,andtheinitialapplicationoftheMicrosoftDREADmodel&SixSigmaQFDtoevaluateandprioritizerisk.– Fall2018:ImprovementstotheassessmentthroughextendingtheuseoftheDREADmodel&QFDtoCSET.
• GeorgetownUniversity’sSchoolofContinuingStudies– MasterofProfessionalStudiesin
TechnologyManagement– 30credithours/oncampus&on-
line/fullorparttime• Focusedcourseworkandpractical,
hands-onexperience.• SpecificCourses:
– Summer2018:MPTM665-40:PerspectivesinAddressingCybersecurity&CriticalInfrastructure:ANationalChallenge– Fall2018:MPTM661-01:InformationAssurance&RiskAssessment
https://scs.georgetown.edu/programs/77/master-of-professional-studies-in-technology-management/
Specialthanksto:• MarkBristow• DarylHaegley• StevenChen• AndrewWonpat
Addressing the gap in ICS/SCADA/IoT cyber security assessments
![Page 4: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/4.jpg)
Cyber Security Evaluation Tool
• IsasoftwareprogramdevelopedthroughconjuncteffortbetweencybersecurityexpertsandNISTunderthedirectionofthethenICS-CERT
• Providesasystematicandrepeatablemethodofassessingcybersecurityposture
• ProducesacomprehensivequestionnairebasedonServiceAssuranceLevel
• SupportsindustrystandardsfromNIST,NERC,TSA,DoDandotherapplicable
• Generatesarangeofreportsfromhigh-leveltodetailedforareview
Source: https://cset.inl.gov/SitePages/Home.aspx
• Helpswithriskmanagementanddecision-makingprocess
• Raisesawarenessandfacilitatesdiscussion
• Highlightsvulnerabilitiesandprovidesrecommendations
• Identifiesareasofstrengthandbestpractices
• Providesamethodtocompareandmonitorriskassessmentsovertime
• Recognizedasacommonindustry-widetoolforevaluatingcybersystems
CSET’s Key BenefitsWhat is CSET?
![Page 5: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/5.jpg)
Limitations of CSET• CSETgeneratesasetofreportsfocusingonthelevelof
compliance• Identifiesareasneedingattentionbasedonitsproprietary
weighting• CSETindicatespotentialvulnerabilities,butstops
there• Useful,but……
Theresultsneededtobecomeactionable
“Risk=ThreatxVulnerability”
Threats&Vulnerabilitiesmustbeidentifiedasapair inordertoassessrisk.
![Page 6: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/6.jpg)
Adding Utility
“Risk=ThreatxVulnerability”
Threats&Vulnerabilitiesmustbeidentifiedasapairinordertoassessrisk.
Prioritization basedupon• Organizationaldrivers• Acceptedmethodologies• Standardframeworks• Operationalneeds
![Page 7: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/7.jpg)
How can CSET be improved?
QFD forQuantitative
Prioritizationbaseduponbothqualitativeandquantitative
methodologies
Ensuringthat
“Risk=ThreatxVulnerability”Threats&vulnerabilitiesmustbeidentifiedasapair inorder
toassessrisk.
DREAD forQualitative
Repurpose existing, accepted industry standards
![Page 8: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/8.jpg)
DREAD & QFD for CSET• DREAD:QualitativeRiskAnalysisMethod
– Givesgranularsegmentationthanconventionalqualitativemethod(Risk=ImpactxLikelihood)
– D,R,E,A,Darenothighlycorrelated– DREADmodelisscalablefromsoftware
bugclassificationtoorganizationalcybersecurityriskassessment
– Rankinggivesafocusonworstvulnerabilities
• QFDappliedtoDREADmodel– Transformsqualitativevalues(High,
Medium,Low)intoqualitativevaluesthatcanbeanalyzedstatistically.
Sources: https://blogs.msdn.microsoft.com/david_leblanc/2007/08/14/dreadful/ ;Knapp & Langill, 2015
QFD forQuantitative
DREAD forQualitative
![Page 9: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/9.jpg)
CSET, DREAD & QFD in Action (Notional)
Threat Agents Exploit this vulnerability Resulting in this threat D R E A D Risk
ScoreCareless,
Negligent & Indifferent Employees (CNI), and
Intruder
No security awareness training
Falling prey to social engineering attacks (i.e., phishing, spear-
phishing, whaling);10 5 5 10 5 7
CNI, Contractor
Lack of training for security policies, procedures, and
processes including mandatory security
programs
Violation of regulatory requirements (i.e., NERC, FERC, FISMA, etc.) 10 10 10 10 10 10
Missing or poor definition of incident response plan or procedure including roles
and responsibilities, communication channel;No regular exercise and maintenance of incident
response plan
Possible to miss the golden time to respond to security incidents,
resulting in greater damage on finance, reputation, and even
human casualties
10 10 5 10 10 9
CNI, Intruder
No security protection (i.e., encryption, additional
credentialing) commensurate with the sensitivity level of data
stored in mobile devices
Increasing the attack surface as mobile devices with remote access capability are an extension to the
corporate network (and ICS network only if HMI application is installed)
5 5 1 5 10 5.2
![Page 10: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/10.jpg)
Research Conclusions• DREAD&QFDenhancesriskanalysisinCSET.• DREADmodelinvolvesjudgmentofassessor(s)whenevaluatingeachthreat
andvulnerability&rankingrisk.• Givensubjectivity,itisimportanttoexerciseconsistencythroughouttherisk
assessmentandfutureassessments.
Potential Next Steps
Editorial Observation
• EnhanceandautomatetheCSETtooltoincludetheDREAD&QFD• Continuedevelopmentonstandard,specificallyanICSAT&TKframework• IncludeattacktreeanalysisandSHODANresults
• Thistypeofactionableresearchisagreatexampleofpotentialpartnershipsbetweenacademia,government,andcommercialorganizations.
![Page 11: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/11.jpg)
BACKUPS
![Page 12: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/12.jpg)
A Bit Of A Primer: “How does CSET work?”
• Step1:ProvideSiteInformation
• Step2:DefinetheSectorandtheDemographics
• Step3:Diagram&NetworkComponentSelection
Ø Step4:Modeselection
Ø Step5:ServiceAssuranceLevelDefinition
Ø Step6:Answerthegeneratedquestions
Source: CSET Version 8.1
![Page 13: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/13.jpg)
Step 4 - Mode selection: Basic or Advanced
Source: CSET Version 8.1
BasicModeØ UsestheprovideddemographicinformationØ SelectsappropriatedefaultquestionsØ Doesnotreferencecybersecuritystandards.Ø Appropriatefor
- Organizationsthatarenotregulatedbyaparticularindustry- Areinthedevelopmentalstageofacybersecurityprogram.
AdvancedModeØ Questions-basedapproachusessimplequestions.Ø Requirements-basedapproachusestheexactwording
fromastandardandisbestsuitedforthoseindustriesregulatedbyaspecificstandard.
Ø Cybersecurityframework-basedapproachallowstheassessortodefineacustomprofilebasedontheCybersecurityFramework.
![Page 14: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/14.jpg)
Step 5 – Security Assurance Level (SAL) Definition
Source: CSET Version 8.1
• Level selectionØ Low SAL – typically 30 to 350 questionsØ High SAL – typically 350 to 1,000 questions• Standards selectionØ Framework based approachØ Baseline framework is automatically populated• Implementation tiersØ Properties
- Risk management processes- Integrated risk management program- External participation
Ø Each property has fours tiers – representing a level of maturity
CSET determines the overall tier level and the equivalent SAL for the assessment, which are commensurate with the total number of questions.
![Page 15: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/15.jpg)
Step 6 - Answer the generated questions
Source: CSET Version 8.1
Every question provides detailed supplemental information that provides guidance to the assessor in the subject being questioned.
![Page 16: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/16.jpg)
Generate CSET Reports
Source: CSET Version 8.1
• ExecutiveSummary• SiteSummary• SecurityPlan• Otherdetailedreports• ComponentGapAnalysis
Again,usefulbut…..
Theresultsneededtobeprioritizedandactionable
![Page 17: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/17.jpg)
Microsoft DREAD Model
• BillGates’“TrustworthyComputing”memo(2002)- asavailable,reliable,andsecureaselectricity,waterservicesandtelephony
• “WritingSecureCode”byMichaelHoward&DavidLeBlancintroducedSTRIDEandDREADaspartofthreatmodeling
• DREADModeloriginallydevelopedtoclassifysoftwarebugs
• DREAD- Damagepotential,Reproducibility,Exploitability,Affectedusers,andDiscoverability
Source: https://www.wired.com/2002/01/bill-gates-trustworthy-computing
![Page 18: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/18.jpg)
Microsoft DREAD Model (Cont.)Rating High Medium Low Indirectly
Measures
DDamage potential
Attacker can subvert the security; get full trust authorization; run as administrator; upload
content
Leaking sensitive information
Leaking trivial information Consequences
R Reproducibility
Attack can be reproduced every time; does not
require a timing window; no authentication
required
Attack can be reproduced, but only with a timing
window and a particular situation; authorization
required
Attack is very difficult to reproduce, even with
knowledge of the security vulnerability; requires administrative rights
Likelihood
E ExploitabilityNovice programmer could make the attack in a short
time; simple toolset
Skilled programmer could make the attack, then
repeat the steps; exploit and/or tools publicly
available
Attack requires an extremely skilled person and in-depth knowledge
every time to exploit; custom exploit/tools
Likelihood
A Affected Users All users; default configuration; key assets
Some users; non-default configuration
Very small percentage of users; obscure feature;
affects anonymous usersConsequences
D Discoverability
Published information explains the attack;
vulnerability is found in the most commonly used features; very noticeable
Vulnerability is in a seldom-used part of the product; only a few users should
come across it; would take some thinking to see
malicious use
Bug is obscure; unlikely that users will work out
damage potential; requires source code; administrative access
Likelihood
Sources: Howard & LeBlanc, 2002; Knapp & Langill, 2015
![Page 19: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/19.jpg)
Quality Function Deployment
• ProductdesignmethoddevelopedinJapanin1966
• "HouseofQuality”
• Transformsqualitative userdemandsintoquantitativeparametersrelatedtoorganizationalcapabilities
Sources: https://www.sixsigmadaily.com/six-sigma-and-quality-function-deployment/https://sixsigma.com.my/training/product/quality-function-deployment/
![Page 20: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/20.jpg)
QFD Likelihood & Impact Definitions
Source: Touhill & Touhill, 2014
![Page 21: Improving The Operational Effectiveness of DHS’S Cyber ... · • Bill Gates’ “Trustworthy Computing” memo (2002) - as available, reliable, and secure as electricity, water](https://reader034.vdocument.in/reader034/viewer/2022050611/5fb27fcb8ce835194c58a2db/html5/thumbnails/21.jpg)
Thankyou