Security Testing Improvement Profile (STIP)

Jürgen Großmann, Fraunhofer FOKUS, juergen.grossmann@fokus.frauhofer.de

An evaluation scheme for security testingSASSI13 – Security Assessment for Systems, Services and InfrastructuresSeptember 2013 at the Technical University (TU) in Berlin

Motivation

Technical Guide to Information Security Testing and Assessment NIST Special Publication 800-115

TMMi, TPI© and TPI NEXT©

\

• TPI, TPI Next are registered trademarks of Sogeti

• TMMi is based on CMM, anddeveloped by the Illinois Institute of Technology The TMMi Model from http://www.tmmi.org/

§ Analysis with respect of the key areas§ Levels are used to assign a degree of maturity to each key area§ Checkpoints are defined to determine the level for each key area§ Each higher level is better than its prior level in terms of time (faster), money

(cheaper) and/or quality (better).

Maturity Scale § Staged representation:

§ Initial§ Controlled§ Efficient§ Optimizing

§ Continuous representation§ A – M or 1 -13

TPI© and TPI NEXT©

KeyAreas

MaturityLevels

ImprovementSuggestionsCheckpoints

Security Testing Improvement Profiles (STIP) enables an objective, detailed analysis and evaluation of security testing processes

§ First introduced to evaluate the case studies of the DIAMONDS project

§ Provide a detailed analysis and evaluation of our research & development

§ Show how tools & techniques have evolved§ Provide a template for other on how to pragmatically integrate the

DIAMONDS results to improve security testing processes on hand.

Security Testing Improvement Profile (STIP)Evaluation of the DIAMONDS Case Studies

Analysis with respect of the key areas§ Levels are used to assign a degree of progress to each key

area§ Each higher level is considered better than its prior level in

terms of quality (e.g. exactness of the outcome) or effectiveness (e.g. automation of activities).

STIP key areas

Inceptionandtargetanalysis

Informationgathering

Securityriskassessmenttechnique

Securityriskassessmentscope

Securitytestidentification

Elaborationandexecution

Testdepth

Generationofsecuritytestmodels

Securitytestgeneration

Securitytestexecutionautomation

TestTechniques

Securityfunctionaltesting

Fuzzing

Securitypassivetesting/securitymonitoring

Staticsecuritytesting

Artifactconsistencyandtoolsupport

Securitytesttoolintegration

Traceability&test

coverage

STIP level definitionKey area: Risk Assessment Technique

A:Informalsecurityriskassessment

Atthislevel,thesecurityriskassessmentisconductedinanunstructuredmannerwithoutaspecificnotation/languagefordocumentriskassessmentresultsoraclearlydefinedprocessforconductingthesecurityriskassessment.

B:Model-basedsecurityriskassessment

Atthislevel,thesecurityriskassessmentisconductedwithalanguagefordocumentingassessmentresultsandaclearlydefinedprocessforconductingtheassessment.

C:Modelandtest-basedsecurityriskassessment

Atthislevel,themodel-basedsecurityriskassessmentusestestingforverifyingthecorrectnessoftheriskassessmentresults.

STIP level definitionsKey area: Security Test Identification

A: Identificationbasedonrequirementsanalysis

Testidentificationcanbebasedontheanalysisofthefunctionalsecurityrequirements(SFR)andtheircoveragethroughtesting.Oftentheserequirementshaveprioritynumbersthatadditionallyprovideguidanceontheimportanceofarequirementandtherelatedtestpurpose.

B: Identificationbasedonthreat/vulnerabilitymodels

Securitythreat/vulnerabilitymodelsadditionallyallowfortheidentificationofpenetrationteststhatarebasedonestimationsonpotentialthreatsandpotentialvulnerabilities.Thisallowstestingforunwantedincidentsthatarenotcoveredbythesecurityfunctionalrequirements.

C: Identificationbasedonthreat/vulnerabilitymodelsandtestpattern

Thecombinationofthreat/vulnerabilitymodelsandtestpatternadditionallyprovidesbestpracticesfortheidentificationandselectionoftestingmeansdedicatedtowell-knownclassesofthreatsorvulnerabilities.Thisapproachprovidesextensiveguidancetoidentifyadequatetestpurposesandtoapplyapprovedsecuritytestingmethods,techniquesandtools.

D: Risk-basedsecuritytestidentification+prioritization

Risk-basedsecuritytestidentificationandprioritizationcombinestheadvantagesofLevel3withaprioritizationofthetestpurposesbyconsideringprobabilitiesoftheunwantedincidentandestimationsontheirconsequences(quantifiedsecurityrisks).Theintegrationoftestidentificationwithsecurityriskassessmentallowsforaproblemandbusinessspecificprioritizationoftheidentifiedtestspurposesandtestingapproaches.

Analysis and improvement suggestions

• A security testing matrix defines the current state of a process (blue background).

• Profiles define optimal and well aligned security testing levels (red line).• Improvements suggestions are to be defined on basis of dependencies

between key areas and their levels (red background)• e.g. Security test identification B requires Security risk assessment

technique B (green arrow)

Securitytestingsolutionsforsixindustrialdomainsin8casestudies

• Banking• Automotive• Radioprotocols• Smartcards• Telecommunication• Industrialautomation

Application of STIP Evaluation of the DIAMONDS case studies

Evaluation of the DIAMONDS Case StudiesSTIP results for the international case studies

Evaluation of the DIAMONDS Case StudiesProgress in all case studies

Banknote processing machine case study

§ STIP is an evaluation and improvement scheme for security testing processes§ First introduced to evaluate the case studies of the DIAMONDS project§ Provide a detailed analysis and evaluation of security testing processes on hand§ Provide a template to pragmatically improve security testing processes on hand

§ First version is available at http://www.itea2-diamonds.org/evaluation/stip/index.html§ Can be used in addition to TMMi or TPI to emphasize security testing aspects.§ FOKUS plans to offer consultancy and certification optinos on basis of STIP in the near

future

Contact:Jürgen GroßmannFraunhofer Institute for Open Communication Systems FOKUSMOTION – Modeling and Testing for System and Service SolutionsKaiserin-Augusta-Allee 31, 10589 Berlin, GermanyE-Mail: juergen.grossmann@fokus.fraunhofer.de

Summary & Conclusion