Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

How To Eat A Mammoth

Experiences With the Evaluationof Complex Software Products

Under the Common Criteria

Gerald Krummeck (atsec), Bill Penny (IBM)

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Agenda

Our Experience Challenges from complex systems Evaluations under the Common Criteria The influence of complexity Strategies in mastering complexity Summary

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

atsec‘s Experience

Evaluation Labs in Germany, USA, Sweden More than half of all OS evaluations performed world-wide

• z/OS (IBM Mainframes)

• z/VM (IBM Mainframes)

• Linux (SuSE, Red Hat, Oracle)

• AIX

• Cray

• PR/SM, AIX LPAR Databases

• IBM DB2

• Oracle DB Tivoli System Management Products

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

IBM‘s experience

ISO 9001 Certified since 1993 WW development organization

• US, Canada, Germany, Australia, US• Mexico, Russia, China

Historically Independent Long History of IT Management

• Project Management• System Management• Process Control

Large Complex Systems• HW, SW• New Function and Service Models

Support Largest WW Business Requirements• High availability, security, integrity

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Challenges from complex systems

Dimensions of complexity in evaluations Size of the product Size of the TOE (what part will be evaluated) Amount of security functions

• Protection Profiles Depth of evaluation (EAL) Global distribution of development

• Multi-national

• Large number of organisational units

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Evaluation under Common Criteria

Security Target

FunctionalSpecification

High-LevelDesign

Low-LevelDesign

Implemen-tation

Tests

Vulnerability Analysis

Guidance documentation

Development Process (Life Cycle)

Delivery and Operation

Configuration Management

Product

Processes

SecurityPolicyModel

Design

Correspondence

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Influence of Complexity

Simple Systems• „Isolated“ evaluation possible

• Without knowledge of its origin and heritage

• Emphasis on design, test, guidance, vulnerability analysis

Complex Systems• Cannot be fully investigated

• Need to find additional ways to establish assurance/trustworthiness

• Establish trust in the development process

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Example: IBM z/OS Version 1Release 8

Size• Several Millions LOC (Assembler, PL/X, C, Java)

• Over 30 years development history

• Over 300 Manuals (120.000 pages)

• Over 630 Claims on security functions in the ST

• 10 development sites distributed globally 10 CM systems Common Corporate Standards and Processes

• Toute la Gaule est occupée… Toute?

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Interim Result

You cannot look at everything But you don‘t need to

• Security functions can be located quite accurately and can be tested thoroughly

• Requires sufficient experience and product know-how of the evaluators Development processes become very important Build trust in the developer to comply with his duties for every

piece that has not been scrutinized by the evaluators Again: Evaluators need experience and product know-how:

• It is an illusion to assume that everybody can perform a good evaluation just by applying the CC methodology (not everybody can eat the mammoth without choking on it)

• Customers need to identify the right laboratory for them with evaluators skilled in their type of product

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Strategies to master complexity

Not everything at once How to eat the mammoth Assistance Site Certification

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Not everything at once

Start modest• Focus on core functionality

• Start with lower assurance level (EAL2 oder EAL3)

• Pro: Get your first certificate in due time

• Con: lower assurance level than competition Example Linux:

• Start with EAL2, restrictive configuration

• Now EAL4, CAPP/LSPP, almost all packages included

• In between: write low-level design, add audit functions

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Example z/OS

MVS: Orange Book B1 (in the mist of times…) V1R6 – 2005

• EAL3, CAPP+LSPP (multilevel security)• Core functions: RACF, BCP, JES2, CS390, …

V1R7 – 2006• EAL4• Additional security functions

V1R8 – 2007• Major expansion of security functionality

V1R9• …

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

How to eat a Mammoth?

Bite by bite, of course! Don‘t become intimidated by the size Don‘t try to swallow it in one piece, either Important factors:

• Experience

• Confidence

• Perseverance

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Assistance

2 Teams from evaluation lab Evaluators

• Working on-site with developers is beneficial

• Additional testers with product know-how Consultants

• Help developer to gather evidence,prepare required documents

• Do not influence product itself or developer‘s decisions

Experienced certifiers help, too

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Developer committment

Multi-year committment Strong project management to coordinate all participating

organizations Strong technical leadership „Divide and Conquer“

• Strong leaders at distributed locations

• Educate, track, report

• Focus by area (ST, CM,HLD, Test) Communicate with Evaluation Team

• Open, early and frequent discussions

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Site Certification

Reduce complexity of the evaluation by reference to certification of sites

Idea• Certify development process for one site

• Re-use certificate in all applicable evaluations BSI tasked with development of site certification methodology Since 2005 development and test of certification process 2006 first pilot certification Acceptance in CC community Still more experience needed.

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Conclusion

Evaluation of complex products fits well in CC scheme

Medium to long term strategy (and committment!)

• Start modest

• Increase assurance level and functionality Processes must fit Find the right partner with experience and product

know-how

• ITSEF and certification body

Cop

yrig

ht a

tsec

info

rmat

ion

secu

rity,

IB

M,

2007

Questions, Comments