cc in the real world
DESCRIPTION
CC in the Real World. Fiona Pattinson. NASA are owners of this picture produced by The Visible Earth. When technologies based on the state of the art in a rapidly maturing discipline are put into practice some “real world” issues surface for the practitioners to cope with. - PowerPoint PPT PresentationTRANSCRIPT
1
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
NA
SA
are
ow
ners
of
this
pic
ture
pro
duce
d by
The
Vis
ible
Ear
th
When technologies based on the state of the art in a rapidly maturing discipline are put into practice some
“real world” issues surface for the practitioners to cope with.
Fiona Pattinson
CC in the Real World
2
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
In the U.S. GAO report:(CC consumers)
• March 2006– difficulty in matching agencies’ needs with the
availability of NIAP-evaluated products – vendors’ lack of awareness regarding the
evaluation process – a reduction in the number of validators to
certify products – a lack of performance measures and difficulty
in documenting the effectiveness of the NIAP process
http://www.gao.gov/new.items/d06392.pdf
3
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
Vendors Issues
• May 2007 GCN: “Symantec: Common Criteria is bad for you” – CC is a closed standard development– Does not address areas where vulnerabilities
occur – Starts late in the development process – It is difficult to produce a complete ST at the
beginning of a project– Assurance vs convenience balance
http://www.gcn.com/online/vol1_no1/44205-1.html
4
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
General Issues with CC…• Timeliness
– Takes too long
• Cost– Costs too much
• Validity/Scope of certification– Too narrow
• Mutual Recognition– limited
• Composition– Was not addressed
• Comprehensibility– Too hard to cope with
David Brewer: Is the Common Criteria the only way?”: https://www.ipa.go.jp/event/iccc2005/pdf/B2-10.pdf
5
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
• Over the last few years:– greater emphasis in understanding the
risks presented by a greater number and variety of IT products
– Emphasis on certification as a solution to help assist risk based decisions
• Scalability–Can the CC schemes handle the
needed volume?
“The product I want to buy is not on the list.”
6
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
Rise in number of evaluations
Number of Evaluated Products reported on the CC Portal
0
20
40
60
80
100
120
140
160
180
200
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
yearJuly
ITSEC Evaluations*
(*From “Is the Common Criteria the only way?”
, David Brewer, 2005)
7
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
• This *is* a FAQ from new vendors & teams– Most national schemes publish good beginner
info– So does the CC Portal– Many labs provide basic information (and
consultancy!)– “I’ve heard that CC is expensive, takes forever,
needs lots of specialised documents, is hard work.. etc”
“I don’t know where to start with CC.”
'Does Common Criteria need marketing?', ICCC 2006, Tokyo.
8
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
• Effects– Some claim it is used as a trade control
measure– Prices smaller companies out of the market
• Compare– FIPS 140-2: Smaller scope, smaller cost
• Consultation and testing components
– ISMS: • Consultation and auditing costs
“It costs HOW much?! ”
9
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
• This may be true for a first time evaluation
• There’s a learning curve for product teams using CC
• As a product and dev team’s “evaluation career” progress it is possible to have certification coincide to within a few weeks of the end of the development cycle
“Our products are released every 6 months, but we were told that
our CC evaluation would take 9-18 months.”
10
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
Example: Linux
200720062005200420032002
IBMSLES 8
HPRHEL3-U3
IBMRHEL3-U2 Unisys
RHEL3-U5RHEL4
HPRHEL3
IBMRHEL4-U1
HPRHEL4-U2
SGIRHEL4-U4
HPRHEL5
IBMRHEL5
IBMSLES8 SP3
HPSLES8 SP3
HPSLES9
SGISLES9 SP2
RHEL 5RHEL 4RHEL 3
SLES10 SP1SLES10 SLES9SP2
SLES9 SLES8 SP3 SLES9 SP3SLES8
SP4
RHEL 4U1
RHEL 4U3
Distribution version release
Certification date for platform evaluation under CC
OEL4 U4/U5
11
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
• CC is not a direct process or product improvement standard
• If an organization has assurance provision as part of a product-line strategy then security process and product improvement are vital– Informal, or formal (SSE:CMM ISO/IEC 21827, CMMI,
TickIT, even ISO/IEC 9001 and 90003)– Processes may even address the needs of particular
standards
• Secure System Design– Design to provide assurance
How can I make CC easier next time?
Improving development process with assurance provision in mind
12
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
• Unrealistic boundaries are specified in order to:– Reduce evaluation effort (time/cost)– Avoid problem areas
• CC is deliberately flexible, and does not prevent ST writers defining unrealistic boundaries
• The specification of inappropriate boundaries bring CC into disrepute as consumers are left with claims that are meaningless in their objective of assessing their risk
“But you didn’t evaluate anything I use or that’s
remotely useful.”
13
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
• In practice this can be alleviated by:– Scheme policies– Specification and adoption of appropriate PPs– Ethical laboratories and consultants advise their
customers appropriately
• Compare with :– Specifications e.g. FIPS 140-2
• Boundary is more rigorous but still mis-used sometimes
– ISMS(& QMS) • (At least for QMS) the specifications for scope have been
mis-used and caused disrepute.
14
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
“It doesn’t evaluate a whole product”
• Security functionality only– No assurance given for non-security functionality
• Some product vulnerabilities appear from the TOE environment and are “dismissed” by broad assumptions– This is technically correct– Hard for real-world users to understand– Does it help assess risk more accurately in the real-
world?• Somewhat true also for FIPS 140-2, although
CMVP certificates and security policies tend to reference the whole product
15
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
• CC is very flexible – SFR extensions– PP– Interpretations– Ongoing standards development
• It is difficult to produce a complete ST at the beginning of a project– “Before and during the evaluation, the ST specifies “what is to be
evaluated”. In this role, the ST serves as a basis for agreement between the developer and the evaluator on the exact security properties of the TOE and the exact scope of the evaluation…. After the evaluation, the ST specifies “what was evaluated…” (CC part 1 A.3.1)
• Compare– FIPS 140-2 which is more restrictive, and actually means that there are
several products that it can’t be used for or that it is very difficult to be used for.
– ISMS can be used for any organization
“CC is inflexible; it can’t be used for my product!”
16
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
Yes it is!
• The more assurance end users need, the harder work it is
• Internal effort is related to evaluation strategy, and can be reduced– Process improvements
“It’s hard work”
17
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
“It assumes a waterfall model for development”
• No it doesn’t! CC paradigm is flexible and it fits all development models.
• Spiral methods, pair programming, iterative and incremental, agile, open source development, rapid prototyping etc can and have been used.
• The evaluation too can be built in to the processes of these models and use similar paradigms
• Examples: – The successful evaluation of Open Source development
paradigm for various Linux evals, the provision of evaluation artifacts and evidence back to the community.
– An iterative evaluation methodology to track and pace iterative product development.
18
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
“It requires too much documentation”
19
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
It is a formalized methodology, and documentation that would not otherwise be produced is required in order to provide high assurance– E.g. ST, Correspondence analyses,
• Other documentation is often produced “just for CC” that the lab should be able to find in a regular mature product development – E.g. Design documentation, Process
documentation, Vulnerability assessment
20
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
“Documentation is never the cause of a security failure so
why emphasise that!”• Documentation is an evaluation tool. It shows
– The design is sound: A product can (and often is) designed poorly. Design flaws => vulnerabilities
– The design is implemented to the level of abstraction specified– It shows the system was designed to implement the security
requirements
• Specialised documentation is an important part of providing assurance: Especially boundaries– PPs and ST for CC – Security Policy, Finite State Model (FSM) for FIPS 140-2– Scope and Statement of Applicability (SOA) for ISMS
• Labs must be prepared to read and use existing documentation in any format.
21
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
“We want ONE evaluation that’s accepted everywhere”
• Mutual Recognition• CCRA /SOGIS are examples• ISO/IEC 15408
• Failed for Smartcards• Failing because of need for higher levels
of assurance (medium level robustness PPs) which are not covered under current MRAs
22
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
Addressing these issues using the CC paradigm
• Developing useful Protection Profiles• Using a “staged” evaluation strategy, so that initial
evaluation of a product at a lower evaluation level builds a good platform for later evaluation at a higher level
• Pursuing ongoing communication between consumers, vendors, evaluation labs, and schemes, so that emerging consumer requirements on mature product lines are clearly understood by all parties in the evaluation process
• Conducting development, Common Criteria consulting, and Common Criteria evaluation efforts in parallel with development efforts, so that certification of new products is timely (atsec example: Red Hat Linux evaluation project)
http://www.atsec.com/downloads/pdf/efficient_cc_evaluations.pdf
23
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
Conclusion• Real World Issues occur with every certification
frameworkBoth standard and the certification scheme have to
be sound.Some specialised documentation is necessary, but
this should be minimal.The definition of clear boundaries are vital.Balance “certification as a tax” vs “real-life
improvement and benefits of certification”Communication is vital
Mutual Recognition arrangementsIncluding training for all stakeholders (Overseers,
Labs, Vendors, Consumers, Sponsors)
24
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
Thank You
Fiona Pattinsonatsec Information Security Corporation
Austin, Texas, USA
Phone: +1 512 615 7382
Email: [email protected]
www.atsec.com
25
© 2
007
a
tse
c in
form
atio
n s
ecu
rity
8th ICCC: CC in the Real World
References & Bibliography• Olthoff, K. G. 1999, 'A cursory examination of market forces driving the Common Criteria', in New Security
Paradigms Workshop, ACM Press, pp. 61-66.• Gordon, L. A. & Loeb, M. P. 2001, Economic Aspects of Information Security, Rainbow Technologies.• Mahon, G. 2002, 'Developers backward engineering their products to meet the Common Criteria', in International
Common Criteria Conference (ICCC), Ottawa.• Merkow, M. 2003, Security assurance through the Common Criteria, New Riders Pub., Indianapolis, IN.• Apted, A. J. , Carthigaser, M. & Lowe, C. 2002, 'Common problems with the Common Criteria', in International
Common Criteria Conference, Ottawa.• Aizuddin, A. , The Common Criteria ISO/IEC 15408– The insight, some thoughts, questions and issues.• Smith, R. E. , Trends in Government Endorsed Security Product Evaluations, Secure Computing Corporation.• The common criteria -- Good, bad or indifferent? , Information Security Technical Report, vol. 2, no. 1, pp. 5-6 PY
- 1997 AU - L.• Security evaluation and certification: The future of a national scheme. , Information Security Technical Report, vol.
2, no. 1, pp. 6-6 PY - 1997 AU - H.• Brewer, D. 2005, 'Is the Common Criteria the only way?', International Common Criteria Conference, Tokyo.• Oldegård, M. & Farnes, M.-L. 2005, 'Does Common Criteria need marketing?', International Common Criteria
Conference, Tokyo.• Mueller, S., 4/10/2006, ”Efficient Common Criteria Evaluations” :
http://www.atsec.com/downloads/pdf/efficient_cc_evaluations.pdf• Katzke, S. 2004, The Common Criteria (CC) Years (1993-2008): Looking Back and Ahead.• GAO. 2006, Information Assurance: National Partnership offers benefits, but Faces Considerable Challenges,
United States Government Accountability Office.• Government, Computer News, 2007/05/04, “Symantec: Common Criteria is bad for you” Jackson, J:
http://www.gcn.com/online/vol1_no1/44205-1.html• CCMB-2006-09-01 "Common Criteria for Information Technology Security Evaluation Version 3.1: Part 1:
Introduction and general model" The Common Criteria Management Board.