2004 policy management with sparcle - universit© de montr©al

© 2007 IBM Corporation

2004

Workshop on Computer Privacy in Electronic CommerceWorkshop on Computer Privacy in Electronic CommerceWorkshop on Computer Privacy in Electronic CommerceWorkshop on Computer Privacy in Electronic CommercWorkshop on Computer Privacy in Electronic Commerce

Workshop on Computer Privacy in Electronic CommerceUniversity of Montreal5th May 2007

Improving Privacy Policy Management with SPARCLE John Karat, Clare-Marie Karat, and Carolyn BrodieIBM T. J. Watson Research [email protected]


Workshop on computer Privacy in electronic CommerceUniversity of Montreal5th May, 2007

SPARCLE Policy Workbench Project OverviewContext: Changing legal requirements, social pressures, and technologies are making privacy policy management issues increasingly important to organizations and society at large. For electronic commerce, personalization and privacy conflict.

Project Scope: The SPARCLE (Server Privacy ARchitecture and CapabiLityEnablement) project will create a highly usable policy workbench that enables organizations to:Create privacy policies (Author)Connect policy definition to system entities (Implement)Check policy compliance (Audit)

The functional prototype: Provides analysis of conflicts and redundancies in structured natural language privacy policies, displays results for review, and generates the machine-readable XML version of the policies.

The goal: Close the gap between privacy policy intention and machine execution.



A quick introduction to privacy policies

Enterprises collect large amounts of personally-identifiable information (PII)Because of the potential for abuse, it is desirable that access to PII be restrictedA privacy policy is a set of rules for how PII can be accessed and used by the enterpriseA privacy rule has up to 6 components: User categories, Actions, Data categories, Purpose, [Conditions, Obligations]



An example privacy rule

Marketing employees

name, address, and phone number

for the purpose of direct advertising

if the customer has opted-in.

can collect and useUser category

Actions

Data categories

Purpose

Condition



The privacy policy authoring challenge

More specific

Privacy legislation, corporate ideals

For more than 150 years, our company has been a trusted symbol of service and reliability. We safeguard your customer information carefully….

Structured rulesBilling representatives can use customer address for the purpose of mailing invoices

ImplementationMary Q. Employee is allowed READ access to database record #729 at 12:37pm on August 12

STARTGOAL

Less specific

NATURAL LANGUAGE

STRUCTURED LANGUAGE CODE



SPARCLE

SPARCLE is a Web-based application for writing structured privacy policy rules in natural language (and so much more!)SPARCLE usability goals:Help users write properly structured rulesHelp users write rules that match their intentionsHelp users understand policies



The User-Centered Design of SPARCLEPhase I: Questionnaire Research to Identify Organizational Needs

Phase II: Established Core Design Scenarios Based on In-Depth Interview Research

Phase III: HCI Based Architectural Analysis

Phase IV: Iterated on Design and Evaluation of the Policy Management Capability

Phase V: SPARCLE Policy Workbench Functional Authoring Tool Prototype

Phase VI: Ongoing Research and Development: Socially ResponsibleSystems



At DrugsAreUs, our business goals are to answer customer questions when they call in (Customer Service), fulfill orders for prescriptions while protecting against drug interactions (Pharmacists), and to provide customers valuable information about special offers (Marketing). We will ask the customers to provide us with full name, permanent address and contact information such as telephone numbers and email addresses, and a variety of demographic and personal information such as date of birth, gender, marital status, social security number, and current medications taken….

A privacy-policy authoring task: DrugsAreUs



Phase I - Questionnaire Research to Identify Organizational Needs

Recruited 51 Participants from Industry and Government from:– North America– Europe– Asia Pacific

Sent Participants Privacy Questionnaires by E-MailAnalyzed Data by Industry (N=23) and Government (N=28) SegmentsQuestionnaire Response Rate was Approximately 80% from Customers



Questionnaire Content

What are your top three privacy concerns regarding your organization?What are the top three types of privacy functionality you would like to have available to address your privacy concerns regarding your organization?At this time, what action is your organization preparing to take to address the top privacy concerns you listed above?



Main Concerns Expressed (N=51; Responses Expressed as Percentages)

0

5

10

15

20

25

Laga

cy d

ata

Web

dat

a

Exte

rnal

Inte

rnal

Partn

ers

Cou

ntrie

s

Bein

g Su

ed

Econ

omic

har

m

Take

n to

cou

rt

Keep

ting

up

othe

r

Privacy Concern

Tim

es S

elec

ted

IndustryGovernment



Privacy Functionality Desired (N=51; Responses Expressed as Percentages)

0

5

10

15

20

25H

ide

from

staf

f

Aut

horin

gto

ol Stic

kyPo

licy

One

for

Lega

cy

One

for

Web

One

for A

ll

Oth

er

Function

Tim

e Se

lect

ed

Industry

Government



Actions Taken To Date (N=51; Responses Expressed as Percentages)

0

5

10

15

20

25

Purc

hase

Dev

elop

Con

sult

No

Plan

s

Begu

n

Don

e

Oth

er

Action

Tim

es s

elec

ted

Industry

Government



Most organizations store personal information (PI) data in heterogeneous server system environments.

Currently they do not have a unified way of defining or implementing a privacy policy that encompasses both web and legacy applications across the different server platforms.

This makes the management of PI data difficult for both enterprises and end users.

Goal: Create an integrated set of privacy solutions for use across heterogeneous configurations covering all data.

Privacy Research Statement



Phase II: Established Core Design Scenarios Based on In-Depth Interview Research

Recruited 13 Customers from the sample of 51 One-hour structured interviewData transcribed, concepts extracted and organized into Affinity DiagramsResults summarized and presented to participantsThen developed core scenarios to drive the design and development of the capability based on the interview dataScenarios illustrated the privacy management capability in the health care, finance/banking, and government domains.Reviewed and updated the scenarios through presentation and discussion of them with target customers.



Phase III : HCI Based Architectural Analysis of Tactical and Strategic Options for Addressing Customer Need

Analyzed ability to provide privacy capability needed as illustrated in core user scenarios on each of the IBM System platforms. Discussed technical possibilities, trade-offs, and options with senior technical staff for each platform. In addition, considered options for competitor platforms in a configuration Created tactical and strategic options for each platform and a cross-platform solution. Reviewed and updated the architectural analysis and proposed tactical and strategic solutions with architects and sponsors.



Phase IV: Iterated on Design and Evaluation of the Capability

With the identified customer requirements that were illustrated through the core user scenarios, the Research team designed and built a Wizard-of-Oz prototype of the capability.

A Wizard-of-Oz prototype provides a realistic experience of capability for a target user without having functional code.

The team completed two rounds of design and evaluation of the privacy policy management capability.

Changes in design were made based on customer data from the evaluation of each version of the prototype.

The team also completed laboratory empirical testing of the value and effectiveness of the tool for users as indicated by the quality of authoring rules created through SPARCLE as compared to other methods.



MachineReadable

PolicyDS

NaturalLanguage

PolicyDS

Author Policy

Transform Policy

VisualizationOf Policy

Generalized Policy Creation Utility

DomainBased

Authorityand

Grammar Files

Natural Lang.Parsing Module

Policy Implementation

Utility

Policy Enforcement

Engine (e.g. RACF,

ITIM/ITAM, PMAC,Hippocratic DB, TPM,

Others)

InternalPolicy Audit

Utility

VariousSystem

Logs

Abstract Architecture for Policy Creation, Implementation, and Auditing

Domain Based

Policy Element Lists



2004 Privacy Policy Rule Authoring: Quality Results

• Unconstrained authoring yielded low quality

• Natural Language and Structured Entry yielded good quality

• Including both methods seems to be most promising direction

• Results published in ACM SIGCHI 2006 paper nominated for Best Paper

Preliminary Quality Evaluation Result

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Unconstrained NL w/Guide Structured

Stan

dard

ized

Qua

lity

Scor

e



2004-2006 Authoring Results – Comparable Features

• Results consistent over 3 years

• Natural Language and Structured Entry both seen as highly valuable

• Policy coverage visualization a key feature

Customer Ratings of SPARCLE Authoring Features

1234567

Enter w

/Templat

e

Enter ru

les w

ith N

L and

Guid

e

Enter ru

les w

ith lis

ts

Modify

rules

with

NL

Modify

rules

with

lists

Write be

tter q

uality

Review

polic

y with

table

Averag

es

Feature

Valu

e (1

=Low

est,

7=Hi

ghes

t)

200420052006



Phase V: SPARCLE Policy Workbench Functional Authoring Tool Prototype

Based on the very positive feedback from customers on the Wizard-of-Oz prototype, the Research team was challenged to design and built the functional SPARCLE authoring tool.

The project scope for the SPARCLE project was defined to be creating a highly usable policy workbench that enables organizations to:

Create privacy policies (Author)Connect policy definition to system entities (Implement)Check policy compliance (Audit)

The 2005 functional prototype provided natural language analysis of textual privacy policies, displays results for expert review, and generates the machine-readable XML version of the policies, with high customer satisfaction and 94% parsing precision.

Sponsor and Research began making plans for incorporation of the capability in STG product line.

The team also completed additional laboratory empirical testing of the SPARCLE policy authoring capability which confirmed previous results.



SPARCLE Free Text Entry



Policy Management ResearchPolicy ManagementWorkbench thatEnables:- Policy creators to

author in natural language, their preferred method

- Parsing of the policy & creation of an XML (machine-readable)version of the policy

- Mapping of this policy to organization’s configuration

- Creation of audit &compliance reportsbased on the policy

- Linking of policy to business operations,reducing organizational risk.More information available at http://www.research.ibm.com/sparcle

Policy rule enteredby user in natural languageParsed andreconstructedrules

Policy ruleelementsfound by parser in the selected rule



SPARCLE Policy Matrix



Mapping Terms to Digital Assets



Phase VI: Ongoing Research and Development

The work continues …Policy Management Framework for Security and Privacy

Joint work with CMU and Purdue Universities



2006 Policy Critic and New Authoring Results

• Redundancy and conflict checking across policies added

• Rule annotation and multiple policy view added

• All features highly rated• Critic features consistently

highest

Customer Ratings of 2006 SPARCLE Authoring and Critic Features

1234567

Average Basic

Authoring

Ident ify co

nflicts/r

edundancies

Resolve conflic

ts/redundan

cies

Analyze across policies

Annotate with ra

tionale

View multip

le policies

Feature

Val

ue (1

=Low

est,

7=Hi

ghes

t)



2006 Compliance Feature Results

• Family of functions rated very highly

• Event-based compliance auditing seen as valuable

• Tie to rules important• Data-subject based

auditing important

Customer Ratings of 2006 SPARCLE Compliance Features

1234567

Averag

e Basic

Authori

ng

Audit b

y rev

iewing

acces

s

Progres

sively

filter

logs

View m

appin

gs

Ident i

fy ac

cess

es de

nied

Audit b

y data

subje

ct

Summary

for a

data

subjec

t

Conne

ct ev

ents

to po

licy ru

les

Condu

ct ge

neral

compli

ance

audit

Alert to

exce

ption

al ca

ses

Feature

Val

ue (1

=Low

est,

7=Hi

ghes

t)



Some SPARCLE Team PublicationsKarat, C., Brodie, C., and Karat, J. (2006). Usable Privacy and Security for Personal Information Management. Communications of the ACM, 49, 1, 56-57.*Karat, C., Karat, J., Brodie, C., and Feng, J. (2006). Evaluating Interfaces for Privacy Policy Rule Authoring. Proceedings of the Conference on Human Factors in Computing Systems. NY: ACM Press, 83-92. *PIC conference, and paper nominated for Best Paper out of 550+ conference submissions.Brodie, C., Karat, C., and Karat, J. (2006). An Empirical Study of Natural Language Parsing of Privacy Policy Rules Using the SPARCLE Policy Workbench. Proceedings of the 2nd Symposium on Usable Privacy and Security. ACM Digital Library.Breaux, T., Anton, A., Karat, C., and Karat, J. (2006). Enforceability versus Accountability in Electronic Policies. Proceedings of POLICY 2006.Karat, J., Karat, C., and Brodie, C. (2006). Human-Computer Interaction Viewed from the Intersection of Privacy, Security, and Trust. In Jacko, J. and Sears, A. (Eds.), The Human-Computer Interaction Handbook, Second Edition, Erlbaum Associates, in press.Karat, C., Karat, J., and Brodie, C. (2007). Management of Personal Information Disclosure: The Interdependence of Privacy, Security, and Trust. In Jones, W. and Teevan, J. (Eds.), Personal Information Management, University of Washington Press, in press.Honorable mention in 2006 PET Awards for 2005 journal article. Please see http://www.microsoft.com/emea/presscentre/topstories/PETWorkshop_280606.msp

2004 policy management with sparcle - universit© de montr©al

Documents