policy governance via social...

Policy Governance via Social Norms

Ozgur Kafalı

Department of Computer ScienceNorth Carolina State University

[email protected]

SoS Quarterly Lablet Meeting at College Park, UMDOctober 26, 2015

Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 1 / 40

Agenda

Today: tutorial on policyPolicies and normsProblem domainsSociotechnical systemsMethods and tools

Tomorrow: specific application of social normsFormalization of norms in temporal logicRelations among normsRevision patternsUse case from healthcare


Research Background

Areas ofInterest

Artificial Intelligence Multiagent Systems(Distributed AI)


Research Background

Areas ofInterest


SpecificExpertise

ComputationalLogic

ModelChecking

AgentPlatforms

Agent Interaction


Research Background

Areas ofInterest


SpecificExpertise

ComputationalLogic

ModelChecking

AgentPlatforms

Agent Interaction

ApplicationDomains

SocialNetworks

Bioinformatics Healthcare E-Commerce


Outline

1 Social Side of Security

2 Policy-Governed Systems

3 Sociotechnical Systems (STS)

4 Social Norms

5 Security Properties

6 Formal Methods and Tools

7 Open Challenges


Social Side of Security

Security Big Picture

Collaboration among multiple users (often confused and careless)Policy at technical level

Access control (logical and physical resources)Logging requirements

Policy at social levelRegulate how users engage with each otherModel exceptions (when something goes wrong)

[Singh, AAMAS, 2015]Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 5 / 40

Social Side of Security

Current Research Efforts

Most security efforts focus on the technical (software) levelRegimentedControl focused

Vulnerabilities due to user behaviorSharing passwordsNot applying important security patches

Some recent efforts directed to the social levelSocial normsSociotechnical systems

Unifying framework to understand tradeoffs between architectures


Policy-Governed Systems

Policy and Governance

Policy: description of behavior (intention)documented statements of an organizationprivacy policy describing the intent of how data will be used

Governance: exercise of control (plan)how specific policy details are carried out as processessubject to laws, norms, powertraditional management: top-down hierarchical modelpeer-to-peer model: administration by the stakeholders



Policies Everywhere

Effect everyday lives of peopleDifferent (possibly conflicting) security policiesOne’s security might be another’s restrictionNCSU policy effort directed to

Understanding the balance between safety and liveness of usersBuilding formal models of security policies and normsResolving conflicts among policiesUnderstanding the effects of personality and sanctions on policyUnderstanding policy complexity



Cybersecurity Domains: Healthcare

HIPAA privacy rule for public healthemergencyOriginal provision: Obtain patient’sagreement to speak with family membersIf president declares a disasterThen, secretary may waive sanctions andpenalties regarding the provision

POLICIES SHOULD BE FLEXIBLE

http://www.hhs.gov/ocr/privacy/hipaa/faq/disclosures in emergency situations© https://www.propublica.org/


http://www.hhs.gov/ocr/privacy/hipaa/faq/disclosures_in_emergency_situations

https://www.propublica.org/


Cybersecurity Domains: Healthcare

HIPAA privacy rule for public healthemergencyOriginal provision: Obtain patient’sagreement to speak with family membersIf president declares a disasterThen, secretary may waive sanctions andpenalties regarding the provisionPOLICIES SHOULD BE FLEXIBLE

http://www.hhs.gov/ocr/privacy/hipaa/faq/disclosures in emergency situations© https://www.propublica.org/


http://www.hhs.gov/ocr/privacy/hipaa/faq/disclosures_in_emergency_situations

https://www.propublica.org/


Cybersecurity Domains: Secure Building

Adaptive securityStatic: Visiting techniciansonly allowed in rooms they’resupposed to carry out workDynamic: People only allowedin the server room withauthorized staff

POLICIES SHOULD BEADAPTIVE

Tsigkanos et al., RE, 2014Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 10 / 40


Cybersecurity Domains: Secure Building

Adaptive securityStatic: Visiting techniciansonly allowed in rooms they’resupposed to carry out workDynamic: People only allowedin the server room withauthorized staffPOLICIES SHOULD BEADAPTIVE

Tsigkanos et al., RE, 2014Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 10 / 40


Privacy Domains: Social Networks and Mobile Apps

Different regulations about sharing of user dataSocial networking sites team up with popular mobile apps

Low user awareness, acknowledgment of privacy policiesUsers at most read the top-level policy, ignore all othersHow does the social network interact with the mobile app?How does the mobile app interact with 3rd parties?

POLICIES ARE COMPLEX

© http://www.coffeexperts.eu/privacy-policy/


http://www.coffeexperts.eu/privacy-policy/


Privacy Domains: Aviation

Should the privacy of the pilot be traded for flight safety?Patient doctor confidentiality prohibits sharing of psychologicaltendencies early onChanges to aviation policies after the incident

Two authorized personnel in the cockpitSharing of psychological assessment with airlines (prior to hiring)

POLICIES NEED REVISION

© https://www.flickr.com/Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 12 / 40

https://www.flickr.com/

Sociotechnical Systems (STS)

STS Definition

A social organization, wherein autonomous agents that representstakeholders interact with each other through (and about)technical components

Normative focusPromote collaboration among stakeholders via social architecture

People and processes, as well as technological systemsProcess definitions outline how the system designers intend thesystem should be usedIn practice, users interpret and adapt them in unpredictable ways,depending on education, experience, culture

[Singh, ACM TIST, 2013] [Sommerville et al., Communications of ACM, 2012]Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 13 / 40


Architecture of an STS

Requirements

Stakeholders Agent . . . Agent

NormsAssumptionsMechanisms

Functional and ControlComponents

interact

realizedin

regulate

identify

specify

Social TierTechnical Tier



Social Organizations: Hospital Administration

RolesAdministratorPhysicianTechnicianPatient

Technical regulationsAccess to patient records (software level)Traceability for sharing patient data (logging level)Access to emergency department (physical level)

Social interactionsCalling out patients in the waiting areaPhysician discussing patient’s condition with lab technician



STS Challenges

How to monitor, detect warning signs?Unusual access to patient records

How to recover from failure?Patient information shared with colleague vs published online

Regulation for safety-critical systemsWho can access the server room?

Nondeterministic behavior and probabilistic verificationEffect of having strict authentication rules on patients’ survivabilityin emergencies?

Sommerville et al., Communications of ACM, 2012Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 16 / 40


Technical vs Social Architecture

Balance between technical and social architectureSocial gain: improve flexibility without trading away security

Grant all available physicians access to emergency records withoutconsentProvide traceability and accountability via authentication logs

Technical gain: improve security without trading away flexibilityProvide central registry for physicians (global authentication)Call out all available physicians from an area in case of disaster



Interoperability in STS

Collaboration among stakeholdersPhysician consulting a colleagueRegulations regarding what can be discussed and howSharing patient information via e-mail is traceable (technical level)Talking about a patient over lunch may not always be traceable(social level)

Composition of STS: hospital and pharmacyWeb service for prescription of drugs (technical level)Pharmacist is prohibited from editing physician’s prescription (sociallevel)



Exceptions

Handle deviations from normal executionAvoiding all exceptions is impracticalException for one user might have no effect on anotherNot all exceptions are bad

Physician informs the public about an outbreakAgainst the regulations for sharing patient related informationViolation is necessary to protect the health and safety of the public


Social Norms

Brief History

A norm represents a social regulation between two autonomousentitiesIdeas from law

Claim (right), privilege, power, immunityEach act of promising is an act of the same type as an act orprocess of enactment of a lawFrom individual to groups and organizations

A popular approach in distributed AINormative systems to model artificial organizationsAgents reason upon norms

[Hohfeld, 1919] [Castaneda, 1975] [Castelfranchi, 1995] [Singh, 1999]Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 20 / 40

Social Norms

Norm Types: Commitment

Practical: ‘I will do [P] if [Q] happens’

e.g., commitment to log out from public computers

c(physician, hospital, public ∧ EHR, log out)

Dialectical: ‘[P] is true’

e.g., ‘I logged out immediately after I’ve reviewed patient’s EHR’

d(physician, hospital, logged out)

Violated if proven otherwise (e.g., via logs)


Social Norms

Norm Types: Authorization

‘You may do [P] if [Q] happens’

e.g., authorization to access patient records

a(physician, hospital, consent, EHR)

Violated if EHR is accessed without consent (if permitted bysoftware)


Social Norms

Norm Types: Prohibition

‘You should not do [P] if [Q] happens’

e.g., prohibition for accessing nonpatient EHR

p(physician, hospital, ¬own patient(Patient), EHR(Patient))

Violated if physician accesses a patient’s EHR other than her own


Social Norms

Norm Types: Sanction

Penalty for violating normsHospital administration punishes the physician for accessingpatient’s records without consentMay be escalated to higher authorityMay be used to update the reputation of a stakeholder


Social Norms

Norm Types: Power

Ability to alter the norms between two or more stakeholdersHospital administration has the power to revise regulationsregarding the use of public computersBoth authorization and power are privileges to perform additionalactions without being required to do so


Social Norms

Norm Lifecycle and Operations

Conditional

Expired

Detached

Satisfied Violated

dant

ant

con con dcon

Conditional

Expired

Detached

Satisfied Violated

dant

antdcon

con con

Conditional

Expired

Detached

Satisfied Violated

dant

ant

con dcon con

commitment authorization prohibition

Norm(Subject, Object, Antecedent, Consequent)Main operations: create, detach, expire, satisfy, violateAdditional operations: suspend, resume, delegate, assign


Social Norms

Time-Aware Norms

Extend norms with deadlines

Absolute deadline: ‘Emergency physician is authorized to accesspatient records until the end of the day’

Relative deadline: ‘If the patient’s condition gets worse, physicianhas to operate within two hours’


Social Norms

Norms in the Literature

Design / revision of norms (minimise misconfiguration)Resolve conflictsMultiagent systems literature

Normative systems for artificial institutionsDelegation of control, powerDoes not go beyond conceptualization

Security norms fairly unexplored


Security Properties

Formal Properties

Liveness: achieve goals of stakeholdersRobustness: how hard to violate norms

Regarding users (privacy)Regarding data (confidentiality)

Resilience: easy recoverability from misuseFind who is responsible (traceability)Apply sanction (accountability)Required number of steps for full recovery


Security Properties

Methods for Evaluation

Formal modelsSpecification of norms in formal logicDesign time verificationRun time monitoringDiagnosis for misuse via loggingRecovery via sanctioning

Empirical techniquesGame design for human involvementUnderstanding policy complexity


Security Properties

Tradeoffs among Designs

Healthcare emergency scenarioLiveness: patient should be operated before the operationdeadlineRobustness: only authenticated physicians can access patient’sEHRResilience: access to EHR should be logged

Nonemergency scenario: operation can be delayed until anauthorized physician is availableEmergency scenario: risk of not operating immediately might beworse than most security concerns


Formal Methods and Tools

Relevant AI and MAS Literature

Agent-based simulationGame theorySocial choiceConstraintsLearningArgumentationHuman modelingMultiagent system engineering



Formal Logic

Propositional logic (atomic propositions)

First order (predicate) logic: ∀, ∃ quantifiers

∀X : physician(X ) → c(X ,hospital , treat)

Temporal logiclinear time: LTL has quantifiers over states (G, F, X, U)

treat U healthy

branching time: CTL has quantifiers over paths (A, E)



Model Checking

Verify whether a specification satisfies certain propertiesSystem behavior formalized as a transition system (FSM)Properties represented in temporal logicGenerate all possible worlds, check properties

Liveness: AG (emergency → AF access EHR)

Safety: AG (¬disclose PHI)

[Clarke et al., MIT Press, 1999]© http://www.csfieldguide.org.nz/FormalLanguages.html


http://www.csfieldguide.org.nz/FormalLanguages.html


AI Planning

How to go from an initial state to a goal state?Associate costs with actionsCome up with a plan, and report total costPlanning tools: STRIPS, HTN, SHOP

[Nau et al., JAIR, 2003]Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 35 / 40


Argumentation

Resolving Goal Conflicts

Competing hypotheses

Categories of argumentsLegal consequences

Organizational policies

Individual preferences

[Murukannaiah et al., RE, 2015]Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 36 / 40


Trust

Trustworthiness of a sociotechnical systemIndividual view points based on the states of norms

c(hospital, patient, records, protect PHI)sanction to hospitals by the government

leads to

T(patient, hospital, records, protect PHI)

[Chopra et al., ER, 2011]Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 37 / 40


Simulation and Modeling Environments

NetLogo multiagentsimulation frameworkSocial simulations withagentsUnderstand the effect ofsanction

IndividualGroup

MetricsUsabilityEffortVariance of solutions

https://ccl.northwestern.edu/netlogo/[Du et al., HotSoS, 2015]Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 38 / 40

https://ccl.northwestern.edu/netlogo/


Ontologies

OWL ontologylanguageTaxonomy of conceptsProperties for conceptsRelations amongconcepts, propertiesSimilarity metrics

http://www.w3.org/TR/owl-features/http://protege.stanford.edu/Ozgur Kafalı (NCSU) Policy Governance via Social Norms October 26, 2015 39 / 40

http://www.w3.org/TR/owl-features/

http://protege.stanford.edu/

Open Challenges

Unexplored Areas for Policy Research

Developing mathematical models for sociotechnical systemsFormalizing security propertiesUnderstanding real-life policiesValidation and dissemination of the developed models


policy governance via social...

Documents