the anatomy of a major university data breach dan … anatomy of a major university data breach dan...

8/29/2016

1

The Anatomy of a Major University Data Breach

Dan Sarazen, CISA, CISSP

2016 Annual Conference - Miami, Florida

Please Ask Questions

2016 Annual Conference - Miami Florida

8/29/2016

2

Information Security(AKA Cybersecurity)

The Information Security Triad

• Confidential is a set of rules that limits access to information• Integrity is the assurance that the information is trustworthyand accurate• Availability is a guarantee of reliable access to theinformation by authorized people.


Why We Are Here(Heroic Doses of Alarmist Rhetoric)

Goals:• Describe IT Governance, Roles, and

Responsibilities• Describe the various IT Operations and

Security Frameworks – Best Practices• Summarize the typical points of IT Security

failure in Higher Ed – Auditable items2016 ACUA Annual Conference - Miami

Florida

8/29/2016

3

Threats(Alarmist Rhetoric)

• Cyber Crime Surpassed Drug Trafficking as Criminal Money Maker in 2008• The average cost of a data breach hit $4 million, representing a 29 percent

increase since 2013, according to IBM Security (6/15/16)• That's approximately $158 for every lost or stolen record. In highly

regulated industries like healthcare, the cost of a breach can be as muchas $355 per record, $100 more than in 2013, the report said.

• US Intellectual Property @ $100 Billion annually; World Economy (2014)$445,000,000,000 (CSIS)

• Cost to Reputation• Sony• RSA• Hollywood Presbyterian Medical Center - February 2016, paid $17,000 ransom via

Bitcoin transfer after patient systems controlled by malware.


In September 2008, an attacker gained unauthorizedaccess to a server that contained the Social Securitynumbers of over 231,000 students and alumni (classesof 1982-2002), and a small number of credit cardnumbers. Records show the principal vulnerabilityoccurred over two days, from Sept. 15-16, 2008, withthe exposure extending until Oct. 24, 2008.The official public notification did not take place untilAugust 5, 2009. On August 21st University AuditDirector read about the breach in a statewidenewspaper.


8/29/2016

4

The Ultimate Responsibility forInformation Security Falls Upon……..

A. Data UsersB. System AdministratorsC. The Chief Information Security OfficerD. The Board of TrusteesE. All of the Above


Information SecurityPolicy

• Policy:• Board Approved• That requires on-going risk assessments and analysis *against a

recognized IT Security Framework (e.g., ISO, NIST 800-53, HIPAA,PCI DSS, etc.)

• Deliverable to Audit Committee (e.g., Risk Scorecard)• Authorizes a specific entity (usually the President’s office) to

established formal procedures, standards and guidelines


8/29/2016

5

Information Security FrameworksWhat do they look like?



8/29/2016

6

Risk Assessmentso Identify the critical assets to the Organization

o This is management’s duty, in conjunction with Business Continuity planningo What systems support key institutional processes?

o Using a Risk Assessment Framework (e.g., SANS)o Operations Self-Assess each controlo Document Resultso Report on existing known vulnerabilities to Audit Committeeo Management determines which risks are acceptable, based on budgeto Management puts mitigating controls into effect

o Short-term planning (e.g., 1 year)o Long-term planning (e.g., 3 years)

o Repeat



8/29/2016

7

UMass Breach• Information Security Policy was over 500 pages long

– Not approved by the BOT; no means of approving edits• Not based on a recognized framework• Liberal use of circular referencing• Not promoted through awareness training• Hacked department not aware of the existence of the Policies• Most departments, if they knew of the policy, hadn’t bothered to

read it• Useless

– Of the (then) 133 ISO27002 controls, 27 could be identified in thevoluminous documentation, much of which was contradictory


Auditable Item #1Information Security Policy

• Does your institution have an Information SecurityPolicy (PCI DSS requires it)?

• What’s it based on? (e.g., ISO27002, NIST 800)• Is it measurable? (e.g., Risk Assessments)• Who approved it? Was it the BOT? Were they at least

advised (e.g., President’s Council, Audit Committee)• If not the BOT, are there any segregation of duties issue

based on the approver?• Is the Policy Communicated?


8/29/2016

8

University of Massachusetts Current InformationSecurity Policy

• Not quite a page and a half long.• Based on ISO 27002

– Adopted a NIST framework• Authorizes the President’s Office to develop standards• Applies to everyone (Staff, students, faculty, venders,

etc.)• Approved at public meeting by the Board of Trustees

12/8/10• SANS conducts on-going University risk assessments


Security Awareness Conducted (Everyone)• 80% Operationally Specific Training for Key IT Staff

• Firewalls, Vulnerability Scanning, Log Management, etc.• 20% General Awareness Training for Everyone

• PII, Phishing, Spear Phishing2016 Annual Conference - Miami Florida

8/29/2016

9

IT Operational ProceduresFormal, written documentation for operationalprocedures, including the controls discussed today, shouldexist.These procedures are reviewed and approved byresponsible Management annually, or as procedureschange.Audit 101


Continuing Ed. (IT Professionals)

• Certifications and Associations• SANS.org• (ISC)2 (CISSP)• ISACA (CISA, CISM, CRISK)


8/29/2016

10

Information Security Officer• Must be appointed by most state privacy laws

(201 CMR 17.00)• Are they trained/certified (CISA, CISM,

CISSP?)• Do they have authority?• Is it the Network Director (SOD Issue)• An IT Auditor’s Best Friend


UMass Breach• Neither the Campus CIO or ISO had authority over decentralized

department server (Career Services)• The “Server Engineer” was a business analysts in possession of a

server used to manage the department’s webpage• No IT Security or Awareness training• No Certifications (CISA, CISM, CISSP)• No written procedures• Only @ 5% of his time was devoted to server activity• At the time UMass Amherst had 27 separate “data centers”,

staffed by over 150 employees, servicing @550 servers.


8/29/2016

11

Auditable Item #2Written Procedures & Training

• Have ISO duties been assigned– Any segregation of duties issue? (e.g., ISO/Network Director)

• Are there written procedures for IT activities? (e.g., back-ups,patching, server hardening, IT Asset Inventories)

• Have the procedures been reviewed and approved bymanagement?

• Does the IT personnel have any industry certifications? Training?• Should they be conducting these duties in the first place? Is there a

business need?


Data Inventories/CategorizationsThe Critical First Step in Protecting Data

• Personally Identifiable Information (PII) is key:• SSN’s, W9s, CC#s, HIPAA, license plates and numbers, DNA, etc.

• Data Inventories• Hard copy - Manual• Electronic data - Automated (e.g., Identity Finder)

• Protected Based on Category• Retention Schedules Established and Followed• Non-essential Data Appropriately Destroyed• Clean Systems Maintenance (User Awareness)


8/29/2016

12

(Physical) Asset Inventories/Management(The critical second first step)

• Olden days: Manual (Work-study students andclipboards)

• Automated - IT Inventory software• Based on IP/MAC Address pairings• On-going inventory reconciliations

• Know when a device, and the data it holds, is unaccounted.

• Disposal of device and removal from inventory• Documentation Maintained


Intrusion Detection/Prevention Systems(IDS)

•Exist•Monitored•Periodically Tested•All Appropriate Audit LoggingEnabled and Secured in a logmanagement system


8/29/2016

13

UMass Breach• The Department had PII (that it didn’t need), and had forgotten about it• That PII sat for 6 years, unused, before it was breached• IP Address physical campus locations were not know• OIT Identifies intrusion during scheduled scanning operations on 9/17/08, 48

hours into the breach• But, they don’t know where the server is• They take the server off the network by disabling its network jack• They leave a message at the help desk for the department to contact the ISO when

they call to get server back on-line• What happened on 9/18/2008?• OIT AGAIN identifies intrusion during scheduled scanning operations on 10/24/08


Auditable Items #3Data and Asset Inventories

• Are there data inventories? Automated?• Has PII identified, but not required for business purposes, been deleted/

destroyed? How?• Has PII identified and required been risk assessed? (e.g., PCI DSS SAQ)• Are records retention schedules being followed?• Has the campus automated their device asset inventory?• Can the campus match IP Addresses to the devices physical locations?• How are obsolete devices sanitized, removed from the official inventory

and appropriately decommissioned? (e.g., destroyed, sold, donated)• And what’s in your cloud? (SalesForce, etc?)


8/29/2016

14

Logical AccessIn information technology, logical access controls are toolsand protocols used for identification, authentication,authorization, and accountability in computer informationsystems.

To:• Servers• Operating Systems• Applications, including:

• Department Specific Apps• Databases• Firewalls• Back-up devices• Etc.


User IDs/Passwords• Exist• IDs based on a consistent naming

convention (i.e.: first initial/last name,employee number, Employee #)

• Specific to user (No “User1”, “Admin” or“Joe”)

• Default accounts disabled


8/29/2016

15

Passwords (Continued)Active Directory/Applications/Servers Configured to:

• Regularly Force Change (every 180 to 365 days)• Be Complex: Alphanumeric, Upper and Lower case, Include

Symbols, minimum 8 characters• Lock the ID out after X failed attempts, for Y minutes

• Two-factor authentication for privileged accounts• Server, Firewall, Network Administrators• Some key financial applications (e.g., accounts with General

Ledger override ability, bank wires, )• Not Shared!


Screensaver/Password Configuration• Password required to access timed-out, active

session• 15 Minutes maximum timeout period• Controlled at Domain level (vs. desktop/laptop)

so users cannot disable or increase the timeoutperiod (Users shouldn’t have Admin rights…in aperfect world)


8/29/2016

16

Administrator Access(Servers/OS/Applications/Firewalls)

• Specific (1 ID assigned to each administrator)• Access is Limited – Assigned based on job

responsibilities• Default Administrator Accounts Passwords

Changed/Disabled (TJX)• Includes Local Administrator Privileges

(desktops/laptops)• Disabled PRIOR to employee separation


Anti-Virus (Servers and Desktops)• Anti-Virus Application Configured to

look for and update new virusdefinitions daily

• Monitored - Applications can beconfigured to automatically notify keypersonal (via email, text message) whenevents are identified


8/29/2016

17

Encrypting Confidential Information

• Required by Massachusetts Privacy Law• “At Rest” on portable devices (Back-ups,

Laptops, Desktops, flash drives, cd’s, etc.)• “In Transit” (Email, remote log-in, remote

back-up, wireless, HTTPS, etc.)• 128 bit minimum, most at 256


UMass Breach• Anti-Virus did not detect the intrusion (40% at best)• Server Engineer did not use two-factor authentication

to protect the administrator credentials (ID/Password)• Server Audit logs stored on the server• Network logs needed to confirm or refute data

exfiltration were overwritten after 2 weeks.• PII not encrypted (why would it be? They didn’t know it

was there)


8/29/2016

18

Auditable Items #4• Are Logical Access controls (password configurations,

screensaver timeout, etc) enforced?• Is there a unique system ID for each user?• Do users have administrator access to their institution

issued devices? (If so, they can negate the controls)• Is Antivirus installed on all servers and desktops?• Is it configure to regularly (e.g., daily) update its

definitions?• IS PII encrypted wherever it can be? Especially portable

devices and in transit?


Auditable Items #4 (Continued)• What audit logs are being saved? For how long?• Where are they being saved?• Who has access to them?• Are audit logs monitored?• How are audit logs monitored?• Is there a Log Management System in place?

SEIM (Security Event Information Managementsystem)?


8/29/2016

19

New User Access/Joiners(Including Transfers/Movers)

• Approved by Authorized Supervisor• Access based on defined, pre-approved

Profiles (Not “cloned” access from activeusers)

• Documentation Saved• Track-it• Email


Terminated User Access/Leavers Regular Communications from

HR/Management to System/ApplicationAdministrators of Terminated users Access Promptly Deleted/Disabled (Within 2

business days for normal termination,Immediate for special circumstances)

• Example: The LendingTree failed to remove Administratoraccess for employees who had terminated. Access abusedfrom 2006 through 2008, ten’s of thousands of files (SSN’s)compromised.


8/29/2016

20

User Access and Profile Reviews Mitigating Control to remove users missed in the

termination/transfer process and eliminate excessiveprivileges Regularly Scheduled Based on System Generated Reports User Access and Profiles Reviewed and Approved by

Management Changes made by System/Application Administrator and

Documentation Saved (email) Changes should only remove access


Physical Access & EnvironmentalControls

• Data Center – Locked at all Times with accessmonitored (Proximity Card Reader)

• Only Approved users have access to the DataCenter/Scheduled Access Reviews

• HVAC/Humidity Controls• Fire Extinguisher(s)• Smoke Detection/Fire Suppression• Visitor Access Logging


8/29/2016

21

Back-upsOS/Applications/Data

• Happen (Daily, Weekly, Monthly)• Monitored for Completion• Access to back-up media restricted• Back-up media stored off-site• Retention periods assigned• Scheduled Restoration Testing


Change Control(O/S, Applications, Servers, Configurations, etc.)

• Requests are approved By Management prior to initiation ofprojects / Tracked

• Acquisition of IT equipment and services approved by appropriatelevel of IT Management

• Changes are tested in a development environment (including UserAcceptance Testing, when appropriate)

• Final Management approval prior to move to productionenvironment

• Data Migrations are tested for accuracy• Back-out Plan• Emergency Change Procedures (Communication)


8/29/2016

22

Server Patching/Hardening• Procedures defined for critical and important

patches• O/S is supported (Lifecycle)

• Windows Server 2008? Manistream supportended in 1/15. Extended support must bepurchased.

• Anything older?• Only essential programs are loaded on the

server (e.g., No media players, web toolbars)


Firewalls (Hardware and Software)• Exist• Deny Default: Everything, not explicitly

permitted, is forbidden• Authorization required to change rule-set• Scheduled reviews of rule-sets• Obsolete access points closed• Updating services must be contracted• Administrators trained


8/29/2016

23

Incident Response Procedures(When everything failed)

• A formal written plan• That has been tested• Identifies who to contact if a potential intrusion is

identified• States who is in charge?• Identifies what logs needed?• Document. Document. Document.• Breach insurance• Bitcoin broker on file


UMass Breach• The “Server” was under a desk• The “Server Engineer” migrated business services to a new server 3

years prior to the intrusion….moved the unneeded PII too• Web service (Yahoo Web Browser) toolbar installed on Web

browser….of the server…from which you shouldn’t browse the web• Windows Media Player Installed on “Server”• Server O/S Unsupported (Windows 2000…installed 2007)• No Incident Response Plan…Nobody knew what to do


8/29/2016

24

UMass Breach (cont’d)• Breach investigation stalled for months at a time• System was “owned” for six weeks AFTER the

initial identification of the intrusion• Formal notice of breach occurs 10 months after

the campus confirmed intrusion• Statewide Editorial criticizes the system’s delayed

response time with lead editorial…• Audit Director screams for Dan


Auditable Items #5• Is the server secured in a data center?• Is the O/S being used supported by the vendor?• Does the campus have hardening standards for

the servers?• Is there an incident response plan?• Does everyone know what to do? And how to do

it?• Has it been tested?


8/29/2016

25

Auditable Items #5 (Cont’d)• Does it name forensic services that are contracted?• Does it name legal counsel (They will determine if the

legal definition of a breach has been met)• Does it appoint communication responsibilities?• Do you have breach insurance?• If medical patients are involved, have you identified a

Bitcoin broker?• Does it require a port-mortem (e.g., audit report)


Timeline of Information Security Events leading up to the Incident:• 2002 – PII is placed on the department server. Career Services imbeds an Oracle

Crosswalk table, which includes Social Security numbers and campus IDs. The tablewas used eight (8) months, but not beyond 2002.

• 2/2007 – 3rd party credit card data placed on department servers. As part of adepartment workaround, Career Services begins processing credit cardinformation for vendors of the campus “Career Fairs.”

• 6/7/2007 – Department server CAREER_SERVER0 is built and Windows Server2000 Operating System (OS), SP4 is installed. Files, including files containing PII, aremigrated from the decommissioned server to the new server.

• 12/7/2007 – Departments are informed of sensitive data requirements. OITprovides explanation of Sensitive Data Inventories and advises on process viamemo. The department did not inventory its PII.

• 9/10/2008 – The last security patch for departmental server, prior to event, wasinstalled. The server was configured for automated updates; however, WindowsServer 2000 mainstream support was retired 6/30/2005. Subsequent support wasavailable only through extended support, which was not purchased


8/29/2016

26

Timeline of Incident Response Events:• 9/15/2008 – 11:03:35AM attack is initiated and malware installed.• 9/16/2008 – 88% of the 1,448 files containing social security

numbers are accessed.• 9/17/2008 – OIT’s periodic review of NetFlow logs identified

suspicious activity, and the server is removed from the network. Nofurther incident response occurs from this activity. OIT does nothave an inventory of campus wide servers and their IP address,which would allow for timely notification. Identifying the servers’owners and operators on the Amherst Campus requires research.These are manual steps, and they were not followed.

• 9/18/2008 – Career Services reconnects server to the internet byswitching its port.


Timeline of Incident Response Events (Cont’d):

• 9/18/2008 – The server Administrator ID and Password arecaptured by installed malware.

• 10/15/2008 through 10/27/2008 – First two weeks of incident -data purged. This date is an estimate. The first two weeks ofNetFlow logs during incident period are unavailable, as files weredeleted due to space constants.

• 10/24/2008 – OIT’s periodic review of NetFlow logs again identifiessuspicious activity and removes server from the network. The OITstaff communicates with the Career Services Systems Administratorand starts to follow documented incident response procedures.

• 10/27/2008 – OIT images (makes a duplicate copy) server.


8/29/2016

27

Timeline of Incident Notification Events (Cont’d):• 12/1/2008 – OIT completes initial forensic analysis of server image, noting the existence

of at least 3,000 social security numbers and 59 credit card numbers; almost all of thecredit cards had expired prior to the breach. Career Services is asked to verify and toprovide identification of individuals whose SSNs have been identified.

• 2/25/2009 – Meeting between OIT, Student Affairs and Legal Counsel takes place.• 3/1/2009 – Amherst CIO notifies President’s Office CIO of event.• 4/2/2009 – Follow-up meeting with CIO and Acting VC for Student Affairs.• 4/15/2009 – Amherst CIO requests follow-up meeting. Known PII on server image

reaches approximately 10,000 Social Security numbers. Additional known PII data isbased on additional forensic data gathered, as conducted by OIT

• 4/17/2009 – Career Services completes its assessment of data on its server, confirmingthe presence of PII data, but not quantifying the PII data.


Timeline of Incident Notification Events (Cont’d):• 5/1/2009 – Assistant Director of Student Affairs IT identifies 230,000+ Social Security

numbers on server image.• 5/7/2009 – Department files police report.• 5/8/2009 – OIT engages 3rd party firm (Stroz Friedberg) for independent assessment.• 7/29/2009 – 3rd party forensic firm issues their report. They confirm the presence of PII

on the compromised server, stating that a definitive statement regarding whether thePII was accessed by the attacker, and exfiltrated, was not possible.

• 7/30/2009 – 3rd Party forensic report reviewed by CIO, Legal Counsel and Executive VCfor University Relations.

• 8/5/2009 – The campus posts notice of intrusion to its web site.• 8/7/2009 – 8/21/2009 – Notice of intrusion is posted in statewide newspapers.• 9/9/2009 – Microsoft announced that it would not patch certain known vulnerabilities

to the Windows 2000 Server OS, because the architecture to properly support TCP/IP(Transmission Control Protocol/Internet Protocol) protection does not exist onMicrosoft Windows 2000 systems, making it infeasible to develop a patch/fix.


8/29/2016

28

Contact Information

Dan Sarazen, CISA, CISSPSenior IT AuditorThe Boston Consortium for Higher EducationInternal Audit [email protected]


the anatomy of a major university data breach dan … anatomy of a major university data breach dan...

Documents