ibm security guardium v9.5 features and updates tech …€¦ · · 2015-03-24value proposition...

© 2015 IBM Corporation

IBM Security

1© 2015 IBM Corporation

IBM Security Guardium v9.5

Features and Updates Tech Talk

Luis Casco-AriasProduct Manager

IBM Security Guardium

Also with support from: Guy Galil, Lior Solomon and Oded Sofer


IBM Security

2

This tech talk is being recorded. If you object, please hang up and

leave the webcast now.

We’ll post a copy of slides and link to recording on the Guardium

community tech talk wiki page: http://ibm.co/Wh9x0o

You can listen to the tech talk using audiocast and ask questions in

the chat to the Q and A group.

We’ll try to answer questions in the chat or address them at

speaker’s discretion.

– If we cannot answer your question, please do include your email

so we can get back to you.

When speaker pauses for questions:

– We’ll go through existing questions in the chat

Logistics

http://ibm.co/Wh9x0o


IBM Security

3

Link to more information about this and upcoming tech talks can be found on the InfoSphere Guardium developerWorks

community: http://ibm.co/Wh9x0o

Please submit a comment on this page for ideas for tech talk topics.

April 7th, 2015: Encrypting Data at Rest in IMS using InfoSphere Guardium Data Encryption

Speaker: Dennis Eichelberger, IMS IT Specialist, IBM

Register here! https://ibm.biz/BdE7tR

Reminder: Upcoming Guardium Tech Talks

April 22nd: Part 2 – Overview of InfoSphere Guardium Data Encryption

Speaker: Ernie Mancill, Executive IT Specialist, IBM

Register here! https://ibm.biz/BdXxhx


https://ibm.biz/BdE7tR

https://ibm.biz/BdXxhx


IBM Security

4

Guardium community on developerWorks

bit.ly/guardwiki

Right nav


IBM Security

55

Information, training, and community

InfoSphere Guardium Tech Talks – at least one per month. Suggestions welcome!

InfoSphere Guardium YouTube Channel – includes overviews, technical demos, tech talk replays

developerWorks forum (very active)

Guardium DAM User Group on Linked In (very active)

Community on developerWorks (includes discussion forum, content and links to a myriad of sources, developerWorks articles, tech talk materials and schedules)

Guardium on IBM Knowledge Center (was Info Center)

Deployment Guide for InfoSphere Guardium Red Book

Technical training courses (provided by IBM Business Partners)

InfoSphere Guardium Virtual User Group. Open, technical

discussions with other users. Not recorded!

Send a note to [email protected] if interested.

5


http://www.youtube.com/user/InfoSphereGuardium

http://www.ibm.com/developerworks/forums/forum.jspa?forumID=2648&start=0

http://www.linkedin.com/groups/IBM-Guardium-Database-DB-Activity-3746392

https://www.ibm.com/developerworks/mydeveloperworks/groups/service/html/communityview?communityUuid=432a9382-b250-4e55-98d7-8e9ee6cbf90e

http://www-01.ibm.com/support/knowledgecenter/SSMPHH/SSMPHH_welcome.html

http://www.redbooks.ibm.com/abstracts/sg248129.html?Open

http://www-01.ibm.com/software/data/education/guardium.html


IBM Security

6

Agenda

Guardium 9.5 ContentNew Platforms S-TAP automatic load balancer

• New GuardAPI commands

Quick Search Infrastructure Update •Enterprise Scope •Investigation Dashboard •Topology Navigator

Outlier Detection enhancementUpgrade path recommendations

Demo


IBM Security

7

InfoSphere Guardium – Data Security & Privacyone-stop-shop to protect against unauthorized access to data, and reduce the cost of compliance

Value Proposition•Prevent data breaches•Ensure data Privacy•Reduce the cost of compliance•Identifying security risks•Enable secure sharing of data

On Premise On Cloud

Data at Rest

Stored(Databases, File Servers, Big Data, Data

Warehouses, Application Servers, Cloud/Virtual ..)

Over Network(SQL, HTTP, SSH, FTP, email,. …)

Data in Motion

Data Repositories

Sensitive Documents


IBM Security

88

Expand Platform Coverage

InfoSphere Guardium v9.5 (1Q15)

DAM pricing and packaging changes

•DAM Big Data new part

•DAM Data Warehouse new part New Platform Support: AsterDB, Pivotal, MariaDB, MS APD/PDW

Updates: Teradata 15, GreenplumDB 4.3, Ubuntu 14

Enhance

Activity Monitoring

Scalability &Reduce Op. Costs

Automatic STAP load balancing (phase 1)

Quick Search analytics improvement

•Investigation Dashboard

Outlier Detection performance and scoring

New monthly based pricing

Cloud deployment

Security Integrations New GuardAPI commands

VulnerabilityAssessment

VA parts consolidation

Entitlement Reporting included in base services

Monitoring Teradata encrypted logon and traffic

(A-TAP)

* Adjust appliance pricing currency


IBM Security

9

InfoSphere Guardium v9.5New Platforms


IBM Security

10

Comprehensive support for structured and unstructured sensitive data:Databases, Data Warehouses, Big Data Environments and File Shares

InfoSphere BigInsights

Guardium

DATABASES

Exadata

D A T A B A S E

HANA

Optim

Archival

Siebel,

PeopleSoft,

E-Business

Master Data

Management

Data

Stage

CICS

z/OS Datasets

Pure Data Analytics

FTP

with BLU Acceleration

DB2®

with BLU Acceleration

DB2®

DB


IBM Security

11

Netezza

Teradata

2011

Netezza

Teradata

BigInsights

Cloudera

2012

Netezza

Teradata

BigInsights

Cloudera

MongoDB

CouchDB

Cassandra

GreenplumHD

HortonWorks

2013

SAP/HANA

GreenplumDB

V8

V9

V9p50

V9.1

Netezza

Teradata

BigInsights

Cloudera

MongoDB

CouchDB

Cassandra

GreenplumHD

HortonWorks

2014

SAP/HANA

GreenplumDB

AsterDB

Pivotal

V 9.1 updates

Platform support Updates in v9.5

• Greenplum DB 4.3 • Hortonworks HDP 2.2 •Teradata 15

New Platform Support in v9.5 :

• Aster DB• MariaDB• MS APD/PDW• Pivotal

OS Support Updates in v9.5

•Ubuntu 14

New Platforms and Updates in v9.5


IBM Security

12

InfoSphere Guardium v9.5STAP Load Balancing (Phase 1)


IBM Security

13

Installing an STAP on a DB server requires the customer to assign a managed unit

appliance for STAP to connect to.

On large sites (> 100’s of managed units), finding an available managed unit requires

skill and time.

Dedicating a collector to an STAP involves periodically consolidating several

indicators (per collector).

The 'STAP Load Balancer’ automates this process

Overview


IBM Security

14

Load Balancer Process Running on the Central Manager

Collects and maintains a 'Load Map' of all the Managed Units (MU).

Allocating least loaded Managed Unit to newly installed STAP(s).

CM

Load

Balancer

MU 3

MU 2

MU 1

DB

Server

Load MapMU 1 : LoadedMU 2 : VacantMU 3 : Loaded

Report load

STEP 2 : Allocate MU Request

STEP 4 : Connect to MU 2

STEP 1 : Install STAP

STEP 3 : Find least loaded MU

Periodic background activity

*MU generally represents a Collector

Load Balancer Highlights


IBM Security

15

Load Collection Mechanism

Periodically collects load information from all the managed units.

Load information relies mostly on 'UNIT UTILIZATION LEVELS' and

'STAP INFO' reports/Tables.

Load Map

Specific load indicators are used to build a 'Load Map' in the Load

Balancer memory.

MU Allocation

Upon receiving a request from STAP to allocate an MU, the Load

Balancer will query the Load Map, and assign an MU for the requesting

STAP.

Load Balancer Function - General


IBM Security

16

Default dynamic load collection Intervals.

The more MU‘s, the bigger the load collection intervals. (Controlled by the

parameter DYNAMIC_LOAD_CHECK_INTERVAL)

Load indicators are used in order to estimate the load level on each MU over

a period of time.

Load indicators are defined by the STAP Load Balancer parameter

LOAD_INDICATORS which is set by default with the following indicators:NO_OF_RESTARTS_UL (Number of Sniffer restart)

ANALYZER_QUEUE_UL (Size of Sniffer analyzer queue)

LOGGER_QUEUE_UL (Size of Sniffer logger queue)

MYSQL_DISK_USAGE_UL (Disk usage)

Load Sampling Interval is critical in order to get an accurate picture of the

load per MU. Larger load sampling interval means we can capture a more accurate snapshot of

load behavior (busy and non-busy hours) rather than capturing spikes.

Load indicators are defined by the STAP Load Balancer parameter

LOAD_SAMPLE_PERIOD which is set by default to 24 hours.

Load Balancer Function – Load Collection


IBM Security

17

After evaluating the load from all load indicators, MU's are placed within the Load Map

in 3 load “levels” : NOT-LOADED (LOAD_LEVEL_1),

MEDIUM-LOADED (LOAD_LEVEL_2),

VERY-LOADED (LOAD_LEVEL_3).

In between load collections, the Load Map is dynamically updated upon each

allocation of an MU to a requesting STAP.

MU's will be replaced in a new load “level” if the number of STAPs per single MU

exceeds the threshold defined by the Load Balancer parameters: MAX_STAPS_PER_MU_THRESHOLD1 (default : 20), MAX_STAPS_PER_MU_THRESHOLD2 (default :40)

> grdapi get_stap_load_balancer_current_load_mapLOAD_MAP:

LOAD_LEVEL_1 MU's:

MU=qa-vm18.guard.swg.usma.ibm.com_5: C_STAPS=1, I_STAPS=0, C_LOAD_LEVEL=1, I_LOAD_LEVEL=1-->


LOAD_LEVEL_2 MU's:




LOAD_LEVEL_3 MU's:




STAP REQUESTS CACHE:

1.1.1.1=1

LAST MU USED PER STAP CACHE:

1.1.1.1=qa-vm18.guard.swg.usma.ibm.com_6

C_STAPS : Current # STAPS assigned to MUI_STAPS : Initial # STAPS assigned to MUC_LOAD_LEVEL: Current Load Level of MUI_LOAD_LEVEL : Initial # load level of MU

Load Balancer Function – Load Map


IBM Security

18

Load Balancer Function - Allocating MU's The Load balancer allows usage of groups in order to assign a pool of STAP(s) to a pool of MU.

Guarantee that not every available MU can be assigned to a requesting STAP. Caveat: Have to define an STAP IP in some group prior to installing it on the DB server

Upon receiving an MU allocation request from STAP, the load balancer will:1. Check if it's assigned to some group of MU. 2. If it find such a group, it will look in the load map for the least loaded MU in that

group. 3. Otherwise, the first available MU in the load map will be allocated.

Once an MU is allocated, the load balancer caches the last allocated MU for the requesting STAP.

If an additional request(s) comes from the same STAP, the load balancer will allocate a different MU (if there is one).

Allocated MU's are re-positioned at the end of Load Map “bucket” queue.If the allocated MU is of LOAD_LEVEL “2” or “3” (“Medium/High Load”), an event is going to be recorded in GDM_EXCEPTION table.


IBM Security

19

STAP Load Balancer GRDAPI

Assign MU group to an STAP group

grdapi assign_stap_load_balancer_mu_tap_group muGroupName=<MU group>

stapGroupName=<STAP group>

Get Current Load Map

grdapi get_stap_load_balancer_current_load_map

Get Load Balancer parameters

grdapi get_stap_load_balancer_params

Set load balancer parameter

grdapi set_stap_load_balancer_param paramName=<param name>

paramValue=<param value>

Un-assign STAP group from an MU group

grdapi unassign_stap_load_balancer_mu_tap_group muGroupName=<MU

group> stapGroupName=<STAP group>


IBM Security

20

STAP Load Balancer GUI – Associating Existing Groups

All Groups of type 'STAP' will be automatically displayed

All Groups of type 'Managed Unit' will be automatically displayed


IBM Security

21

STAP Load Balancer GUI – Associating New STAP Group

A List of available STAP hosts will be automatically available for selection.


IBM Security

22

STAP Load Balancer GUI – Associating New MU Group

All the managed units hosts will be automatically displayed


IBM Security

23

STAP Load Balancer GUI – Associating completion


IBM Security

24

STAP Load Balancer GUI – Disassociating Groups


IBM Security

25

InfoSphere Guardium v9.5Quick Search Updates


IBM Security

26

Quick Search Infrastructure UpdateContinue to improve Quick Search Functionality in Collector environment, change infrastructure from

Lucene to Solr

Benefits :

Real Time Distributed Search

Performance Improvements

Built-in Analytics functions

Caveat :

This engine change may consume extra memory (Requires 24GB of RAM then 16 GB previously)

Investigation Dashboard – Technology PreviewProvides interrelated charts that help reveal patterns, anomalies, and relationships across your data

Best-practice view includes data source-to-user behavior, data source-by-time behavior, data source-to-source

program behavior, and other essential relationships.


IBM Security

27

Enhancing Quick Search for Enterprise Wide Scope

• Search scope spanning Central Manager controlled environment– No need to understand or concern about topology, aggregation, or load balancing

schemes

– Search requests are sent to all nodes, results are gathered, consolidated, and sorted

according to the request, and then the results are displayed to the user centrally

• Leverage familiar Quick Search glass– Real-time

– Forensic

• New additional Investigation Dashboard– Pivot like facility to extract data activity insights

• focus on any specific context :specific data source, user, date, etc

– reveal patterns, anomalies, and relationships across your data

– best-practice default views:

• data source-to-user behavior

• data source-by-time behavior

• data source-to-source program behavior

• other essential relationships.

• New topology navigation facilitator– Narrowing of scope for search (local, distributed, selected sub-set)

Coming

soon


IBM Security

28

Quick Search – Enterprise scope


IBM Security

29

Topology Navigation


IBM Security

30

Active Filters in Quick Search


IBM Security

31

Investigation Dashboard Overview

Business Goals

– Ability to grasp high-level view in a multi-dimensonal environment

– Quick way to inspect different aspects of a forensic case

– Browsing audit data related to a specific context

Investigation Dashboard Solution

– A dashboard containing a combination of interactive heat maps

– Leverage fast indexing engine in Quick Search

– Each heat map offers two dimensional view , related to other two dimensional

views

– Each change filters the other charts to drill down on a specific case

– Highly configurable


IBM Security

32

Investigation Dashboard

•X-axis contains DB users•Y axis contains databases•Intersection shows usage of DB by each user•Color depth represents Intensity of usage•Hover cells for details•Click for interactive filtering


IBM Security

33

InfoSphere Guardium v9.5Outlier Detection updates


IBM Security

34

Enhancements related to :

• New Features • New scorer

• Performance Improvement • User clustering• Multi-Threaded training• New cleaning mechanism for old data from the internal MySQL tables

• Fixes• Improved anomaly scoring• Filtering inputs according to the filtering patterns specified by the user in the GUI• Inputs collected during training are now analyzed

Outlier Detection


IBM Security

35

InfoSphere Guardium v9.5Upgrade paths


IBM Security

36

Transition to Guardium V9.0p500 GPU

Target

Source

32-bit 32-bit 32-bit 64-bit 64-bit 64-bit

V8.2 V8.2 to V9.0p150 bundle patch

(IBM Fix Central) + V9.0p200 (or

later) 32-bit GPU patch

Rebuild with V9.0p200 (or later) 64-bit ISO

(IBM Passport Advantage)

N/A

V9.0 (GA) V9.0p200 (or later) 32-bit GPU patch

(IBM Fix Central)



N/A

V9.0p02 V9.0p200 (or later) 32-bit GPU patch

(IBM Fix Central)



N/A

V9.0p50 V9.0p200 (or later) 32-bit GPU patch

(IBM Fix Central)



V9.0p200 (or later) 64-bit GPU

patch

(IBM Fix Central)

>V9.0p300 V9.0p300 (or later) 32-bit GPU patch

(IBM Fix Central)



V9.0p300 (or later) 64-bit GPU

patch

(IBM Fix Central)

Newly built

appliance

Install V9.0p180 32bit ISO +

V9.0p200 (or later) 32-bit GPU patch

N/A V9.0p200 (or later) 64-bit GPU

patch

(IBM Fix Central)

Enterprise Upgrade Strategy


IBM Security

37

Central Managerlevel

Collectorslevel

Aggregators level

Guardium agentslevel

Upgrading 64-bit system

V9.5p500 (or later) 64-bit

V9.1p300 64-bit

Upgrade

Upgrade

Live Update

Upgrade

Enterprise Upgrade Strategy


IBM Security

38

External link for information on Guardium p500 GPU

http://www-01.ibm.com/support/docview.wss?&uid=swg27045362

Make the following selections on Fix Central:

Product Group: Information Management

Product: InfoSphere Guardium

Installed Version: 9.0/9.5

Platform: Linux

Heading: Appliance Patch (GPU and Ad-hoc)

Click "Continue", then select "Browse for fixes" and click "Continue" again.

How to Access p500 GPU

http://www-01.ibm.com/support/docview.wss?&uid=swg27045362


IBM Security

40

Enterprise (and local) Search is based on Apache Solr

A widely used, highly scalable, open source

enterprise search platform from Apache.

Solr runs as a separate web application under tomcat.

Data is indexed and searched on shards.

A zookeeper is responsible for distributing the

indexing and the search queries to the relevant shards.

–Zookeeper runs on CM in a managed environment.

Search Infrastructure Update


IBM Security

41

On a Guardium collector, data is extracted every 2 minutes

using the same mechanism (datamart) that was used in

previous Quick Search.

This data is indexed into 5 indexes.

Indexing is performed through the zookeeper –

Note that if CM is down indexing is not performed.

The actual index is local to the collector that collects the

data.

Old data is purged using the Purge Object mechanism,

default age is 3 days.

CLI command can be used to modify that age.

Data Flow


IBM Security

42

On upgrade of a Guardium appliance to GPU500, if Quick

Search is enabled, hardware requirements for Enterprise

Search are met and old (Lucene) indexes are present there is

a transition period.

To prevent data loss in the transition period, data is indexed

on both the old engine and the new one.

Searches are performed using old engine.

A clear message is displayed stating "Upcoming new search

options, in ${date}."

The data is calculated to be the date when old index data is

purged and all data is indexed by new engine.

Transitioning to new Search


IBM Security

43

Enterprise Search is not supported on 32 bit

Collectors.

A 32 bit CM can be the zookeeper when it manages

64 bit collectors as long as it complies with the

following HW requirements.

The hardware requirements for Enterprise Search are at

least 4 CPU cores and 24 Gig RAM on a collector

On a CM 4 CPU cores and 24 Gig RAM are required.

Hardware Requirements


IBM Security

44

Solr uses port 8983 for communications. This port must be

bidirectionally open between CM and MUs.

The ports are opened in our internal firewall on registration to

CM and closed on un-registration.

The port is closed to the world by default to preserve the

security of our system.

search data.

Ports


IBM Security

45

cli>grdapi enable_quick_search – if hardware requirements

are met then new search engine is enabled.

cli>grdapi disable_quick_search – Quick Search is disabled.

A new optional parameter all – if api function is invoked on

CM and all=true then Quick Search is disabled on all

managed units.

CLI commands


IBM Security

47

Gracias

Merci

Grazie

ObrigadoDanke

Japanese

French

Russian

German

Italian

Spanish

Brazilian Portuguese

Arabic

Traditional Chinese

Simplified Chinese

Thai

TackSwedish

Danke

DziękujęPolish