2012-03-15 what's new at red hat

What's New At Red Hat

15-MAR-2012

0900 - 1030

Shawn WellsTechnical Director, U.S. Intelligence Programsshawn@redhat.com443-534-0130

Things we get asked . . .

● “As an analyst, I would like to pull up a fused timeline, social network diagram, or virtual dossier of structured and unstructured data on a particular person or entity or a group of individuals.”

● “I would like to search social media data from foreign social media sites in native language for particular hash tags, keywords, or individual account histories”

● “As a product owner, I would like the cloud to ingest, index, and extract entities from 42,000 documents per hour (hitting our 1m docs/day requirement), with a high degree of precision and recall”

● “As an analyst, I would like the ability to upload structured data into a data layer easily, have that data available to run statistical algorithms against, and have those algorithms run in a cloud”

● “As an analyst, I would like a repository to store files, that is not my office share drive, and with which I could send links to files, vice attachments, in my lotus notes”

Things We Do

● Ingest/Egress of Data● JBoss Messaging● MRG-M

● Storage Of Data● Software Storage Appliance● XFS● GFS

● Present the Data● JBoss Enterprise Middleware Portfolio

Things Red Hat Does

● Ingest/Egress of Data● JBoss Messaging● MRG-M

● Storage Of Data● Software Storage Appliance● XFS● GFS

● Present the Data● JBoss Enterprise

Middleware Portfolio

To make these work, you'll need a few supporting technologies:

● Operating Systems

● Identity Management

● Hypervisors

● Security Frameworks

● Virtualization Management

● Patching

● Provisioning

Virtualization Decision: Xen vs KVM

● Xen● Was rapidly losing community traction● Complicated security model● Slow(er)

● KVM (Kernel Virtual Machine)● Adopted by Linux community as standard● Can play with SELinux natively● Fast(er)● Become foundation of Red Hat Enterprise Virtualization

KVM Details

● Linux based hypervisor for running virtual machines

● Integrated into core Linux kernel as a kernel module since 2006

● Requires hardware virtualization support on the CPU● Intel - Intel VT● AMD - AMD-V

● Supports a wide variety of guest OSes including● Linux (RHEL, Fedora, other distros)● Microsoft Windows (7, XP, Server 2003, Server 2008)

KVM Scalability (RHEV 3 / RHEL 6.2)

Support Limit Theoretical Limit

HostCPUs 160 4096Memory 2TB 64TB

GuestCPU 64 4096Memory 512GB 64TB

KVM Timeline

Virtualization in a Shared Environment: sVirt

● Applies security label to all “files” on the system● In actuality, applies it to the inode metadata through a field

called “secmark”

httpdapache_t /etc/shadow

shadow_t

~/public_html

httpd_sys_content_t

DENY

ALLOW

Virtualization in a Shared Environment: sVirt

● sVirt applies SELinux framework to Virtual Machines

alpha_vm

virt_machine_t:01

bravo_vm

virt_machine_t:02

charlie_vm

virt_machine_t:03

DENIED DENIED

/dev/eth0 /dev/eth1

AL

LO

WE

D

DENIED

● Designed for large scale, 500+ hosts and 10,000+ VMs

● Administrative interfaces: WebGUI, RESTful API

● High availability, live migration, self-service, load balancing (DRS), Power Saving (DPM), Template Thin Provisioning, Snapshots, Centralized storage management, etc built in natively

RHEV USER PORTAL

● Tied to Microsoft Active Directory or Red Hat Identity Management (LDAP) users and groups

● Role and object based security delegation

● Complete VM lifecycle management

INDUSTRY LEADING VIRTUALIZATION PERFORMANCE

● SPECvirt_sc2010: As of January 1, 2012, RHEV claims top 6 results and the only 8 socket server scores

INDUSTRY LEADERSHIP: SIGNIFICANT COST ADVANTAGE

● 10 physical hosts (2x4HT, 64GB)● Same density across both

● 10 physical hosts (2x8HT, 256GB)● Same density across both

RHEV COSTS 1/7th VS. VMWARE AND 1/3rd OVER 3 YEARS. SCALE UP COST ADVANTAGE EVEN MORE

Deltacloud: Many clouds, one API

A RESTful API for simple, any-platform access

Deltacloud Quick Start$ deltacloudd -l

Available drivers:* condor* vsphere* opennebula* eucalyptus* rhevm* sbc* azure* gogrid* mock* rackspace* rimuhosting* terremark* ec2

require 'deltacloud'

api_url = 'http://192.168.10.244:5000/api'api_name = 'TK2PJCAN9R1HKG2FK24Z'api_password = 'aLe27rZlRhlBcVoQbL4JsVtaNga12vEL9d9kS5CA'

client = DeltaCloud.new( api_name, api_password, api_url )

# get a list of currently running instances (virtual machines)client.instances.each do |instance| puts instance.nameend

$ deltacloudd -i rackspace -P 10000 -r 192.168.10.200

Structured Data Services

CFDB

CSDS DMDC

GSORTSIDE/AVNGA

FLIS

CSDS_PL

GDSS JOPESClassic

JOPES4.0

Physical Layer (PL)

GTN

Structured Data Services

CFDB

CSDS DMDC

GSORTSIDE/AVNGA

FLIS

CSDS_PL

CSDS_VBL

GDSS JOPESClassic

JOPES4.0

Virtual Base Layer

(VBL)

Physical Layer (PL)

GTN

Structured Data Services

CFDB

CSDS DMDC

GSORTSIDE/AVNGA

FLIS

CSDS_PL

CSDS_VBL

Facilities_VMLMaterial_VML

GDSS

Priv

ate

Dat

a a

nd M

etad

ata

Virtual Mid Layer (VML)

JOPESClassic

JOPES4.0

Virtual Base Layer

(VBL)

Physical Layer (PL)

GTN

Plans_VML

Structured Data Services

CFDB

CSDS DMDC

GSORTSIDE/AVNGA

FLIS

CSDS_PL

CSDS_VBL

Facilities_VMLMaterial_VML

Facilities_VQLMaterial_VQL

GDSS

Plans_VQL

Priv

ate

Dat

a a

nd M

etad

ata

Pub

lic D

ata

Virtual Mid Layer (VML)

Virtual Query Layer (VQL)

(Exposed Views)

JOPESClassic

JOPES4.0

Virtual Base Layer

(VBL)

Physical Layer (PL)

GTN

Plans_VML

SELinux

Quantifying the Problem

● Red Hat Enterprise Linux 5 STIG

● ~587 checks

Minutes per checkto configure

Estimated time forlockdown for all 587 checks

1 9.8 hours

3 29.4 hours

5 48.9 hours

8 78.3 hours

scap-security-guide

Jeff Blank

I4312 Global Mitigations / Commercial

Open Source Project

Really:� http://fedorahosted.org/scap-security-guide

(and yes, I have permission) Why?

� enables agile vendor coordination� ensures consensus among stakeholders� enables development in SCAP formats

Open Source Project

Really:� http://fedorahosted.org/scap-security-guide

(and yes, I have permission) Why?

� enables agile vendor coordination� ensures consensus among stakeholders� enables development in SCAP formats

How is this possible?

SCAP formats

� XML schemas, managed by NIST Standardized format enables re-use,

re-purposing

� Configuration checklist / guide format is XCCDF

� Automated Checking language is OVAL

� others...

XCCDF (“shorthand”) Example

(partial) OVAL Example

Cost/Benefit Argument Costs

� Formats complex� OVAL just a bit verbose </understatement>

Benefits

� Ingestable by range of SCAP-compatible tools OpenSCAP is part of RHEL platform!

� XCCDF Profiles� Standardized outputs/reporting

Interested?

Visit http://fedorahosted.org/scap-security-guide

� public mailing list, wiki, git repo� or send me (jdblank) email internally

Actively seeking input / collaboration

Origins of AMQP

“AMQP was born out of my own experience and frustrations in developing front- and back-office processing systems at investment banks. It seemed to me that we were living in integration Groundhog Day - the same problems of connecting systems together would crop up with depressing regularity. Each time the same discussions about which products to use would happen, and each time the architecture of some system would be curtailed to allow for the fact that the chosen middleware was reassuringly expensive.”

John O’Hara “Toward a Commodity Enterprise Middleware”

AMQP - an Internet Protocol for Business Messaging

AMQP● Practical, comprehensive messaging specification

● Supports pervasive deployments● Open IP governs usage● Supports expanding ecosystem

● Designed for real world requirements● Developed by vendors and user organizations

Bank of America

Barclays Bank

Cisco Systems

Credit Suisse

Deutsche Börse Systems

Envoy Technologies

Goldman Sachs

HCL Technologies

Software AG

Solace Systems

Storm

Tervela

TWIST

WSO2

VMWare Inc

29 West

INETCO Systems

Informatica

JPMorgan Chase Bank

Microsoft

Novell

Progress Software

Rabbit

Red Hat

MRG Messaging Features

● Variety of supported clients● C++, Java/JMS, .NET, Python, Ruby

● Variety of message exchange types● P2P, fanout, Pub-Sub, asynch, direct, header● Custom exchanges (per AMQP)

● e.g. XQuery on XML message's header or body

Exchange

MRG Messaging - a Modular Approach

Exchange

Queue

Queue

QueueBindings

Publisher Application

ConsumerApplication

ConsumerApplication

ConsumerApplication

● Exchange inspects inbound messages from publisher● Routes messages to queues via binding● Consumers receive message via subscribing to queues

● Architectural Strength● Sophistication via myriad exchange/queue combinations● Runtime creation of exchanges, queues, bindings

MRG-Messaging Performance Sample

Infinispan Reliably-Acknowledged Messages/Second

Configuration Specifics

● Capable of 1B+ messages 14 minutes

● JPMorgan only sends ~1B AMQP messages per day