2012-03-15 what's new at red hat
TRANSCRIPT
What's New At Red Hat
15-MAR-2012
0900 - 1030
Shawn WellsTechnical Director, U.S. Intelligence [email protected]
Things we get asked . . .
● “As an analyst, I would like to pull up a fused timeline, social network diagram, or virtual dossier of structured and unstructured data on a particular person or entity or a group of individuals.”
● “I would like to search social media data from foreign social media sites in native language for particular hash tags, keywords, or individual account histories”
● “As a product owner, I would like the cloud to ingest, index, and extract entities from 42,000 documents per hour (hitting our 1m docs/day requirement), with a high degree of precision and recall”
● “As an analyst, I would like the ability to upload structured data into a data layer easily, have that data available to run statistical algorithms against, and have those algorithms run in a cloud”
● “As an analyst, I would like a repository to store files, that is not my office share drive, and with which I could send links to files, vice attachments, in my lotus notes”
Things We Do
● Ingest/Egress of Data● JBoss Messaging● MRG-M
● Storage Of Data● Software Storage Appliance● XFS● GFS
● Present the Data● JBoss Enterprise Middleware Portfolio
Things Red Hat Does
● Ingest/Egress of Data● JBoss Messaging● MRG-M
● Storage Of Data● Software Storage Appliance● XFS● GFS
● Present the Data● JBoss Enterprise
Middleware Portfolio
To make these work, you'll need a few supporting technologies:
● Operating Systems
● Identity Management
● Hypervisors
● Security Frameworks
● Virtualization Management
● Patching
● Provisioning
Virtualization Decision: Xen vs KVM
● Xen● Was rapidly losing community traction● Complicated security model● Slow(er)
● KVM (Kernel Virtual Machine)● Adopted by Linux community as standard● Can play with SELinux natively● Fast(er)● Become foundation of Red Hat Enterprise Virtualization
KVM Details
● Linux based hypervisor for running virtual machines
● Integrated into core Linux kernel as a kernel module since 2006
● Requires hardware virtualization support on the CPU● Intel - Intel VT● AMD - AMD-V
● Supports a wide variety of guest OSes including● Linux (RHEL, Fedora, other distros)● Microsoft Windows (7, XP, Server 2003, Server 2008)
KVM Scalability (RHEV 3 / RHEL 6.2)
Support Limit Theoretical Limit
HostCPUs 160 4096Memory 2TB 64TB
GuestCPU 64 4096Memory 512GB 64TB
KVM Timeline
Virtualization in a Shared Environment: sVirt
● Applies security label to all “files” on the system● In actuality, applies it to the inode metadata through a field
called “secmark”
httpdapache_t /etc/shadow
shadow_t
~/public_html
httpd_sys_content_t
DENY
ALLOW
Virtualization in a Shared Environment: sVirt
● sVirt applies SELinux framework to Virtual Machines
alpha_vm
virt_machine_t:01
bravo_vm
virt_machine_t:02
charlie_vm
virt_machine_t:03
DENIED DENIED
/dev/eth0 /dev/eth1
AL
LO
WE
D
DENIED
● Designed for large scale, 500+ hosts and 10,000+ VMs
● Administrative interfaces: WebGUI, RESTful API
● High availability, live migration, self-service, load balancing (DRS), Power Saving (DPM), Template Thin Provisioning, Snapshots, Centralized storage management, etc built in natively
RHEV USER PORTAL
● Tied to Microsoft Active Directory or Red Hat Identity Management (LDAP) users and groups
● Role and object based security delegation
● Complete VM lifecycle management
INDUSTRY LEADING VIRTUALIZATION PERFORMANCE
● SPECvirt_sc2010: As of January 1, 2012, RHEV claims top 6 results and the only 8 socket server scores
INDUSTRY LEADERSHIP: SIGNIFICANT COST ADVANTAGE
● 10 physical hosts (2x4HT, 64GB)● Same density across both
● 10 physical hosts (2x8HT, 256GB)● Same density across both
RHEV COSTS 1/7th VS. VMWARE AND 1/3rd OVER 3 YEARS. SCALE UP COST ADVANTAGE EVEN MORE
Deltacloud: Many clouds, one API
A RESTful API for simple, any-platform access
Deltacloud Quick Start$ deltacloudd -l
Available drivers:* condor* vsphere* opennebula* eucalyptus* rhevm* sbc* azure* gogrid* mock* rackspace* rimuhosting* terremark* ec2
require 'deltacloud'
api_url = 'http://192.168.10.244:5000/api'api_name = 'TK2PJCAN9R1HKG2FK24Z'api_password = 'aLe27rZlRhlBcVoQbL4JsVtaNga12vEL9d9kS5CA'
client = DeltaCloud.new( api_name, api_password, api_url )
# get a list of currently running instances (virtual machines)client.instances.each do |instance| puts instance.nameend
$ deltacloudd -i rackspace -P 10000 -r 192.168.10.200
Structured Data Services
CFDB
CSDS DMDC
GSORTSIDE/AVNGA
FLIS
CSDS_PL
GDSS JOPESClassic
JOPES4.0
Physical Layer (PL)
GTN
Structured Data Services
CFDB
CSDS DMDC
GSORTSIDE/AVNGA
FLIS
CSDS_PL
CSDS_VBL
GDSS JOPESClassic
JOPES4.0
Virtual Base Layer
(VBL)
Physical Layer (PL)
GTN
Structured Data Services
CFDB
CSDS DMDC
GSORTSIDE/AVNGA
FLIS
CSDS_PL
CSDS_VBL
Facilities_VMLMaterial_VML
GDSS
Priv
ate
Dat
a a
nd M
etad
ata
Virtual Mid Layer (VML)
JOPESClassic
JOPES4.0
Virtual Base Layer
(VBL)
Physical Layer (PL)
GTN
Plans_VML
Structured Data Services
CFDB
CSDS DMDC
GSORTSIDE/AVNGA
FLIS
CSDS_PL
CSDS_VBL
Facilities_VMLMaterial_VML
Facilities_VQLMaterial_VQL
GDSS
Plans_VQL
Priv
ate
Dat
a a
nd M
etad
ata
Pub
lic D
ata
Virtual Mid Layer (VML)
Virtual Query Layer (VQL)
(Exposed Views)
JOPESClassic
JOPES4.0
Virtual Base Layer
(VBL)
Physical Layer (PL)
GTN
Plans_VML
SELinux
Quantifying the Problem
● Red Hat Enterprise Linux 5 STIG
● ~587 checks
Minutes per checkto configure
Estimated time forlockdown for all 587 checks
1 9.8 hours
3 29.4 hours
5 48.9 hours
8 78.3 hours
scap-security-guide
Jeff Blank
I4312 Global Mitigations / Commercial
Open Source Project
Really:� http://fedorahosted.org/scap-security-guide
(and yes, I have permission) Why?
� enables agile vendor coordination� ensures consensus among stakeholders� enables development in SCAP formats
Open Source Project
Really:� http://fedorahosted.org/scap-security-guide
(and yes, I have permission) Why?
� enables agile vendor coordination� ensures consensus among stakeholders� enables development in SCAP formats
How is this possible?
SCAP formats
� XML schemas, managed by NIST Standardized format enables re-use,
re-purposing
� Configuration checklist / guide format is XCCDF
� Automated Checking language is OVAL
� others...
XCCDF (“shorthand”) Example
(partial) OVAL Example
Cost/Benefit Argument Costs
� Formats complex� OVAL just a bit verbose </understatement>
Benefits
� Ingestable by range of SCAP-compatible tools OpenSCAP is part of RHEL platform!
� XCCDF Profiles� Standardized outputs/reporting
Interested?
Visit http://fedorahosted.org/scap-security-guide
� public mailing list, wiki, git repo� or send me (jdblank) email internally
Actively seeking input / collaboration
Origins of AMQP
“AMQP was born out of my own experience and frustrations in developing front- and back-office processing systems at investment banks. It seemed to me that we were living in integration Groundhog Day - the same problems of connecting systems together would crop up with depressing regularity. Each time the same discussions about which products to use would happen, and each time the architecture of some system would be curtailed to allow for the fact that the chosen middleware was reassuringly expensive.”
John O’Hara “Toward a Commodity Enterprise Middleware”
AMQP - an Internet Protocol for Business Messaging
AMQP● Practical, comprehensive messaging specification
● Supports pervasive deployments● Open IP governs usage● Supports expanding ecosystem
● Designed for real world requirements● Developed by vendors and user organizations
Bank of America
Barclays Bank
Cisco Systems
Credit Suisse
Deutsche Börse Systems
Envoy Technologies
Goldman Sachs
HCL Technologies
Software AG
Solace Systems
Storm
Tervela
TWIST
WSO2
VMWare Inc
29 West
INETCO Systems
Informatica
JPMorgan Chase Bank
Microsoft
Novell
Progress Software
Rabbit
Red Hat
MRG Messaging Features
● Variety of supported clients● C++, Java/JMS, .NET, Python, Ruby
● Variety of message exchange types● P2P, fanout, Pub-Sub, asynch, direct, header● Custom exchanges (per AMQP)
● e.g. XQuery on XML message's header or body
Exchange
MRG Messaging - a Modular Approach
Exchange
Queue
Queue
QueueBindings
Publisher Application
ConsumerApplication
ConsumerApplication
ConsumerApplication
● Exchange inspects inbound messages from publisher● Routes messages to queues via binding● Consumers receive message via subscribing to queues
● Architectural Strength● Sophistication via myriad exchange/queue combinations● Runtime creation of exchanges, queues, bindings
MRG-Messaging Performance Sample
Infinispan Reliably-Acknowledged Messages/Second
Configuration Specifics
● Capable of 1B+ messages 14 minutes
● JPMorgan only sends ~1B AMQP messages per day