managing certificates and tls endpoints
TRANSCRIPT
PowerPoint Presentation
Managing Certificates and TLS Endpoints
Charles Neill & Henry Yamauchi Security EngineeringRackspace Hosting
1
Charles Neill & Henry Yamauchi of the Security Engineering team at Rackspace
Tasked with managing a large number of TLS/SSL endpoints
Since Heartbleed, everyone's eyes are on TLS/SSL
Many headaches later, we now have a tool called TLSManager to help us out! (More on that later)www.rackspace.com2What's this all about?
Wire encryption protocol
Provides authentication, integrity and confidentiality
Based on Public Key Infrastructure (PKI)
SSL originally designed at Netscape for eCommerce in the early-mid 1990sSSLv1 was never released, leaving us with SSLv2 and v3 + TLSv1.0, 1.1, and 1.2 today
Widely deployed on a variety of devices (smart phones, laptops, servers, appliances, etc.)
Deceptively simple to get started with (but can be much harder to maintain)www.rackspace.com3What is TLS/SSL?
Time is fast approaching where TLS/SSL will be mandatory
Browsers beginning to warn users about sites not using TLS/SSL
Firefox ChromeEncryption overhead, cost are no longer valid excusesInitiatives like Let's Encrypt (letsencrypt.org) are aiming to provide a free SSL certificate authority very soon to provide free certificates for all
Despite becoming easier/cheaper to set up, TLS/SSL can't just be turned on and left alone in most cases, as we'll seewww.rackspace.com4TLS adoption is increasing
4
EmailIRCAPIs and web servicesVPNAppliances (load balancers, proxies, etc)Smart phone apps
www.rackspace.com5TLS/SSL not just for web servers
5
Certificate Authorities (CAs)ProtocolsCiphersKey exchangeKey sizesHashesRandom Number Generators (RNGs)www.rackspace.com6TLS/SSL moving parts
Multiple protocol versions (SSLv2, v3, TLSv1, v1.1, v1.2)
Certificates expire at different times
Server misconfiguration, multiple servers for single domain
Outdated systems
Large number of endpoints to keep track of
www.rackspace.com7Issues managing TLS/SSL:
Intermediate certs expire
Server certificates can accidentally get checked into version control and have to be revoked
Lax or compromised CAs may have to reissue their certs (e.g. Comodo attack)
Use of self-signed certificateswww.rackspace.com8Also:
New SSL vulnerabilities are discovered (e.g. POODLE)
Bugs in encryption software like OpenSSL or LibreSSL (e.g. Heartbleed)
Regulation compliance (e.g. PCI-DSS 3.1 will disallow TLSv1 in 2016)
SHA1 in certs will be flagged insecure in Jan 1st, 2017 (browser warnings now)
www.rackspace.com9Oh, and don't forget:
10www.rackspace.com
11www.rackspace.comCiphers
When first connecting, client and server engage in negotiation
Client initially sends the cipher suites it supports
Server then (ideally) selects the strongest cipher that both supports
Server should be configured with only strong ciphers
There's a catch: the client essentially gets to decide what cipher is used by controlling the ciphers that they send to the server. In some man-in-the-middle attacks, the attacker can potentially downgrade ciphers in this way (e.g. LOGJAM).
www.rackspace.com12TLS cipher negotiation
Combination of encryption, MAC and key exchange algorithms
Available suites depend on the SSL/TLS version
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 ^ ^ ^ ^ | | | | | | | MAC hash algo | | Symmetric key attrs (algo, key size, mode) | Authentication algo Key exchange algowww.rackspace.com13Cipher suites
RecommendedAvoidKey ExchangeECDHE_ECDSADHE_RSARSANULL*_anonMAC HashSHA384SHA256NULLMD5SHA1CiphersAES_256_GCMAES_128_GCMAES_256_CBC AES_128_CBCNULLRC2_*RC4_*DES_*IDEA_*3DES_**_EXPORT_*
www.rackspace.com14Cipher suite suggestions
15www.rackspace.comOther configuration options
Avoid DES, RC4, MD5, 3DES key length < 128, export ciphers
Prefer AES, GCM, SHA2, TLSv1.2, OpenSSL >= 1.0.1p / 1.0.2d
Not all permutations of these are supported (see: cipher suites)
Sometimes you are dependent on vendor support (e.g. hardware load balancers)
Some servers deliberately configured with weak ciphers to support old browsers (IE6 still in use)
Cert expiration is just one reason to replace them (other reasons: use of SHA1, weak key sizes, cert revocation, CA compromises, etc)www.rackspace.com16Server configuration
Example nginx configurationssl_protocols TLSv1.2 TLSv1.1 TLSv1;ssl_ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA; ssl_prefer_server_ciphers on;
Example Apache configurationSSLProtocol All -SSLv2 -SSLv3SSLCipherSuite HIGH:!aNULL:!MD5SSLHonorCipherOrder on
More in-depth examples can be found on https://cipherli.st/www.rackspace.com17Configuration examples
Certificate ordered/installed has too short a key (< 1024 bits)
Full certificate chain (CA bundle) not installed
Preferred cipher not set correctly
Honor cipher order feature not turned on
SSL engine/library outdated
Server not sufficiently protected (so private key is not protected)
Servers in load balance group configured inconsistentlywww.rackspace.com18Other server configuration issues
Protects past conversations even if server private key is compromised
Uses ephemeral keys in key agreement protocols
ECDHE_ECDSA, DHE_RSA
www.rackspace.com19PFS (Perfect Forward Secrecy)
Server configuration to only accept secure connections for this site
Server initially sends Strict-Transport-Security header to browser
Browser makes a note that site should only be accessed with HTTPS
If an attempt is made to connect to port 80, force user to reconnect via HTTPS (port 443 by default)
Particularly useful if sites mix secure and unsecure resources
www.rackspace.com20STS / HSTS
21www.rackspace.comVulnerabilities
Heartbleed (leaks random memory in plain text. Fixed in OpenSSL 1.0.1g)
Logjam (force export ciphers) / WeakDH (weak/reused Diffie-Hellman params)
BEAST (browser issue but can be mitigated server side. TLSv1 and earlier)
CRIME (TLSv1 compression)
FREAK (export ciphers)
POODLE (affects SSLv3)
www.rackspace.com22Some recent vulnerabilities
Problem: MITM can force client and server to agree to use a lower protocol version than what they actually support
Note: The above is possible because SSL/TLS handshake is in cleartext (session key has not been agreed to yet)
Solution: Browser include a dummy cipher to tell the server to check whether its highest protocol version is greater than ClientHellos. If so, abort communication.
Fake ciphers (x00FF)EMPTY-RENEGOTIATION-INFO-SCSV TLS_FALLBACK_SCSV
* Signaling Cipher Suite Valuewww.rackspace.com23TLS downgrade attack protection (SCSV*)
Ciphers allowed by the US Government for the export market
Devised in the 1990s. Laws restricting encryption export are no longer in force.
TLS 1.2 no longer includes them
Export ciphers use short keys (40-56 bit), making them insecure
Identified by the presence of EXP or EXPORT in their name
www.rackspace.com24Export ciphers
25www.rackspace.com
Just because an endpoint is acceptable today does not mean it will remain so tomorrow since:
Certificates expireServers are reconfiguredNew vulnerabilities are uncoveredIncreasing computing power can make old standards/assumptions obsolete (e.g. SHA1)
This means you need to be constantly monitoring your endpoints
www.rackspace.com26In summary
www.rackspace.com27Constantly?!
28www.rackspace.comHow to keep up?!
OpenSSL's "s_client" is useful for debugging SSL. On most Linux/UNIX systems, you can call it like so:openssl s_client [-debug] host host.com port 443
The "-debug" flag will give you more information about the actual connection, showing a hex dump of the handshake and negotiation process
Doesn't tell you much about vulnerabilities, but gives you the information you need if you already know what you're looking forwww.rackspace.com29OpenSSL's s_client
From the testssl.sh site:
"testssl.sh is a free command line tool which checks a server's service on any port for the support of TLS/SSL ciphers, protocols as well as recent cryptographic flaws and more."www.rackspace.com30testssl.sh
Reports SSL/TLS issues such as:SSL/TLS versionWeak protocols, ciphers and hashesKnown vulnerabilities
Uses an easy to understand letter rating system
Scoring is calculated by assigning weights to:Protocol support (30%)Key exchange support (30%)Cipher strength (40%)
Can be accessed through API Official API client here: https://github.com/ssllabs/ssllabs-scanwww.rackspace.com31Qualys SSL Labs
ScoreMeaningA+Domains with no warnings and use HSTS and TLS_FALLBACK_SCSVADomains with 2048 bit keys, at least TLS 1.2 support and no warningsA-Domains with 2048 bit keys, at least TLS 1.2 support but with some warningsBUses SSL 3.0, RC4, missing intermediate certificates, vulnerable to BEAST or CRIME, NS >= 65CDomain is vulnerable to POODLE, NS >= 50DNumeric Score >= 35 ENumeric Score >= 20 MDomain and cert host mismatchTCertificate is not trustedFUses MD5, keys with length < 1024, or SSLv3 is the strongest protocol supported
www.rackspace.com32Qualys letter rating system
33www.rackspace.comYou want an A+, right?
https://cipherli.st
Collection of good configurations for TLS/SSL, covering several different servers (Apache, nginx, HAProxy, Postfix, OpenSSH )
Useful for making recommendations to product teams for software you might not have configured before
Simple copy-and-paste format takes headaches out of implementers making decisions about certain ciphers or other settings (though you should vet these configurations based on what clients you want/need to support!)
www.rackspace.com34Cipherli.st
Hosts several endpoints with broken SSL configurations
Great resource for vetting SSL verification code / test scripts
Examples:https://expired.badssl.com/https://self-signed.badssl.com/
Many, many more!www.rackspace.com35badssl.com
www.rackspace.com36Wanna manage all those endpoints manually?
www.rackspace.com37One possible solution: TLSManager
Automatically scans daily all endpoints in its database
Uses Qualys Labs SSL tool (https://www.ssllabs.com/ssltest)
Reports SSL vulnerabilities
Proactively warns of any certificates about to expire
Emails reports to interested parties
Keeps a history of the resultswww.rackspace.com38Features
38
39www.rackspace.comDEMO TIME
40www.rackspace.com
On Github: https://github.com/rackerlabs/TLSManager OR http://bit.ly/1MUQiss
License: Apache 2.0
Pull requests most welcome!www.rackspace.com41Go get it!
42www.rackspace.comCurrent Events
www.rackspace.com43Google + BoringSSL
www.rackspace.com44Let's Encrypt
www.rackspace.com45SHA-1 IS TOTES BROKEN! SAVE YOURSELF!
www.rackspace.com46SHA-1 IS TOTES BROKEN! SAVE YOURSELF!
www.rackspace.com47Diffie-Hellman APOCALYPSE!1!!
SSL/TLS management is more involved than it looks at first
Lots of moving parts
A constantly changing landscape: a server rated A+ today can be downgraded tomorrow if:Certs expireNew cipher vulnerabilities are uncoveredBugs in SSL/TLS libraries are reportedServer is misconfiguredCert or CA compromise
It's helpful to monitor SSL endpoints in an automated way to keep from missing any issueswww.rackspace.com48Closing thoughts
www.rackspace.com49Questions?
Charles NeillTwitter: @ccneillGithub: cneillEmail: [email protected] YamauchiEmail: [email protected]
Group that focuses on the security of one of the largest open source projects in the world!
IRC channel: #openstack-security on Freenode
Weekly meetings: #openstack-meeting-alt @ 1700 UTC Thursdays
Mailing list: openstack-dev ML with [Security] tag
www.rackspace.com50OpenStack Security Project
THANK YOURACKSPACE | 1 FANATICAL PLACE, CITY OF WINDCREST | SAN ANTONIO, TX 78218US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM RACKSPACE LTD. | RACKSPACE AND FANATICAL SUPPORT ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM
51