recipe for good secrets management
DESCRIPTION
Discussion of secrets management for DevOps and a recipe for success.TRANSCRIPT
![Page 1: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/1.jpg)
Recipe for Good Secrets Management
Boston Chef Meetup - August 2014
![Page 2: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/2.jpg)
Secrets Management
A strength when done right
Otherwise…
![Page 3: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/3.jpg)
◁ CodeSpaces
◁ AWS Credential leaks in Github
◁ HIPAA breach violations
◁ Heartbleed update woes
⊃ Was tricky even for the experts
⊃ Required BOTH openssl library update AND certificate rotation
First the bad news
![Page 4: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/4.jpg)
“I don’t care about compliance”
Really? Would your company ever like to get
6-figure checks from large enterprises?
![Page 5: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/5.jpg)
Secrets
Traditional approach
ConfigurationManagement(CM)
Artifacts
Orchestration
New Machine
“Secrets has always been kind of a hacky bit; like GPG encrypt a piece of data and stick it in a YAML file”
- Anonymous CM Technologist
![Page 6: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/6.jpg)
Secrets Management is pretty hard
![Page 7: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/7.jpg)
Requirement : separation of duties
One single actor should not be able to do everything.
Use separate roles for:
a) Loading credentials
b) Retrieving and using credentials
![Page 8: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/8.jpg)
Requirement : least privilege
Only give each actor as much power
as is necessary to get the job done.
![Page 9: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/9.jpg)
Requirement : leak-resistance
Don’t leave secrets lying around on:
◁ unencrypted persistent disks◁ backups◁ snapshots
![Page 10: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/10.jpg)
Requirement : audit
Record changes:
◁ policies which govern access◁ each time a secret is changed◁ each time a secret is fetch and used
![Page 11: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/11.jpg)
Requirement : rotation
A secret (e.g. database password, cloud credential)
Should be changed regularly
![Page 12: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/12.jpg)
Here’s a recipe for good secrets management
![Page 13: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/13.jpg)
![Page 14: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/14.jpg)
![Page 15: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/15.jpg)
![Page 16: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/16.jpg)
can admin
can admin
can update
can update
can fetch
can fetch
“break glass”account
Environment A : “stage” Environment B : “production”
![Page 17: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/17.jpg)
can admin
can admin
“break glass”account
Have an emergency access credential
![Page 18: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/18.jpg)
Environment A : “stage” Environment B : “production”
Separate security environments
![Page 19: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/19.jpg)
can update
can fetch
can fetch
Environment A : “stage” Environment B : “production”
Design for “robot” actors
![Page 20: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/20.jpg)
Minimize human access to production
can update
can fetch
Environment B : “production”
![Page 21: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/21.jpg)
can update
Environment B : “production”
Rotate credentials with privileged robots
![Page 22: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/22.jpg)
Environment A : “stage” Environment B : “production”
Automatically record system activity
![Page 23: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/23.jpg)
Where do Configuration Management
and Orchestration fit in?
![Page 24: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/24.jpg)
Environment A : “stage” Environment B : “production”
Provisioning robots into the system
![Page 25: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/25.jpg)
Bootstrapping machine identity
Orchestration Server
Launch Scriptor
Console
NewMachine
RobotIdentity Server
● New machine calls to Orchestration Server for identity
● Orchestration passes a credential (token) to Robot Identity Server
● Robot Identity assigns robot identity● Orchestration / CM installs identity on the new
machine
![Page 26: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/26.jpg)
A new machine is impotent
until identity is acquired
![Page 27: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/27.jpg)
Fetching secrets
password = secrets_manager.secret([node.
chef_environment,
'mysql/server_root_password'].join('/')
mysql_database 'phpapp' do
connection (host: 'localhost', username:
'root', password: password)
action :create
end
http://gettingstartedwithchef.com/first-steps-with-chef.html
● Replace sensitive attribute data with secrets from the secrets manager
● Use the environment name to separate secrets into permissions namespaces
![Page 28: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/28.jpg)
template '/dev/shm/mysql.conf' do …endlink '/dev/shm/mysql.conf' do to '/etc/mysql.conf'end
Keep secrets separate from dataGeneral strategy for Linux - install secrets to /dev/shm
template '/mnt/etc/mysql.conf' do …endlink '/mnt/etc/mysql.conf' do to '/etc/mysql.conf'end
General strategy for ec2 - install secrets to /mnt
![Page 29: Recipe for good secrets management](https://reader034.vdocument.in/reader034/viewer/2022051816/546e1cafb4af9f662c8b56e6/html5/thumbnails/29.jpg)
Keeping secrets separate from
data helps to satisfy important
compliance and security standards
such as PCI and HIPAA