growing pains: avoiding module hell

Growing Pains: Avoiding Module

Hell Chad Metcalf

@metcalfc

Saturday, August 24, 13

As do many stories, it all begins with ...


$ sudo


sudo::conf { 'web': source => 'puppet:///files/etc/sudoers.d/web', } sudo::conf { 'admins': priority => 10, content => "%admins ALL=(ALL) NOPASSWD: ALL\n", } sudo::conf { 'joe': priority => 60, source => 'puppet:///files/etc/sudoers.d/users/joed', }


sudo::rule { "extra_rule": ensure => present, who => 'bob', commands => "/usr/sbin/systemctl", nopass => false, comment => "what ever you like", }


Infrastructure as Code

Module as an API

Source Code Management

Verification and Validation

Release Management


semver.org


MAJOR.MINOR.PATCH


MAJOR version when you make incompatible API changes

1.x.x


MINOR version when you add functionality in a backwards-compatible manner

x.0.x


PATCH version when you make backwards-compatible bug fixes

x.x.0


Its OK to go:

1.0.0


Easy to learnEasy to useHard to misuseEasy to


Gather requirementsUse casesDon’t over specifyPrivate as possible


Nifty

Nifty::Firewall

Nifty::Monitoring

Nifty::Supervisor

iptables

Icinga

Monit

Firewall::Nifty

Monitoring::Nifty

Supervisor::Nifty


Nifty

Nifty::Firewall

Nifty::Monitoring

Nifty::Supervisor

pf

Sensu

Supervisord

Firewall::Nifty

Monitoring::Nifty

Supervisor::Nifty


Don’t Specify ALL THE THINGS!!!


define apache::vhost ( $server_admin $server_name = '', $docroot = '', $docroot_create = false, $docroot_owner = 'root', $docroot_group = 'root', $port = '80', $ssl = false, $template = 'apache/virtualhost/vhost.conf.erb', $priority = '50', $serveraliases = '', $env_variables = '', $passenger = false, $passenger_high_performance = true, $passenger_max_pool_size = 12, $passenger_pool_idle_time = 1200, $passenger_max_requests = 0, $passenger_stat_throttle_rate = 30, $passenger_rack_auto_detect = true, $passenger_rails_auto_detect = false, $passenger_rails_env = '',23 $passenger_rails_base_uri = '',24 $passenger_rack_env = '',25 $passenger_rack_base_uri = '',26 $enable = true,27 $directory = '',28 $directory_options = '',29 $directory_allow_override = 'None'30 )


<?xml version="1.0" encoding="UTF-8"?> 2 3  4 <configuration> 5 <property> 6 <name>mapred.job.tracker</name> 7 <value>xwing07.ul.wibidata.net:8021</value> 8 </property> 9 <property>10 <name>mapreduce.job.counters.max</name>11 <value>120</value>12 </property>13 <property>14 <name>mapred.output.compress</name>15 <value>false</value>16 </property>17 <property>18 <name>mapred.output.compression.type</name>19 <value>BLOCK</value>20 </property>21 <property>22 <name>mapred.output.compression.codec</name>23 <value>org.apache.hadoop.io.compress.DefaultCodec</value>24 </property> <property> <name>mapred.map.output.compression.codec</name> <value>org.apache.hadoop.io.compress.SnappyCodec</value> </property> <property> <name>mapred.compress.map.output</name> <value>true</value> </property> <property> <name>io.sort.factor</name> <value>64</value> </property> <property> <name>io.sort.record.percent</name> <value>0.05</value> </property> <property> <name>io.sort.spill.percent</name> <value>0.8</value> </property> <property> <name>mapred.reduce.parallel.copies</name>47 <value>10</value>48 </property>49 <property>50 <name>mapred.submit.replication</name>51 <value>2</value>52 </property>53 <property>54 <name>mapred.reduce.tasks</name>55 <value>24</value>56 </property>57 <property>58 <name>io.sort.mb</name>59 <value>233</value>60 </property>61 <property>62 <name>mapred.child.java.opts</name>63 <value> -Xmx978972870</value>64 </property>65 <property>66 <name>mapred.job.reuse.jvm.num.tasks</name>67 <value>1</value>68 </property>69 <property>70 <name>mapred.map.tasks.speculative.execution</name>71 <value>false</value>72 </property>73 <property>74 <name>mapred.reduce.tasks.speculative.execution</name>75 <value>false</value>76 </property>77 <property>78 <name>mapred.reduce.slowstart.completed.maps</name>79 <value>0.8</value>80 </property>81 </configuration>

{ "foobar-cluster": { "mapred-options": [ { "name": "mapred.map.output.compression.codec", "value": "org.apache.hadoop.io.compress.SnappyCodec" }, { "name": "mapred.compress.map.output", "value": "true" }, { "name": "io.sort.factor", "value": "64" }, { "name": "io.sort.record.percent", "value": "0.05" }, { "name": "mapred.map.tasks.speculative.execution", { "name": "mapred.reduce.tasks.speculative.execution",54 "value": "false"55 },56 {57 "name": "mapred.reduce.slowstart.completed.maps",58 "value": "0.8"59 }60 ]61 }62 }


growing pains: avoiding module hell

Technology