Getting big without getting fat, in PerlDean Hamstead, dean@bytefoundry.com.auSydney PM, 18th August 2015

Trigger Warning:Startup lingo for the next few slides.

I’ve given you fair warning :)

Whipupitude and Getting Things Done!

● Whipupitude is a Perl strength○ Great at getting from 0 to 1

● Quick perl scripts tend to become enterprise software (possibly better written?)

● Getting Things Done > Unfinished ‘Better’○ A business making money from in-production code

is a business with the luxury of rewriting in-production code!

○ The quality (hence cost) of the solution needs to match the problem being solved

○ (We are artists, so trade offs don’t always sit well)

Going from 1 to 1+● Initially we had nothing, now we have

something● It works well enough that now we have the

luxury of rewriting it● “I was right” attitudes forget that we had to

get from 0 to 1 :)● As our business/platform is now making

money, its a “problem” that demands a better solution

Quick lol

Make the pain go away● Structuring your code to add features easily

& quickly

● Ensure that existing features keep working and new features work as planned

● Providing the right buttons and knobs to make life easier for humans

The four elements of code that grows sensibly

Testing.Won’t be covered in this talk.

Logging.See my talk from last month.

tl;dr?

● Try Log::Log4perl● Try Log::Dispatch● Profit!

Plugins.Ok, let’s dive in.

Plugins 101● Easily add functionality● Easily remove functionality

○ Make things ‘pluggable’!

● Simplify parallel development● Facilitate orthogonal requirements/needs

● Loose coupling and separation of concerns● Simplify testing!

Dog Food Philosophy● State your API publically● Move all functionality into plugins, use only

your stated API● Keep only basic functionality

in the core of the application● Ideally, the core should rarely

change

By using your own API you will quicklydiscover it’s weaknesses!

Measurements of success● Separation of concerns achieved

(I have no idea how you measure this?)● You can change the internals without

changing the plugins!● Others can make a plugin without

understanding all (or any!) internals● It’s so easy (and fun!) to write plugins

that people are doing it whimsically

API Nice-to-haves:● temporarily disable plugins

● per-plugin configuration

● per-plugin storage

API Strategies● Hooks● Delegation● Roles● Mix-ins

● Mix and match!

Hooks● The plugin registers with the program

○ “I can read image/png files”○ “Tell me when the user enters a URL”○ “Call me when a new connection is opened”

● Fine grained control● Helps provide a clear API● Can be a pain to put in hook

calls everywhere it’s(possibly) needed

Hooks as overrides: Net::ServerIn Net::Server you override the default hook handlers (which do nothing). The basic flow is shown below:

$self->configure_hook; # Hook

$self->configure(@_);

$self->post_configure;

$self->post_configure_hook;

$self->pre_bind;

$self->bind;

$self->post_bind_hook;

$self->post_bind;

$self->pre_loop_hook;

$self->loop;

# lots of others inside $self->loop$self->pre_server_close_hook;$self->server_close;

Others (not in ->loop):

$self->can_read_hook()

$self->allow_deny_hook()

$self->request_denied_hook()

$self->post_process_request_hook()

$self->post_client_connection_hook()

$self->other_child_died_hook($pid)

$self->write_to_log_hook()

$self->fatal_hook()

$self->post_child_cleanup_hook()

$self->restart_open_hook()

$self->restart_close_hook()

$self->child_init_hook()

$self->pre_fork_hook()

$self->child_finish_hook()

E.g. Net::Server (cont.)use strict; use warnings;

package My::Thing;

use parent qw( Net::Server::PreFork ); # pick your personality

sub _rename_me { $0 = q|Me: | . $_[0] }

sub post_bind_hook { _rename_me(q/listening/) }

sub process_request { _rename_me(q/running/); do_stuff() }

package main;

My::Thing->run(port => 12345);

The overrides way:● Uses normal perl inheritance● In Net::Server you can override stuff that’s

not intended as a hook for full blown perl madness

● AUTOLOAD can be a lot of fun here● Only allows one sub to be called on each

hook, although your sub could call SUPER::

Class::Trigger● Allows your main program to define hooks (triggers) and

when they are called● Provides methods so that your plugin attaches it’s subs

to these hook points

● Triggers can optionally have callbacks!

● Triggers are inherited!● Triggers are called in the

same order they are attached

DelegationThe program tells the plugin/extension/module: You go take care of this stuff

● “you take care of the database connection”● “you parse the XML”

Not so good for small tweaks and add-ons

Inheritance is / can be a variation of this

Roles● Are when a package (class) is required to

implement certain sub’s (methods)● DIY with sub foo { die ‘missing

foo’ } in your base class (don’t though)● Use Role::Tony, Moo::Role or similar● Both complain early and loud that things are

missing● Plus other helpful stuff

Mix-ins● Add methods into existing base classes● Good for adding new stuff to frameworks● Not so good for changing how something

works● Usually not good for application plugins

The Catalyst framework uses this method, which is how methods magically appear on $c

Flat-file vs ModulesFlat-file● OK for simple stuff● You’ll have to

locate and load them

● Namespace dramas

● Crude entry/exit points

Modules● Full-access to Perl

OO goodness● Perl can take care

of locating them● Elegant touch

points

Flat File & .pm comparedcat Plugins/Example.pl

#!perl

use strict;

use warnings;

sub _hello {

print qq|hello!\n|

}

# just one action, final statement

sub { _hello() }

cat Plugins/Example.pm

#!perl

use strict;

use warnings;

package App::Plugins::Example;

our $VERSION = 0.1;

sub _hello {

print qq|hello!\n|

}

sub action1 { hello() }

sub action2 { print qq|bai!\n| }

1

Loading? Don’t do this….

Use Module::Pluggable#!perl

package MyClass;use Module::Pluggable;use MyClass;my $mc = MyClass->new();# returns the names of all plugins installed under MyClass::Plugin::*my @plugins = $mc->plugins();

# Or if you want to look in another namespaceuse Module::Pluggable search_path => ['Acme::MyClass::Plugin', 'MyClass::Extend'];

# You can limit the plugins loaded using the except option, either as a string, array ref or regexuse Module::Pluggable except => 'MyClass::Plugin::Foo';

# Or if you want to instantiate each plugin rather than just return the nameuse Module::Pluggable instantiate => 'new';

# Alternatively you can just require the module without instantiating ituse Module::Pluggable require => 1;

M::P::Object for OO#!perl

package MyClass;use Module::Pluggable::Object; my $finder = Module::Pluggable::Object->new(%opts);print "My plugins are: ".join(", ", $finder->plugins)."\n";

Other options as per Module::Pluggable. There are so so many :)

Nice examples● Plagger

● qpsmtpd

● HTML::FormFu (maybe)

Config (CLI)● GetOpt::LongDoes it need introducing? I am open to alternatives though...

● Pod::UsagePrints a usage message from embedded pod documentation (do things once!)

Config (file)“Config::Any provides a facility for Perl applications and libraries to load configuration data from multiple different file formats. It supports XML, YAML, JSON, Apache-style configuration, Windows INI files, and even Perl code.”

Config (file) (cont.)● Config::General deserves an extra look● Apache style config files● Lots of control over how the config is

interpreted and processed for you:○ (Dis)allow multiple identical options?○ Lower case all names?○ Include directories, globs? over and over?○ Merge duplicates?○ Taint cleaning○ Conversion to boolean (“true” and “false”)○ Much much more!

Config (file) (cont.)Fragmenting config files is really good idea. We know this because Debian does it religiously: What Would Debian Do?

/etc/whatever/conf.d/*conf

Facilitates separation of concerns, works well with plugins

Makes writing helper scripts super easy

Just email me or find me on Facebook

Dear Internet,

Please contact me and tell me where I can do better!

I don’t (yet) know everything

Questions?

Other good stuff:● Writing Pluggable Software

http://www.slideshare.net/miyagawa/writing-pluggable-software

● Build Easily Extensible Perl Programshttp://www.askbjoernhansen.com/archives/2005/08/Build_Easily_Extensible_Perl_Programs.pdf