triple blitz strike

Triple blitz strike !!!

1. Cluster SSH2. Running legacy CGIs like a boss

3. NoSQL trolling

Cluster SSHProblem: you have more than 1 but not too much hosts to control - 

                                        * le admin

e.g. 20-30 hosts. And you have a problem like ...

Cluster SSH

* le customer

                         - Please give me report from log file xyz.log                             from hosts 67 to 90 for October 25th...                                                                      * le admin    ..  OR ...

ClusterSSH... you need to edit some file on 20 hosts and strucrure of this file is slightly different on some hosts. You need to logon to all hosts manually and make grep/edit or write complex sed/vi scripts - rage and anxious are increasing ->                                ->                             ->

ClusterSSHThere are many ways how we can solve it...

Clusterssh - is one of it.

Install is simple - emerge -av clusterssh - Gentooyum install clusterssh - RHEL / CentOS (EPEL / Rpmforge)apt-get install clusterssh - Debian based

Source installation simple too - because it's Perl (perl-Tk and Perl-Tk-X11 required)

DEMOalso there

Running legacy CGIs

like a boss

Running legacy CGIs

There are many still good CGI software in a wild - e.g.

Smokeping http://smokeping.org  

Running legacy CGIs

But you have Nginx installed and running, and do not want to install Apache or Lighttpd instead - also you do not want to use legacy and slow CGI protocol (actually smokeping uses SpeedyCGI but it's also very old and legacy now )

Running legacy CGI

What we will need : 1. PSGI/Plack http://plackperl.org"PSGI is an interface between Perl web applications and web servers, and Plack is a Perl module and toolkit that contains PSGI middleware, helpers and adapters to web servers.PSGI and Plack are inspired by Python's WSGI and Ruby's Rack"

    By using Plack/PSGI we can run the smokeping.cgi as a web application on any number of backends (proxied, FreeCGI, mod_perl, etc). 

2. CGI::Emulate::PSGI / CGI::Compile - Perl modules from CPAN, it's an easy way to convert CGI script for running persistent as Plack application.

smokeping.cgi

#!/usr/bin/perl -w# -*-perl-*-

use lib qw(/usr/pack/rrdtool-1.3.0-rp/lib/perl);use lib qw(lib);use strict;use warnings;use Smokeping 2.004002; Smokeping::main("etc/config");... changes to ...

smokeping.psgi

use CGI::Emulate::PSGI; use CGI qw(); use warnings; use strict; use Smokeping 2.004002;   my $smokeping = sub {          CGI::initialize_globals();          Smokeping::cgi("etc/config"); }; CGI::Emulate::PSGI->handler( $smokeping );

Or even eazier -

use CGI::Compile;use CGI::Emulate::PSGI;use warnings; use strict; my $smokeping = CGI::Compile->compile                                ("/path/to/smokeping.cgi");CGI::Emulate::PSGI->handler( $smokeping );

RunningServer / proxied -cd /path/to/smokeping starman -Ilib -p 8081 -E deployment \        --workers 2 &"Starman is a high-performance, preforking and PSGI compatible HTTP server"

nginx.conflocation /smokeping { proxy_pass http://localhost:8081; }location /smokeping {alias /path/to/smokeping/public/;}

Running

FastCGI - cd /path/to/smokeping  plackup -s FCGI --listen /var/run/smokeping.sock \                --daemonize --nproc 4 nginx.conflocation /smokeping { fastcgi_pass unix:/var/run/smokeping.sock; }location /smokeping {alias /path/to/smokeping/public/;}

We did it !

P.S.

Latest Smokeping from SVN already support FastCGI, but it was fun, isn't it?   Also now you know kung fu, I hope - and I'm happy. :)

NoSQL trolling

NoSQL Modern buzzword, isn't it?

"SQL is old, not scaling well, only vertical scaling, not fit well for web,                          ...  blah blah blah..." -

    "NoSQL is silver bullet, simple, scalible, distributed key store value, software engineers and admins are happy " -

  MongoDB, Cassandra, CouchDB, HBase - many of them exists.

NoSQL

Everybody was very enthusiastic - "Facebook moves from MySQL to Cassandra" ! "Twitter moves from MySQL to Cassandra" ! "Foursquare is based on MongoDB" !

        RUN NOSQL                                      EVERYWHERE !

NoSQLBut what is in reality?

1. Facebook.Runs on MySQL (was 1800 servers and only 2 DBAs, now over 2500 servers) and DO NOT PLAN to move to anything!Cassandra was used only in inbox search, now replaced with HBase.

2. Twitter.Gave up migrating to Cassandra after a year of tryouts - "For now, we're not working on using Cassandra as a store for Tweets. This is a change in strategy. Instead we're going to continue to maintain our existing Mysql-based storage. "

NoSQL

3. Foursquare.Had 11-hours downtime because of MongoDB - http://blog.foursquare.com/2010/10/05/so-that-was-a-bummer/ 4. DiasporaAbandon MongoDB for MySQL because "MongoDB's document-based approach didn't fit with social data, which maps more naturally to a relational model."

   WHY THESE               THINGS HAPPENS?

NoSQL is bit of new and scaryand SQL is reliable and well known solution...

NoSQL vs SQL

"These distributed databases like Cassandra, MongoDB, and CouchDB aren't actually very scalable or stable. Twitter apparently has been trying to move from MySQL to Cassandra for over a year. When someone reports using one of these systems as their primary data store for over 1000 machines for over a year, I'll reconsider my opinion on this."

Adam D. Angelo, former Facebook VP & CTO,founder of Quora.

Quora was founded in the end of 2010 and using MySQL as main storage.

NoSQL

So, there is (somewhat trolling and discussable) conclusion -

NoSQL solutions are good, but SQL is still proven and reliable data store for PRIMARY data source of project. NoSQL can be used for caching purposes or for some specific functions. Use primary NoSQL storage only for small or personal project, or if you are brave and/or have enough money to risk :)

PROBLEMS ?