317-2011: the day the server said...

1

Paper 317-2011

The Day the Server Said Tweet! William W. Viergever, Viergever & Associates, California

Daniël Kuiper, SAS Institute B.V., The Netherlands Koen Vyverman, SAS Institute B.V., The Netherlands

ABSTRACT It was an idyllic summer afternoon at the SAS® Netherlands Technical Support call center. The pervasive hum and buzz of problem-solving activity was punctuated by the shrill cries of woodland birds drifting in through the open windows. Then, a dormant screen sprang to life, and panicky error messages and warnings started scrolling by! A distant siren wailed. Support staff gathered swiftly around the screen. It was the MIS server tweeting messages of impending doom! A SAS® Portal guru sat down at a console, logged on to the server, diagnosed, and fixed the problem. Not five minutes later, the screen went black again and relative serenity returned. The Managing Director had not even noticed the portal’s hiccups. This paper shows how to use Twitter to let your mission-critical SAS systems tweet their warnings and errors straight to your local support team for immediate follow up, and thus helps to minimize downtime!

INTRODUCTION System administrators typically have a variety of tools at their disposal to monitor the health of the systems and applications which they are support and maintain for their business users. Some of these monitoring solutions involve physically inspecting a reporting and diagnostics console application. Others are centered around a mechanism that sends diagnostic e-mail messages to the administrators. SAS Platform administrators in particular have access to the SAS 9.2 Enterprise Business Intelligence Audit and Performance Measurement (EBIAPM) utilities package, which monitors a number of aspects of a SAS Business Analytics server environment that are critical to the health of the system: is the SAS OLAP Server running? Is the Portal up? Is SAS Web Report Studio working?

Whatever the monitoring tool being used however, the bottom-line is always that an administrator first needs to notice that something is out of kilter. They will then begin to investigate what's going on, and attempt to fix things. While this will work in some cases, a full-fledged SAS Business Analytics environment can be a complex thing, and troubleshooting will take time, perhaps to eventually lead to the conclusion that SAS Technical Support needs to be involved. Valuable time is lost, the problem may become aggravated, and a business critical system goes from being merely unstable to being truly unavailable.

In this paper we present a mechanism to dramatically shorten the time between a problem occurring — or merely threatening to occur! — and your local Support team being on the case and potentially resolving the situation before it becomes a real issue, and hence reduce or avoid downtime. The idea is in essence a simple one: a server in the SAS environment (hereafter: the Server) tweets real-time diagnostic information to a specific Twitter account. The Support team receive these tweets on a dedicated screen and take action as needed! The main advantage of tweeting rather than sending diagnostic e-mails to the Support center is that the tweets are instantly visible whereas e-mails will sit in the incoming queue amidst a zillion other messages until they are read and processed by the operational dispatcher. Ideally, the two mechanisms complement one another. Bursts of tweets appearing on a monitor will attract Support staff attention, while an accompanying e-mail may contain more detail about the situation…

The first chapter of this paper explains what needs to be done on the Twitter side: setting up Twitter accounts for the Server and the Support center; ensuring that only the Support center is able to read the Server’s tweets; an introduction to OAuth, the authentication mechanism used by Twitter, and the various OAuth implementations available on the web. In the following chapter we choose a specific OAuth for Twitter implementation using PHP, and explain how to set up the PHP environment and scripts.

The next chapter explores three practical examples of Server-side SAS batch processes detecting certain worrisome events and tweeting about these: running out of disk space on the WORK volume; an important SAS server process going down; performance degradation by runaway user processes. We conclude the paper with some remarks about practicalities and suggestions for future development.

Throughout this paper we assume that both Server and client machines are running some Windows OS. On other operating systems, the necessary utilities will need to be replaced by suitable alternatives. The SAS code was developed and tested in a SAS 9.2 Maintenance 3 environment.

Social Media and NetworkingSAS Global Forum 2011

PREREQTHE BASTwitter is a140-characTwitter messonorous cTwitter site

So the Serhave unche

Tweet privathe Server to be used

QUISITES 1 SICS arguably the mocter messages ssage volume,cacophony. Wee (www.twitter.c

rver’s Twitter acecked all unne

acy is off by deand the Suppoby the Suppor

– TWITTER

ost popular mic— ‘tweets’ — there are also

e want our Servcom) we sign u

Figure 1. C

ccount is SNLBcessary check

Fig

efault, and the sort center! We rt center: SNLT

R

cro-blogging se— out into the aeo quite a few apver to tweet to up for a new ac

Create the SNL

BAF, as in SASboxes. We log

gure 2. SNLBA

setting (Fig. 2) proceed to log

TECHSUPP. (F

2

ervice nowadayether. And, judgpplications, botan audience of

ccount. (Fig. 1)

BAF Twitter ac

S Netherlands Bon to the Twitt

AF Twitter acco

to protect the toff from Twitte

Fig. 3)

ys. Hundreds oging from a curts, and assortedf one though, n

ccount for the S

Business Analyter site as SNL

ount settings.

tweets is vital ter, and in simila

of millions of hursory examinatd automatons cnot to the entire

Server.

ytics FramewoLBAF and edit t

to keeping thinar vein sign up

umans use it totion of the dailycontributing to e world. On the

rk. Note that wthe account se

gs private betwfor another ac

o send y the

e

we ttings:

ween ccount


Just like wewhile settinjust want afollow, and

Logging onnow has 1

To display (www.tweeapplication

There is noSNLTECH

We then lo

Figu

e did for SNLBng up the accoua bare account. send a reques

Fi

n again as SNLfollower, viz. S

the Server’s twetdeck.com) Aft offers to sign

o need to creatSUPP Twitter a

og on to the Tw

re 3. Create th

AF, we log on unts, it is possi While logged st to follow SNL

Fig

igure 5. SNLBA

LBAF we see thSNLTECHSUPP

weets at the Suter installing Twin with a Twee

te a TweetDeckaccount creden

witter site as SN

e SNLTECHSU

to the Twitter sible to define inon as SNLTECLBAF. (Fig. 4)

gure 4. Send a

AF has a pendi

hat there is a pP.

upport center, wweetDeck on atDeck account

k account. We ntials. (Fig. 7)

NLBAF and sen

3

UPP Twitter ac

site as SNLTECnterests and imCHSUPP, we s

follow-request

ing follow reque

ending request

we deployed a Support cente, or alternative

simply click the

nd a test tweet.

ccount for the S

CHSUPP and amport friends. Nsearch for SNLB

t to SNLBAF.

est from SNLT

t. (Fig. 5) After

freeware applier machine andly, to add a soc

e ‘Add Twitter’

(Fig. 8)

Support center.

activate TweetNo need for all tBAF whose twe

TECHSUPP.

r accepting the

ication called Td launching it focial network ac

button and pro

.

Privacy. Note that however, weets we want t

request, SNLB

TweetDeck. or the first timeccount. (Fig. 6)

oceed to enter

that we o

BAF

e, the

the


The link is good. But rTwitter site

made! (Fig. 9) remember that

e. Which leads

Figure

Whatever SNLwe need SNLBus to the matte

Figure 6. Upo

e 7. Adding a T

Figure 8. A te

LBAF tweets isBAF to generaer of how Twitte

4

on running Twe

Twitter account

est tweet from S

s instantly displte its tweets ther handles auth

eetDeck.

t to TweetDeck

SNLBAF.

ayed in SNLTEhrough code, anhentication…

k.

ECHSUPP’s Twnd not by typin

weetDeck. So g stuff into the

far, so


OAUTH Until the mwasn’t veryHTTP. Sinc(www.oaut

For those kaccessible permit appsecrets are

Somewhatthe ApplicaTwitter usewe’ve namauthenticatit will use th

Clever, huhOAuth flowthe HMAC-SHA1 is a concepts fu

Luckily we have done libraries in these, keepthe chosenon a PHP-b

But before client via hSNLBIRDIEsigned in wname SNL

Twitter recobrowser soof code runchoose ‘cli

Figure

middle of 2010, y secure, but ace then howevth.net) which is

keen on undersfrom the aforelications to act

e being exchan

t oversimplifiedation) to be ableer SNLBAF. In

med SNLBIRDIEte itself: an OAhese, and there

h? There is mow diagram desc-SHA1 messagsecure hash aurther, any atte

don’t need to cthis before altha variety of lanping in mind th

n variety shouldbased solution

we get to the Pttp://twitter.comE is going to be

with). The fact tBIRDIE, and m

ognizes two apo we pick ‘cliennning in batch oent’, the registr

e 9. SNLTECH

Twitter used a t least it was eaer, Twitter has a different bea

standing OAuthementioned site

on their behalfnged.

perhaps, but we to send tweeorder to do tha

E) to pretend thAuth token, an Oe’s no need for

ore to it of courscribing the sevege authenticatiolgorithm well-kempt at explain

code SHA-1 enhough not in SAnguages: C/C+at the authenti

d integrate withand a couple o

PHP part of oum/oauth_clientse given permisthat the applica

make some cho

pplication typest’ as the applicon some serveration form insi

HSUPP can see

very basic autasy to send a tditched Basic

ast altogether.

h in its full glorye. Suffice it to sf without actua

what happens ets. That is, theat, the SNLBAFhat it is SNLBAOAuth token ser the Applicatio

se. The Twitteren steps involveon code to guanown in cryptoing them is wa

ncryption and HAS. The Twitte+, JAVA, Perl, cation in itself

h as little fuss aof really easy P

r set-up, we nes/new (Fig. 10) sion to act on t

ation itself is at oices.

s, client and brocation type. An er without any fists on an appl

5

e the test tweet

hentication metweet from a piAuth in favour

y, we refer to thsay that it is an ally providing th

is essentially thSAS program

F Twitter user teAF. Twitter thenecret, and a PIon to be aware

r developers sited in authentic

arantee the inteography. While ay out of scope

HMAC messager Developers s

PHP, Python, is not one of th

as possible withPHP-scripts by

eed to register This is where

the behalf of ththis point none

owser (Fig. 11)odd choice perront-end, but thication website

t from SNLBAF

echanism (Basiiece of SAS coof the open au

he Beginner’s G authentication

he user’s crede

his: we need a must use the T

ells Twitter than gives the AppN code. Wheneof the actual S

te (see Recomcation. Furthermegrity and authewe encourage for this paper…

ge authenticatiosite lists a slewRuby, you nam

he major pointsh our applicatioy one Joe Chun

our tweet-sendwe tell Twitter

he Twitter user existent is irrele

). Our SAS prorhaps in view ohe right choice e, so we enter a

F in TweetDeck

ic Auth) and lifeode simply by muthorization sta

Guide to OAuthn mechanism thentials. Instead,

SAS program Twitter servicet it is OK for th

plication a buncever the Applic

SNLBAF creden

mmended Readmore, the OAutenticity of trans

e the reader to e…

on from scratchw of publicly avame it. We invess of interest for ons written in Sng.

ding applicationthat an applicaSNLBAF (whic

evant. We mer

ocesses are noof the fact that nonetheless. Ea popular one j

k.

e was easy. It ameans of PROCandard OAuth

h which is hat allows user, tokens and to

(in OAuth spea on behalf of the application (w

ch of stuff to cation needs tontials.

ding) has a greath protocol reliesmitted messagexplore these

h, because otheailable OAuth stigated a few othis paper, and

SAS code. We s

n as a Twitter Oation named ch we are presrely register it b

t going to run fthey’ll merely bEven though wjust to comply.

also C

rs to oken-

ak: he which

o tweet

at es on ges.

ers

of d that settled

OAuth

sently by its

from a be bits

we We


also set thePOST requ

With theseread and wassociated

C

C

We take nointeracting Provider. Smust give t5849 OAut

Note (Fig. applicationkey and sepresses…

e default accesuest methods in

choices we gewrite access. Th with the SNLB

onsumer Key:

onsumer Secre

ote of these sinwith a Service

So the Consumto Twitter to proth specification

12) that it is po. This amounts

ecret in this pap

ss type to ‘Rean its interaction

Figure 10.

Figure 11. Ad

et a page (Fig. he confirmationBIRDIE applica

ilHsE6CDOCQ

et: pfaGMqejR

nce we will nee Provider on be

mer Key and theove its identity. (see Recomm

ossible to reset s to asking Twiper for didactica

d & Write’, whins with the Twit

The Twitter OA

dditional choice

12) confirmingn page also givation:

QR9tZ5sJUxQ

62un72kG3zc6

ed them in the Pehalf of a Usere Consumer Se. For more deta

mended Readin

the Consumertter for a new aal purposes sin

6

ch will enable Stter API.

Auth Applicatio

es to make at A

that the applicves us two long

6ogEh7VmC8P

PHP scripts. Inr. A User is the ecret are really ails on OAuth tg).

r Key and Consapplication ID ance we will hav

SNLBIRDIE to

on Registration

Application Reg

cation SNLBIRg alphanumeric

Pq3pyUD8Zldr4

n OAuth speak, individual who

y the credentialsterminology and

sumer Secret aand password, ve reset everyth

use both HTT

Form.

gistration

DIE was createc strings, which

4

a Consumer iso has an accous that the applid definitions we

associated withand therefore

hing by the tim

P GET and HT

ed by SNLBAFh are apparently

s an Applicatiount with the Serication SNLBIRe refer to the R

h the SNLBIRDit is safe to pue the paper hit

TTP

F with y

on rvice RDIE RFC

DIE blish ts the


Figur

PREREQTHE SOFAs explainebetween ourunning. ThWindows. Wour SAS sevariable.

The PHP pavailable o

To install Pa web servbut go with e.g. the OFor the pur

JOE CHUWhile seartutorial writbare-bonescommunitywe get five

The scripts

re 12. Twitter A

QUISITES 2 FTWARE ed in the previour SAS code ahe downloads sWe downloadeerver environm

project site onlyon the web, see

PHP, simply clicver’. The rest ofh the suggestedracle extensionrpose of this pa

UNG’S SCRIPrching for a suittten by Joe Chs PHP scripts ty at https://githue scripts and a f

s need some m

Application Reg

– PHP

ous chapter, wend Twitter. So section of the P

ed the php-5.3.ment. We then a

y offers 32-bit ce e.g. www.anin

ck through the f the installatiod extensions. An needs certainaper nothing be

PTS table OAuth to ung: http://nullito go with his tuub.com/joechunfew accompan

Figu

minimal editing.

gistration Confi

e plan to use awe need a PH

PHP project sit5-nts-Win32-V

added the locat

compiled Windondya.com

installer and wn is straightforw

Adding more exn Oracle client eyond the defa

Twitter impleminfo.wordpressutorial. These cng/oauth_twitteying files. (Fig.

ure 13. Joe Ch

We first edit th

7

rmation with O

an extant set ofP interpreter oe (www.php.neC9-x86.msi instion of the php.

ows binaries. F

when presentedward. A word oxtensions maysoftware to be ult component

mentation, we c.com/oauth-twican be downloaer After unzippi. 13)

hung’s OAuth T

he globals.php

OAuth Consume

f PHP scripts ton the machine et) offers a numstaller and so in.exe to the Win

For those need

d with a web seof warning howintroduce depepresent — and selection is ne

came across thitter/ Joe also paded from the ing the archive

Twitter scripts.

script, which c

er Key and Con

o handle the Owhere the SAS

mber of compilenstalled the PHndows PATH sy

ding a 64-bit bin

erver choice, chwever: do not peendencies on 3d PHP will refueeded.

he following maprovides a set GitHub open s

e to a suitable P

contains a num

nsumer Secret

OAuth authenticS code will be ed binaries for HP 5.3.5 releasystem environm

nary: there are

hoose ‘Do not serform a full ins3rd party softwause to even sta

arvelously lucidof self-containe

source code shPHP scripts dir

ber of definition

t.

cation

se on ment

some

set up stall are —rt up.

d ed aring

rectory

ns


used in thein the prev

// Fildefinedefine

Since we aThe readerof the DO_

CURLOPCURLOPCURLOP

In order to work. The and feed g

The Get Re(oauth_tok

R

R

ht

Note that thAuthorizati

Clicking the

P

We ignore the PIN… W

e other scripts. ious section:

ll in the ne('OAUTH_COe('OAUTH_CO

are behind a wer should provid_POST function

PT_PROXY =>PT_PROXYPORPT_PROXYUSE

authorize our SGet Request Tetreqtok.php to

equest Token sken_secret), an

Request Token

Request Token

ttp://api.twitter.

he URL containon API. Upon c

e ‘allow’ button

IN = 6582183

the instruction We will need th

Near the top o

next two conONSUMER_KEY'ONSUMER_SECR

eb proxy we alse their own pro

ns:

> 'your.proxRT => 80 ERPWD => 'pr

SNLBIRDIE apToken script (geo the PHP exec

Figure 1

script returns thd a URL:

= tew9CNU8xe

Secret = GhEK

com/oauth/aut&oauth_to&oauth_c

ns the Requestcopying and pa

Figure

n, we are rewar

to return to thehe PIN though

of the script we

nstants ', 'ilHsE6CRET', 'pfaG

so need to addoxy name, port

xy.here'

roxy_uid:pr

pplication to accetreqtok.php) ccutable. (Fig. 1

4. Upon runnin

hree items, a R

epcBtgmxFxbL

KtGRjmULRAq

horize?oauth_token_secret=G

callback_confirm

t Token and thasting the URL

e 15. Talking to

rded with a PIN

e SNLBIRDIE ain the next and

8

provide our Co

DOCQR9tZ5sJMqejR62un72

d a few Curl opt, and user–pas

oxy_pwd'

cess the SNLBcan be submitte

4)

ng the Get Req

Request Token

LmNDUkoQpCP

uTAoUehO5x4

token=tew9CNGhEKtGRjmULmed=true

e Request Tokin a browser w

o the Twitter Au

N-code (Fig. 16

application to ed final step of s

onsumer Key a

JUxQ'); 2kG3zc6ogEh7

tions to allow cssword, both in

BAF Twitter acced without edits

quest Token sc

(oauth_token)

PMmv904aglzw

4poVpDePuVF

NU8xepcBtgmxLRAquTAoUehO

ken Secret, andwe are asked to

uthorization AP

6):

enter the PIN, bsetting up our O

and Consumer

7VmC8Pq3pyU

connecting to thn the definition

count we need s. We open a c

cript.

), a Request To

w

FgSBgZMg

xFxbLmNDUkoO5x4poVpDeP

d feeds these to confirm acce

PI.

because there OAuth connecti

Secret as obta

UD8Zldr4');

he outside worof the DO_GE

to do some mocommand wind

oken Secret

QpCPMmv904PuVFgSBgZMg

to the Twitter ss. (Fig. 15)

is nowhere to ivity.

ained

rld. T and

ore ow

4aglzw g

enter


The last steWe edit theSecret, and

// Fil$reque$reque$oauth

We procee

The Get Ac(oauth_tok

A

A

Note that thall of your Anot currentindefinitely

So let’s put(tweet.phpmessage to

// Fil$acces$acces$tweet

Upon feedi

ep in preparinge Get Access Td PIN code:

ll in the nest_token='est_token_sh_verifier=

ed to feed the s

ccess Token scken), and an Ac

ccess Token =

ccess Token S

he Access TokAPI calls” indictly expire Accesy. Or at least un

t everything we). We edit the so be tweeted.:

ll in the nss_token='2ss_token_set = 'Frabjo

ing the tweet.p

Figure 16

g our Twitter OAToken script (ge

next 3 variatew9CNU8xep

secret='GhEK= 6582183;

script to PHP. (

Figure 1

cript returns theccess Token Se

= 256076484-3c

Secret = a2nS1

ken starts with acates the end oss Tokens andntil Twitter chan

e did so far to tscript to contain

next 2 varia256076484-3cecret='a2nS1ous day!';

php script to PH

6. The SNLBIRD

Auth connectivetacctok.php) b

ables. pcBtgmxFxbLKtGRjmULRAq

Fig. 17)

17. Upon runni

e two essentiaecret (oauth_to

cKiB4ae5saibK

wLtL9YtIk1me

a user ID. The of all the manua Access Token

nges its policy…

the test and seen our SNLBAF

ables. cKiB4ae5sai1wLtL9YtIk1

HP we get a loa

9

DIE application

vity is to requesby inserting ou

mNDUkoQpCPMuTAoUehO5x4

ng the Get Acc

l items we’ll neoken_secret):

K7TXYQAOvm

CqozY4q0LW0

phrase “Use thal steps which wn Secrets. The …

e whether we cAccess Token

bK7TXYQAOvmmeCqozY4q0L

ad of details ba

n is now author

st an Access Tor newly obtaine

Mmv904aglzw'4poVpDePuVFg

cess Token scr

eed to start twe

FcDMTF1bCw

01ECHsO9agG

he new oauth_we had to go thones which we

can now use Jon and Access T

mFcDMTF1bCwXLW01ECHsO9ag

ack from the Tw

rized.

oken and Acceed Request To

'; gSBgZMg';

ript.

eeting: an Acce

wX153orU

GK0kQ

_token and oauhrough. And ine just obtained

oe Chung’s TwToken Secret, a

X153orU'; gGK0kQ';

witter API. (Fig.

ess Token Secoken, Request T

ess Token

th_token_secrdeed, Twitter d should remain

weet script and provide the

. 18)

ret. Token

ret for does n good

e


And indeedthe ‘via SN

d, over at the SNLBIRDIE’ footn

F

Support center,note, indicating

igure 18. Upon

the TweetDecg that an applic

10

n running the Tw

ck shows the Scation SNLBIRD

Tweet script.

erver’s first scrDIE sent the tw

ript-generated weet on behalf o

tweet! (Fig. 19of SNLBAF.

) Note


APPLICANow that wfollowing thtweet.php stweet_tempelegance ocoding stan

TOO MUCIn our first gigantic da90% full, therror. The

option %let m%let t%let s%let d%let w filenafilenafilena data _ leng cm wo work cmd= callrun;

Figure 19. SNL

ATIONS we can send twhree examples script to send cplate.php. The or performancendards, nor wit

CH WORK! example we try

ata set by a zillihe Support stafSAS batch job

ns noxwait;

max_bytes threshold_pscripts_dirdir_out worklib

ame dirout ame tweettmame tweetmo

_null_; gth md $ 100 ork_volume k_volume=su="dir"||" "l system(cm

LTECHSUPP c

weets by callingare derived fro

custom messagreader should , but rather for th ours for that

y to detect the on variables atff can hopefullyto keep an eye

= 250048pct = 90; r = c:\whe = c:\tem = %sysfu

"&dir_outmp "&scriptsod "&scripts

$ 3; ubstr(symget||work_volu

md);

can see the scr

a script, the poom frequently rges in each casalso keep in mconceptual simmatter…

WORK library t once. By definy take action bee on WORK-sp

8507904;

erever\the\mp\dir_out.unc(getopti

t"; s_dir\tweets_dir\tweet

t("worklib"ume||" > &d

11

ript-generated t

ossibilities for sreported SAS Tse, and for rea

mind that the complicity. That is

running out of ning a suitable efore the entirepace could look

php\scriptstxt; on(work));

_template.p.php";

),1,3); ir_out.";

test tweet from

sending real-timTechnical Suppasons of code rode presented s, it’ll do the job

disk space, e.g threshold, say

e environment gk like this:

s\are;

php";

m SNLBAF in Tw

me diagnosticsport issues. Wereadability we rhere has not b

b but it may no

g. because a uy the WORK vogrinds to a halt

weetDeck.

s are endless. Te will use the rename the scrbeen optimizedt comply with y

user is trying toolume running ot on a ‘Disk Ful

The

ript to for

your

o sort a over ll’


12

data foo; infile dirout truncover; input line $60.; run; data _null_; length from to positions bytes_free pct_full 8; set foo end=last; if last then do; from=find(line,')')+2; to=find(line,'bytes'); positions=to-from; bytes_free=input(compress(substr(line,from,positions),','),20.); pct_full=round(100-bytes_free/"&max_bytes"*100); call symput('pct_full',trim(left(put(pct_full,2.)))); end; run; data _null_; length bytes 8 tweet_msg $ 140; retain tweet_msg; if _n_=1 then tweet_msg="$tweet = 'the WORK volume is "||"&pct_full" ||"% full - %sysfunc(datetime(),datetime19.)';"; infile tweettmp; file tweetmod; input; if not (_infile_=:"$tweet") then put _infile_; else put tweet_msg; if &pct_full < &threshold_pct then call symput('tweetflag','0'); else call symput('tweetflag','1'); run; data _null_; if &tweetflag=1 then call system('php'||" "||"&scripts_dir"||'\tweet.php'); run;

A few comments about the code:

We will interact with the OS through a DOS command window. Setting the NOXWAIT option allows the SAS code to continue running instead of waiting for a human to close the command window.

Some global variables and assorted definitions: Max_bytes is the total storage capacity of the volume where the WORK library lives, expressed in bytes; Threshold_pct is the volume usage percentage above which we want to send a warning tweet; Scripts_dir is the directory where Joe Chung’s PHP scripts were saved; Dir_out points to a txt-file which will capture the output of a DIR command; Worklib is the directory path to the WORK library; Tweettmp points to the tweet_template.php script; Tweetmod points to the tweet.php script, which will be the modified copy of the template to be executed.

We extract the volume name from the WORK library path, and using CALL SYSTEM we run a dir command on it while piping the output thereof to the text-file dirout.

Read the output of the dir command and store as text strings in a temporary data set.

The last line of dir command output contains the number of free bytes on the volume, so we parse the last observation to pry the number (bytes_free) out of the text, and calculate the percentage full for the volume (pct_full).

Using pct_full we construct the actual tweet-message (tweet_msg), or rather the entire line in the tweet_template.php script that needs to be replaced to send a custom tweet. Note that we include a datetime stamp within the body of our tweet. We did this while developing and testing the code because Twitter imposes restrictions on tweeting the same message multiple times. We found it rather practical however since it shows the


tweet-ti

We thedefinedtweet_mmessag

If the pethen theotherwi

If the tw

When the t

ANYBODIn this examMeasuremcan be conThis is dongenerates

option %let s%let e%let x libnamfilenafilenafilena data _ file put put run; x &xcm

ime quite visibl

en read the twed, we simply oumsg we construge substituting

ercentage full fe tweetflag is sise nothing sho

weetflag is set,

threshold is pa

Figure 20. The

DY HOME? mple we use daent (EBIAPM)

nfigured to run ne by schedulinthe sas92_bi_h

ns noxwait;

scripts_direbiapm_homexcmds

me apmdataame xcmds ame tweettmame tweetmo

_null_; e xcmds; "cd &ebiap'runchecks

mds;

y in the Tweet

et_template.phutput the _INFILucted earlier. Tthe default one

for the volume set to 1. For theows.

we call PHP a

ssed, a tweet i

e SNLTECHSU

ata generated package as refregular checks

ng the runcheckhealth data set

r = c:\wheree = c:\sas\e = c:\temp\

a "&ebiapm_ "&xcmds";

mp "&scriptsod "&scripts

pm"; server.bat';

Deck app, so w

hp script line byLE_ buffer agaThe result is a ce.

is below the the purpose of th

nd feed it our c

s sent and it sh

UPP TweetDec

by the SAS 9.2ferenced in the

s on a whole lotkserver.bat (wht, which we ins

ever\the\phebiedieg\le\xcmds.bat;

_home\data\; s_dir\tweets_dir\tweet

;

13

we decided to l

y line. Unless tin. For the line copy of the twe

hreshold we sethis demo we ha

customized twe

hows up on the

ck shows the w

2 Enterprise Bue introduction. Wt of things. Likehich comes witpect in the follo

p\scripts\av1\ebiapm92

status";

_template.p.php";

eave it in.

he line is the o commencing ‘

eet_template.ph

t a tweetflag toave temporarily

eet.php script.

e Support cente

warning tweet fr

usiness IntelligWhen a SAS ae whether any h EBIAPM) jobowing code:

are; 2;

php";

one where the b‘$tweet’ we subhp script but w

o zero. If it excey lowered the th

er’s TweetDec

rom the SNLBA

ence Audit andadministrator inof the vital SAS

b to run periodi

body of the twebstitute the

with our own

eeds the threshhreshold to 70%

k. (Fig. 20)

AF server.

d Performancenstalls EBIAPMS Servers are dcally. The batc

eet is

hold %

, it down.

ch job


14

data servers_down; set apmdata.sas92_bi_health; if server_status=0 then output; run; data _null_; dsid=open("servers_down"); nrobs=attrn(dsid,"nobs"); rc=close(dsid); if nrobs=0 then call symput('tweetflag','0'); else do; infile tweettmp end=last; file tweetmod; input; if not (_infile_=:"$tweet") then put _infile_; else put "$tweet = 'One or more SAS Servers are down - %sysfunc(datetime(),datetime19.)';"; if last then call symput('tweetflag','1'); end; run; data _null_; if &tweetflag=1 then call system('php'||" "||"&scripts_dir"||'\tweet.php'); run;

The structure of the code is similar to the previous example. Items that were already explained or commented upon will not be reiterated here:

Some global variables and assorted definitions: Ebiapm_home is the home directory of our EBIAPM installation; Xcmds points to a text-file which we’ll use to run a few X commands in one go; Apmdata points to the location where the EBIAPM runcheckserver.bat job writes its data sets;

The OS commands to run the EBIAPM runcheckserver.bat job are written to a text-file and subsequently executed by means of an X statement. Note that ordinarily the frequency of running this particular job should be kept as low as possible due to the potential impact on system performance.

We then inspect the output of the runcheckserver.bat process — the data set sas92_bi_health — and retain only those records where a server down condition was detected.

If any servers are found to be down we create a custom tweet.php from the tweet_template.php script by inserting an appropriate tweet message in the right place.

And run the script if necessary…

An obvious improvement to the above would be to send more specific tweets based on the contents of the servers_down data set, explicitly mentioning in the tweet body which servers are impacted. For now, figure 21 shows the generic warning popping up in the Support center’s TweetDeck.


BOGGEDAs a final eprocesses environmereference ssessions th

The standarun as the Pooled Wooutput of aindividual u

In order to previously us/sysinter

The followi

option %let s%let p%let x filenafilenafilenafilenafilena data _ file put put put run; x &xcm

Figure 2

D DOWN example we attgobbling up Cnt is configuredsystem howevehat are typically

ard set-up on W‘NT AUTHORI

orkspace Serve DOS tasklist c

users’ accounts

monitor how loby SysInternal

rnals/bb896649

ing code detec

ns noxwait;

scripts_dirpstools_homxcmds

ame pslist ame tsklistame tweettmame tweetmoame xcmds

_null_; e xcmds; "cd &pstoo"pslist sa"tasklist

mds;

21. Another om

tempt to detectPU cycles and d and used it mer, it is not. So y active in a SA

Windows is thatTY\SYSTEM’ u

er is running ascommand, any s.

ong processes s, nowadays to

9

ts user Worksp

r = c:\whereme= c:\pstoo = c:\temp\

'c:\temp\t 'c:\temp\mp "&scriptsod "&scripts "&xcmds";

ols_home"; as > c:\temp/V /FI ""im

minous warning

t system perforbogging down

may be perfectlyhow do we det

AS environmen

t the Metadata user. Furtherms the SAS GeneSAS.exe proce

have been runo be found on t

pace Server pro

ever\the\phols; \xcmds.bat;

\pslist.txt\tasklist.ts_dir\tweets_dir\tweet;

p\pslist.txmagename eq

15

g shows up in th

mance degradn the entire systy legitimate to tect the relevannt?

Server, OLAPore, everythingeral Server Useesses left are W

nning we use ththe MS site: htt

ocesses runnin

p\scripts\a

'; xt'; _template.p.php";

t"; sas.exe""

he Support cen

ation by hangintem. Dependinhave long-runnnt SAS.EXE pr

P Server, Conneg running on aer (SASSRV). Workspace Se

he PsList utilitytp://technet.mic

ng for more tha

are;

php";

> c:\temp\t

nter’s TweetDe

ng user Worksng on how exacning user procerocesses amon

ect Server, andStored ProcesSo if we filter trver processes

y which is part ocrosoft.com/en

an 2 days:

tasklist.tx

eck.

space Server ctly the SAS esses. In our ng the mass of

d Share Serverss Server and athose out of thes running unde

of the PsTools -

xt";

SAS

r all a e r the

suite,


16

data pslist; infile pslist firstobs=4; input name $ pid pri thd phd priv @49 cpu_time time11. @65 elapsed_time time11.; run; data tsklist; infile tsklist firstobs=4; input name $ pid system $ session memory comma10. kilobyte $ status $ user $40.; run; proc sql; create table processes as select tsklist.user, pslist.elapsed_time from pslist, tsklist where pslist.pid=tsklist.pid ; quit; data user_processes; set processes; if upcase(user) in ("NT AUTHORITY\SYSTEM","FOO\SASSRV") then delete; run; proc sort data=user_processes; by descending elapsed_time; run; data _null_; dsid=open("user_processes"); nrobs=attrn(dsid,"nobs"); rc=close(dsid); if nrobs=0 then abort return; run; data _null_; set user_processes(obs=1); if elapsed_time<172800 then abort return; run; data _null_; infile tweettmp end=last; file tweetmod; input; if not (_infile_ =: "$tweet") then put _infile_; else put "$tweet='One or more Workspace Server user processes is running longer than 2 days! - %sysfunc(datetime(),datetime19.) ';"; run; data _null_; call system('php'||" "||"&scripts_dir"||'\tweet.php'); run;

Some additional notes:

Some global variables and assorted definitions: Pstools_home is the home directory of our PsTools installation;


Xcmds Pslist pTsklist

In the xredirectsas.exe

After ruinto a p(PID), wavailab

We the

And filte

Then soobserva

If the resession

If the loalso ba

We onldays. W

Figure 22 s

CONCLUWe have senvironmeadequatelyHowever, imechanismTwitter by ucan be donA fun exerc

points to a texpoints to a temppoints to a tem

xcmds.bat file wt the output to te processes, an

unning these copair of data setswhereas the tskle in the text-fil

en join the user

er out anything

ort by descendation.

esulting data sen, preventing th

ongest running ail out.

y get to this paWe modify the t

shows the code

Figure 22

USION hown how to snt to tweet diagy protected agan case someth

m should providusing Joe Chune through SAScise!

xt-file which weporary text-file

mporary text-file

we first move tothe pslist.txt filend redirect the

ommands in bas. The pslist daklist data set wles, it’s interest

name to the e

g that is not run

ding elapsed tim

et is empty we he rest of the co

user process h

art of the code itweet message

e-generated tw

. Another warn

set up a mechagnostic messagainst downtime hing goes awryde some additiong’s PHP scripS code. To mak

’ll use to run a wherein we ca

e for capturing t

o our PsTools he. We also calloutput to the t

atch with an X sata set will essewill link PIDs to ting to see wha

lapsed process

nning under an

me, so that the

don’t need to dode being exec

has an elapsed

if there is at leae as in the prev

weet in the Sup

ing is tweeted

nism allowing Sges to a suppoby solid failove

, it is always goonal peace of mpts, but it shouldke things intere

17

few X commanpture the outputhe output of a

home directory the tasklist coasklist.txt file.

statement, we entially give ususers. For dida

at other proces

s time on PID.

individual user

longest runnin

do anything elscuted.

d time of less th

ast one user prvious examples

port center’s Tw

to the Support

SAS processesort center. Businer mechanismsood to involve tmind for SAS Pd be interestingesting, this wou

nds in one go;ut of a PsList cTasklist comm

y, then call the ommand with op

proceed to reas the elapsed timactical purposess-related inform

r account.

ng user process

se and ABORT

han 2 days (ex

rocess with an s, and call PHP

weetDeck.

t center’s Twee

s running in a Sness critical sys at all levels: dthe experts as

Platform adming to see whethuld require cod

command; mand.

PsList tool for ptions for verbo

ad the newly geme for each sa

es we also readmation is in the

s is represente

T RETURN kills

pressed here i

elapsed time oP to send it.

etDeck for follo

SAS Business ystems should odatabase, hardquickly as posistrators. We to

her the entire Oing SHA1 encr

SAS processeosity and filteri

enerated text fias.exe processd the other fieldere.

ed by the first

s the current SA

n seconds) the

of more than tw

w-up.

Analytics servof course be

dware, applicatssible. The propook a shortcut

OAuth authenticryption in Base

s and ng for

les ID

ds

AS

en we

wo

er

ions. posed to

cation e SAS.


18

RECOMMENDED READING • SAS 9.2 Enterprise Business Intelligence Audit and Performance Measurement for Windows Environments

http://support.sas.com/rnd/emi/EbiApm92/sas92.ebiapm.win.pdf

The installation and configuration guide for the EBIAPM package gives a good idea of the variety of SAS environment aspects that can be monitored by the utility.

• The Twitter Developers Authentication page http://dev.twitter.com/pages/auth

For those so inclined, we recommend having a look at the details of the OAuth authentication flow as explained on this site.

• RFC 5849 – The OAuth 1.0 Protocol http://tools.ietf.org/html/rfc5849

The complete RFC document describing version 1 of the OAuth protocol. This is hardcore.

ACKNOWLEDGEMENTS The authors should like to thank SAS Global Forum section chair Pete Lund for his interest in this somewhat maverick application of Social Media abuse in a SAS systems administration context, and the inimitable Joe Chung for making his excellent scripts publicly available and taking the bite out of OAuth!

CONTACT INFORMATION Your comments and questions are valued and encouraged. Contact the authors at:

William W. Viergever Viergever & Associates Health Data Analysis / Systems Design & Development 2920 Arden Way Suite N Sacramento, CA 95825 USA [email protected] www.viergever.net +1 (916) 483-8398 Daniël Kuiper / Koen Vyverman SAS Institute B.V. Flevolaan 69 1272PC Huizen The Netherlands [email protected]

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Other brand and product names are trademarks of their respective companies.


317-2011: the day the server said...

Documents