eclipse test & performance tools platform, part 2: monitor ...€¦ · the eclipse test &...

Eclipse Test & Performance Tools Platform, Part 2:Monitor applicationsCollect and analyze a variety of log files

Skill Level: Intermediate

Martin Streicher ([email protected])Editor in ChiefLinux Magazine

25 Apr 2006

In this "Eclipse Test & Performance Tools Platform" tutorial series, learn how to usethe capabilities of the Eclipse Test & Performance Tools Platform (TPTP) to convertapplication log files into a structured format. Then, using TPTP and other specializedtools designed to process and analyze log files, you can quickly discern usagepatterns, performance profiles, and errors.

Section 1. Before you start

About this series

Writing code for an application is the first stage in the long process required todeliver robust production-quality programs. Code must be tested to vet its operationand accuracy. Code must often be profiled to remove bottlenecks that impedeperformance and to remove wasteful or inadvertent use of resources, especiallymemory. Code must also be monitored -- to pinpoint failures, of course, but also toidentify usage patterns, opportunities for further enhancement and optimization, andattempted and actual intrusions.

The Eclipse Test & Performance Tools Platform (TPTP) is a software architectureand several realized components (so far) that extend the Eclipse platform to includetest, performance, and monitoring tools. This "Eclipse Test & Performance ToolsPlatform" series explores the capabilities of TPTP. Part 1 demonstrates how toprofile a Java™ technology application. Part 2 demonstrates how to capture and

Monitor applications© Copyright IBM Corporation 1994, 2008. All rights reserved. of 37

mailto:[email protected]

http://www.ibm.com/developerworks/views/opensource/libraryview.jsp?search_by=eclipse+test+performance+tools+platform:



http://www.ibm.com/developerworks/edu/os-dw-os-ecl-tptp-i.html

http://www.ibm.com/legal/copytrade.shtml

transform arbitrary log files to the widely supported Common Base Events (CBE)format. Part 3 explains how to manage application testing.

About this tutorial

This tutorial shows how to use the capabilities of the Eclipse TPTP to convert atypical application log file into CBE occurrences. With a modicum of specificationsand some light coding to create a series of rules, you can transform virtually any logfile into a unified, structured format. Then, using the Eclipse TPTP and otherspecialized tools, you can combine, process, and quickly discern usage patterns,performance profiles, and errors.

Objectives

In this tutorial, you learn how to write an adapter to transform a typical Linux®software service log file into CBE data. You create the transform incrementally anddebug the transform with the Eclipse TPTP Adapter Configuration Editor, then theGeneric Log Adapter (GLA) to input, transform, and emit the data.

Prerequisites

You should have experience with software development and the entire softwaredevelopment life cycle, including testing and profiling. You should also haveexperience installing software from the command line, and setting and managingshell and system environment variables, such as the shell's PATH variable and theJava CLASSPATH. Additionally, it's vital that you have some experience reading andwriting regular expressions. Acquaintance with Eclipse and the Eclipse userinterface (UI) paradigms is also beneficial.

System requirements

You can run Eclipse on any system that has a JVM, such as Solaris, Linux, Mac OSX, or Windows. If you don't have a JVM and Eclipse installed on your system, makesure you have at least 300 MB of disk space free for all the software. You also needenough free physical memory to run the JVM. In general, 64 MB or more of freephysical memory is recommended.

You must install several software packages on your UNIX®, Linux, Mac OS X, orMicrosoft® Windows® system. You need a functioning Java Virtual Machine (JVM),a copy of the Eclipse SDK, a copy of the Eclipse TPTP runtime, and severalprerequisites and co-requisites on which the Eclipse TPTP depends. You also needa copy of the Eclipse TPTP GLA, which allows you to transform log files in astand-alone application or in your own application. Here's everything you need:

• Download Java technology from Sun Microsystems or from IBM

developerWorks® ibm.com/developerWorks

Monitor applications of 37 © Copyright IBM Corporation 1994, 2008. All rights reserved.

http://www.ibm.com/developerworks/edu/os-dw-os-ecl-tptp3.html

http://java.sun.com/j2se/downloads/

http://www.ibm.com/developerworks/java/jdk/


• Eclipse V3.1 Software Development Kit (SDK)

• Eclipse Modeling Framework (EMF) SDK V2.1

• XML Schema Infoset Model (XSD) SDK V2.1

• Version 1.1.1 of Eclipse Unified Modeling Language (UML) 2

• The Eclipse TPTP runtime

• GLA runtime

Section 2. Transforming and analyzing log files

To allow ongoing monitoring, a complex application -- and certainly an applicationexpected to run continuously -- is typically instrumented during development to emita log file, which is a record of application activity. Some activity can be detailedinternal diagnostics, which is information crucial for isolating a bug or untanglinginteractions with other system and software components. Some activity logged mightbe initiated by the application itself -- say, to read a configuration file or to open aport for listening. Other activity might be generated by requests for service.

The problem: Ongoing monitoring for legacy applications

Depending on the application's purpose, a systems administrator might review theprogram's corresponding log file from time to time -- when an error occurs or even inreal time to react to emergent events. Logs are often full of valuable historicalinformation, too. Think of the traffic and usage patterns found only in Apache HTTPServer logs, for instance.

It would be ideal if all log files captured at least a minimum of information. It wouldbe even better -- certainly from a systems administrator's point of view -- if the formatof all log files was uniform. Consistency would make reading logs far easier, andhomogeneity would certainly facilitate (not to mention cheapen the expense of) thedevelopment of automated tools that weed out vital events from the informational.

But invariability is not reality. Applications differ greatly (as do underlying operatingsystem facilities and programming language libraries). Some applications areentrenched and cannot be revised ("legacy applications") to be brought to uniformity.And it's an ugly truth that expensive and scarce developer cycles are usually spenton new features, not retrofits.

The solution: Transform log file data

Short of the ideal and realizing that one solution cannot ever fit all, it is far more

ibm.com/developerWorks developerWorks®


http://www.eclipse.org/downloads/

http://www.eclipse.org/emf/

http://www.eclipse.org/xsd/

http://www.eclipse.org/uml2/

http://www.eclipse.org/tptp/home/downloads/downloads.php

http://www.eclipse.org/downloads/download.php?file=/tptp/4.1.0.1/TPTP-4.1.0.1/tptp.gla.runtime-TPTP-4.1.0.1.zip


practical to transform log file data to meet evolving standards, de-facto or otherwise,and to apply state-of-the-art analysis tools. For example, the CBE format is part ofan effort to define a broad standard for recording, tracking, and analyzing events,which are occurrences and situations that take place in computing systems. Manytools exist to process and analyze CBE data, which is based on XML.

But while transformation from arbitrary log file to CBE may be practical, the processmay not be easy or inexpensive. Given the variety of applications and the sheernumber of log file formats, writing so many transforms can be a Herculean task initself.

The Eclipse TPTP GLA and Adapter Configuration Editor simplify the creation oftransforms, thereby easing the migration to CBE. The GLA applies an adaptercreated by the Adapter Configuration Editor to a log file and yields CBE data. TheAdapter Configuration Editor can run a handmade Java class if need be -- a staticadapter -- or it can run a series of rules to divide the log file into records, fields, andvalues and reassemble them as CBE data. The latter form of adapter is arules-based adapter and requires no coding. Better yet, the Adapter ConfigurationEditor runs in Eclipse and provides a rich adapter development environment in whichyou can incrementally define and test your adapter. Finally, you can choose tointegrate the GLA with your own code or use third-party tools, such as the IBM Logand Trace Analyzer, to probe and investigate the resulting CBE event files.

This tutorial shows how to use the capabilities of the Eclipse TPTP GLA and AdapterConfiguration Editor to convert a typical Linux application log file to CBE events.With a log file in hand and a little regular expression know-how, you can transformthe log into a unified, structured CBE format.

Section 3. Installing the prerequisite software andcomponents

Before you can begin, you must install and set up the required software andcomponents (see Prerequisites).

Install J2RE V1.4

Download and install Version 1.4 or 1.5 (also called Version 5.0). (If your systemalready has J2RE V1.4 or later, you can safely skip this step.)

Typically, the JRE is distributed as a self-extracting binary. Assuming that youdownloaded the J2RE packages to your home directory, installation (on Linux) istypically as easy as Listing 1.

Listing 1. J2RE V1.4 installation




% cd ~% mkdir ~/java% cd ~/java% mv ~/jre-1_5_0_06-linux-i586.bin .% chmod +x jre-1_5_0_06-linux-i586.bin% ./jre-1_5_0_06-linux-i586.bin...

% rm ./jre-1_5_0_06-linux-i586.bin% ls -Fjre1.5.0_06/

The commands in Listing 1 install J2RE V1.5, but the steps to install J2RE V1.4 areidentical (except for the file name).

Install the Eclipse V3.1 SDK

Download the Eclipse V3.1 SDK that's appropriate for your platform. You can findthe SDK at the Eclipse Downloads. Typically, installation is as easy as unpackingthe Eclipse tarball (.tar.gz) file into the directory of your choice.

For example, if you're using Linux, download the Eclipse V3.1 SDK tarball andunpack it in a directory such as ~/java/ using the commands in Listing 2.

Listing 2. Eclipse V3.1 SDK installation

% cd ~/java% mv ~/eclipse-SDK-3.1.1-linux-gtk.tar.gz .% tar zxvf eclipse-SDK-3.1.1-linux-gtk.tar.gz...% rm eclipse-SDK-3.1.1-linux-gtk.tar.gz

To verify that you successfully installed Eclipse, remain in the directory where youunpacked Eclipse, make sure the java executable is in your PATH, and run java-jar eclipse/startup.jar. For example:

Listing 3. Verify installation

% export JAVA_DIR=$HOME/java% export JAVA_HOME=$JAVA_DIR/jre1.5.0_06% export PATH=$JAVA_HOME/bin% export CLASSPATH=$JAVA_HOME% cd $JAVA_DIR% java -jar eclipse/startup.jar

If Eclipse prompts you to choose a directory for your workspace, use$HOME/java/workspace. This directory retains all the projects you create in Eclipse.(Of course, if you have many projects, you can create other workspaces later,perhaps to contain one project per workspace.) Now, quit Eclipse to install theEclipse TPTP, its prerequisites and co-requisites, and the GLA.



http://www.eclipse.org/downloads


Install the TPTP and GLA runtime

The Eclipse TPTP runtime contains the software required to create, debug, and runadapters. To install the Eclipse TPTP software, download the Eclipse TPTP andGLA runtimes. Both are typically distributed in zip format. Move both files into thedirectory that contains the J2RE and Eclipse and extract them (see Listing 4). Ifyou're prompted to overwrite any files, simply choose All.

Listing 4. Eclipse TPTP and GLA installation

% cd ~/java% mv ~/tptp.runtime-TPTP-4.1.0.zip .% mv ~/tptp.gla.runtime-TPTP-4.1.0.1.zip .% unzip tptp.runtime-TPTP-4.1.0.zip...

% unzip tptp.gla.runtime-TPTP-4.1.0.1.zip...

% rm tptp.runtime-TPTP-4.1.0.zip% rm tptp.gla.runtime-TPTP-4.1.0.1.zip% ls -FGenericLogAdapter/ eclipse/ jre1.5.0_06/

Install the EMF SDK V2.1

You must install EMF SDK V2.1 for TPTP to work properly.

Quit Eclipse if it's running and download the EMF SDK V2.1. Then change to thedirectory that contains the Eclipse folder and run unzip emf-sdo-SDK-2.1.0.zip (seeListing 5).

Listing 5. EMF SDK V2.1 installation

% cd $JAVA_DIR% lseclipse jre1.5.0_06% mv ~/emf-sdo-SDK-2.1.0.zip .% unzip emf-sdo-SDK-2.1.0.zipcreating: eclipse/features/creating: eclipse/features/org.eclipse.emf.ecore.sdo_2.1.0/creating: eclipse/features/org.eclipse.emf_2.1.0/inflating: ......

% rm emf-sdo-SDK-2.1.0.zip

Install the XSD SDK V2.1

As with the previous file, change to the directory that contains the Eclipse directoryand run unzip xsd-SDK-2.1.0.zip (see Listing 6).

Listing 6. XSD SDK V2.1 installation




% cd $JAVA_DIR% mv ~/xsd-SDK-2.1.0.zip .% unzip xsd-SDK-2.1.0.zip% rm xsd-SDK-2.1.0.zip

If prompted to confirm the overwrite of any files, simply press y (lowercase) toanswer "yes" to each question.

Install the UML V2.0 Metamodel Implementation

To use the UML features of the Eclipse TPTP, you must install the UML V2.0Metamodel Implementation. If using Eclipse V3.1.1, download Version 1.1.1 of UML2 and unpack its archive file in the same directory that contains Eclipse (see Listing7).

Listing 7. UML V2.0 Metamodel Implementation installation

% cd $JAVA_DIR% mv ~/uml2-1.1.1.zip .% unzip uml2-1.1.1.zip...

% rm uml2-1.1.1.zip

Install the Agent Controller

The Agent Controller is a vital component of the Eclipse TPTP that allows Eclipse tolaunch applications and interact with those applications to extract profiling data.Download the Agent Controller runtime appropriate for your operating system. Next,create a directory named tptpd in the same directory that contains Eclipse andunpack the Agent Controller archive into that directory (see Listing 8).

Listing 8. Agent Controller installation

% mkdir $JAVA_DIR/tptpd% cd $JAVA_DIR/tptpd% mv ~/tptpdc.linux_ia32-TPTP-4.1.0.zip .% unzip tptpdc.linux_ia32-TPTP-4.1.0.zip

If you see two errors like these:


linking: lib/libxerces-c.sowarning: symbolic link (lib/libxerces-c.so) failed

linking: lib/libxerces-c.so.24warning: symbolic link (lib/libxerces-c.so.24) failed

recreate the two links manually by typing the following:





% cd $JAVA_DIR/tptpd/lib% rm libxerces-c.so libxerces-c.so.24% ln -s libxerces-c.so.24.0 libxerces-c.so% ln -s libxerces-c.so.24.0 libxerces-c.so.24

Add the Agent Controller directory

To use the Agent Controller, you must add its lib directory to yourLD_LIBRARY_PATH. For example, if you're running Linux and have adopted thesame directory structure shown in the steps above, you'd add $JAVA_DIR/tptpd/libas follows:

% export LD_LIBRARY_PATH=$JAVA_DIR/tptpd/lib:$LD_LIBRARY_PATH

You must also ensure that the contents of the Controller's lib and bin directories areexecutable. To do that, run:

% chmod +x $JAVA_DIR/tptpd/{bin,lib}/*

Now, add the scripts that configure, and start and stop the Agent Controller to yourPATH:

% export PATH=$JAVA_DIR/tptpd/bin:$PATH

Configure the Agent Controller for your environment

Finally, you configure the Agent Controller to match your environment. Change tothe Agent Controller's bin directory, then run SetConfig.sh.

% cd $JAVA_DIR/tptpd/bin% ./SetConfig.sh

When the configure script prompts you, accept the defaults. Running the configurescript creates the file config/serviceconfig.xml in the Agent Controller's hierarchy offiles.

Test the Agent Controller

To test the Agent Controller, run RAStart.sh. To stop the Controller, run RAStop.sh:


db% RAStart.shStarting Agent ControllerRAServer started successfully% RAStop.sh




RAServer stopped, pid = 5891RAServer stopped, pid = 5892RAServer stopped, pid = 5893RAServer stopped, pid = 5894RAServer stopped, pid = 5895RAServer stopped, pid = 5896RAServer stopped, pid = 5897RAServer stopped, pid = 5898RAServer stopped, pid = 5899RAServer stopped, pid = 5900RAServer stopped, pid = 5901RAServer stopped, pid = 5902RAServer stopped, pid = 5904RAServer stopped, pid = 5905RAServer stopped, pid = 5906

Finished! Restart Eclipse and you should see a new button on the Eclipse toolbarthat looks like Figure 1. That's the TPTP Profile button -- the indication that yourinstallation of TPTP has been successful. You're ready to continue with the tutorial.

Figure 1. The TPTP Profile button

Section 4. Creating an adapter

The GLA uses an XML configuration file to control how it parses and transforms logfiles, and how it emits data. A configuration file contains one or more contexts inwhich each context defines how to transform one log file. In some cases, contextswithin a configuration file can run simultaneously.

The adapter configuration file

Begin by creating an adapter configuration file to process the Linux log file nameddaemon.log. On your test system, running Debian Linux, daemon.log capturesmessages from the POP3 (e-mail), THTTPD (the "trivial" HTTP server -- a small, fastWeb server that only serves static files), and MyDNS (a small, easy-to-configureDomain Name System (DNS) server) daemons. Daemon.log also records when theMySQL daemon starts and stops.

Listing 12 shows a snippet of the file with log entries created by the POP3 andTHTTPD servers.

Listing 12. A snippet of the Linux daemon.log file

Mar 2 07:24:54 db popa3d[8861]: Sessionfrom 66.27.187.89Mar 2 07:24:55 db popa3d[8861]: \Authentication passed for joanMar 2 07:24:55 db popa3d[8861]: \




1422 messages (11773432 bytes) loadedMar 2 07:24:57 db popa3d[8861]: \0 (0) deleted, 1422 (11773432) leftMar 2 07:26:28 db thttpd[7784]: \up 3600 seconds, stats for 3600 seconds:Mar 2 07:26:28 db thttpd[7784]: \thttpd - 0 connections (0/sec), 0 maxsimultaneousMar 2 07:26:28 db thttpd[7784]: \map cache - 0 allocated, 0 active (0bytes)...Mar 2 07:26:28 db thttpd[7784]: \fdwatch - 1589 selects (0.441389/sec)Mar 2 07:26:28 db thttpd[7784]: \timers - 3 allocated, 3 active, 0 freeMar 2 07:27:35 db popa3d[8911]: \Session from 71.65.224.25Mar 2 07:27:35 db popa3d[8911]: \Authentication passed for martinMar 2 07:27:35 db popa3d[8911]: \1350 messages (10880072 bytes) loadedMar 2 07:27:36 db popa3d[8911]: \4 (11356) deleted, 1346 (10868716) leftMar 2 07:29:54 db popa3d[8963]: \Session from 66.27.187.89

Each context in an adapter configuration file defines six components: the contextinstance, the sensor, the extractor, the parser, the formatter, and the outputter. Thecontext instance sets parameters for the general operation of the transformation,including whether the log is appended to continuously and how frequently the log isamended. The remaining five components (conceptually) act in sequence, readinginput, performing a task, and passing results on for further processing (except forcertain outputters, which simply write results to a file or to the console):

• The sensor reads the log file in pieces until it reaches the end of the fileand pauses. Then, when the sensor detects that the log file has grown, itreads the additional data. The sensor passes its data to the next stage,the extractor.

• The extractor reads data and divides it into individual records. Oneregular expression defines what the start of a record looks like, andanother regular expression defines the end of record. Individual records,when identified, are passed on to the parser for additional processing.

• The parser reads one record at a time from the extractor anddecomposes each into fields and values. Furthermore, the parser canmake decisions based on the content of a record and apply one or moresets of rules to yield fields and values. For instance, if a log file indicatesthe start, interim, and end of an event, the parser can decompose eachrecord into a set of fields and values unique to that event. Ultimately, theparser's objective is to map fields and values in each log file entry to theproper elements, attributes, and values in a CBE XML record. Theformatter reads the output of the parser.

• The formatter's job is simple: It reads the elements, attributes, and valuesthe parser creates, and it creates an object suitable for consumption bythe last stage in the context, the outputter.




• And the outputter consumes objects from the formatter and emits theobject. Outputters can emit XML to a file or to the console. They can alsocreate a new log file or pass the data to a daemon.

The next five sections describe how to define each of the six components of acontext.

Create an adapter configuration file

To begin, create a simple Eclipse project to contain the adapter configuration file:

1. Click File > New and expand Simple. Choose Project and click Next.

2. Name the project My Adapter and click Finish.

3. Click File > New > Other and expand Generic Log Adapter. ChooseGeneric Log Adapter File and click Next.

4. Choose My Adapter and name the adapter file my.adapter. Click Next.

5. Choose a template for the log file you want to process with this adapter(see Figure 2).You can use a snippet of the actual log file you want to process or anaccurate representation of the log file -- say, from a detailed specification.Click Browse, navigate to the file system, and open the template. Aftermaking your selection, click Finish. Click Yes when prompted to switchperspectives.

Figure 2. Choose a template that represents the log file to transform




Figure 3 shows the Generic Log Adapter perspective. As you can see, the UIdisplays a context instance in which the sensor's properties point to the template logfile you just chose. The context instance also includes an extractor, a parser, aformatter, and an outputter, which you must define further.

Figure 3. The Generic Log Adapter perspective




Configure the context

Each context instance describes how to process one log file. You can set severaloptions in a context instance. To see the options, click Context Instance belowConfiguration. You should see a panel that resembles Figure 4.

Figure 4. Context instance options




You can edit the Description to capture the intent of this particular context. Inaddition:

• If your log file is continuously updated, as is the case with daemon.log,select the Continuous operation check box.

• Maximum idle time is the number of milliseconds a context should waitfor the log file to change before the context instance is shut down.

• Pause interval controls how long the context should wait after it reachesthe end of a log file.

• Because log files aren't only ASCII text, you can set the ISO languagecode (using two lowercase letters), the ISO country code (using twouppercase letters), and the file's Encoding (using a value from theInternet Assigned Numbers Authority (IANA) character set registry). Bydefault, these parameters are set to en, US, and the default encoding ofthe JVM.

• Finally, because some log files do not denote time zone, year, month, andday and because CBEs require all four values, you can provide substitutevalues in the Timezone GMT offset and Log file creation date fields.

Because daemon.log grows continuously, select the Continuous operation check




box. Because mail is typically polled often, set Maximum idle time and Pauseinterval to 120. The test machine is located in Colorado, so the GMT is -7.Daemon.log doesn't specify the year, so a default of 2006 is provided as asubstitute. After making these changes, save the file.

Section 5. Specifying the sensor

A sensor reads a log file and forwards the data collected to the extractor. The nextstep is to specify how your sensor should work.

Specify how the sensor works

Click the sensor. Its properties are shown in Figure 5, which also shows the valuesset for the daemon.log sensor.

Figure 5. Setting the sensor for daemon.log

Because daemon.log is a single file, you don't need to change the Sensor typeoption. The Description field provides for clarity of purpose. Maximum blockingdefines the number of lines to read before passing the input along to the extractor.Because entries in daemon.log tend to span many lines, 10 is a reasonable setting.The value for Confidence buffer size dictates the size of a buffer to contain the lastn bytes of the log file. If the log file changes -- that is, the last n bytes differ fromwhat's retained in the Confidence buffer -- the sensor reads more input. The defaultis 1,024 bytes, which is sufficient for this example.

Some logs append a footer to the end of the log file (each time new data is written).Usually, this data is best ignored, so to skip the footer, specify the number of bytesto skip in File footer size. Daemon.log doesn't have a footer, so the value is set to0.

If you expand the Sensor type (by clicking on the arrow), you'll see two additional




properties: directory and fileName. These properties are initially set to the locationand name of your template log file, but you'll soon switch them to process live data.

Don't forget to save the configuration file after setting the sensor properties. And, ingeneral, always save the configuration file before you attempt to run the adapter.

Section 6. Editing the extractor

The role of the sensor is to collect input. The role of the extractor is to divide theincoming input stream into individual records. (The next component in the chain --the parser -- divides each record into fields.)

Configure the extractor properties

To edit the extractor, click Extractor. Its properties are shown in Figure 6. Theproperties of the extractor specify the delimiters of each record and control whetherthose delimiters should be included in the record passed on to the parser.

Figure 6. The extractor properties




In the example log file, daemon.log, each line of the log is a separate event. Thismakes the extractor particularly easy to configure. (Figure 6 is the appropriateconfiguration for daemon.log.)

• The Contains line breaks check box is cleared, because each line indaemon.log is a record. However, if an entry were to span many lines, asis the case with MySQL or IBM DB2® database logs, you'd select thischeck box.

• The Replace line breaks check box is also cleared in this example. If thelog file contained line breaks, though, you could select this check box toeither delete each line break or replace each one with a special marker --useful for parsing. To delete line breaks, simply select the check box; toreplace each line break with a token, select the check box and provide thedelimiter in the Line break symbol field. It's best to choose a symbol thatdoesn't appear in the log file.

• The Start pattern and End pattern are regular expressions that describethe start and end of each record. Here, where each line is a record, thebeginning of the line, or ^ (caret), marks the start of the record. The endof the line, or $ (dollar sign), marks the end of each record. Because ^and $ do not capture any content, neither need be included in the record




itself.

Save your work before continuing.

A MySQL example

For comparison, create another example extractor for MySQL's slow query log, aspecial log used to capture suboptimal queries. Each entry in the slow query logspans at least three lines (see Listing 13).

Listing 13. A snippet of MySQL's slow query log

# Time: 030207 15:03:33# Query_time: 13 Lock_time: 0 Rows_sent: 0 Rows_examined: 0SELECT l FROM un WHERE ip='209.xx.xxx.xx';# Time: 030207 15:03:42# Query_time: 17 Lock_time: 1 Rows_sent: 0 Rows_examined: 0SELECT l FROM un WHERE ip='214.xx.xxx.xx';# Time: 030207 15:03:43# Query_time: 57 Lock_time: 0 Rows_sent: 2117 Rows_examined: 4234SELECT c,cn,ct FROM cr,l,un WHERE ci=lt AND lf='MP' AND ui=cu;

An extractor for the slow query log might look something like Figure 7.

Figure 7. A sample extractor for the MySQL slow query log




Figure 8 shows the second of the three records, each successfully processed by theextractor.

Figure 8. An extracted record from the slow query log




Testing your work so far

Returning to the daemon.log adapter, you can now test the sensor and extractorcomponents to verify that data is being acquired and divided into records.

Rerun the adapter

Glance at the two panes at the bottom of the Generic Log Adapter perspective. Youshould see something resembling Figure 9. At left is the Extractor Result pane; atright, layered, are the Formatter Result pane, the Sensor Result pane, and theProblems pane. A series of buttons that control the adapter appear within theExtractor Result pane. Figure 10 labels the buttons (or you can slowly mouse overeach button to see a tool tip.)

Figure 9. Context components display panes




Figure 10. The adapter control buttons

Click Rerun adapter to restart processing from the beginning of the log file template.Then click Next event to process the first event.

• The Sensor Result pane should show the first 10-20 lines of the log file.

• The Extractor Result pane should show the first line of the log file, Mar 206:27:35 db popa3d[7964]: Session from 71.65.224.25.

• The Problems pane should be empty. However, pay close attention to thispane whenever you run your adapter. If you've omitted required CBEproperties, specified an illegal regular expression, or used anunsupported value, this pane should point those out.

• The Formatter Result pane is irrelevant because a parser has yet to bedefined. However, it does show an initial XML CBE for the current record:Listing 14. Initial XML CBE for current record

<CommonBaseEventcreationTime="replace with our message text"globalInstanceId="A1DAABE6C7876D20E8E9E8C475042F1B"version="1.0.1">

</CommonBaseEvent>




As you'll see, as you define your parser, additional elements and attributes willautomatically be added to the XML.

To have the extractor produce the next record, click Next event again. Tofast-forward to the last record (in the input the sensor has collected so far), clickShow last event.

Section 7. Producing the parser

The sensor reads data. The extractor subdivides the data into records. The role ofthe parser is to extract specific fields from each record and use those values toconstruct a complete CBE XML record.

The role of the parser

The parser may extract some fields from the log file directly, such as a time stamp,host name, daemon name, and a text message. The parser may also infer data froma record. For example, the parser may detect that the record originated with asoftware service and set the CBE componentIdType attribute to ServiceName. Inother instances, the parser may add data to a record. In particular, if a log entrydoesn't record the day, month, year, time, and time zone of the event, the parsermust add that data to create a valid CBE.

To put the parser for the daemon.log example in perspective, Listing 15 shows avalid CBE XML record for the log entry Mar 2 06:27:35 db popa3d[7964]:Session from 71.65.224.25. Some of the attributes are plainly derived fromthe original log entry; others will be manufactured from implied data. (Many of thevalues of the attributes come from the Common Base Events Specification. It'shelpful to use that document while creating your parsers.)

Listing 15. The CBE equivalent of the first record of daemon.log

<CommonBaseEventcreationTime="2006-03-02T13:27:35.000Z"globalInstanceId="A1DAABECA2ACB4F0E8E9E8C475042F1B"msg="Session from 71.65.224.25"version="1.0.1">

<sourceComponentIdcomponent="popa3d"componentIdType="ServiceName"location="db.linux-mag.com"locationType="Hostname"subComponent="7964"componentType="daemon"

/><situation



http://www.ibm.com/developerworks/autonomic/books/fpy0mst.htm#HDRAPPA


categoryName="StartSituation"><situationType

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:type="StartSituation"reasoningScope="EXTERNAL"successDisposition="SUCCESSFUL"situationQualifier="START INITIATED"/>

</situation></CommonBaseEvent>

Also keep in mind that (at a minimum) every CBE must define the creationTimeattribute, the msg attribute, and the sourceComponentId element, which in turnmust have the six attributes shown in Listing 15. The situation element (amongothers) is optional, but is part of the example to elaborate upon the event.

How the parser works

Click Parser in the Generic Log Adapter perspective to begin the process of definingit. Figure 11 shows what the completed parser looks like. There is one parser taskfor every attribute and element in the CBE shown in Listing 15.

Figure 11. The complete parser for daemon.log

The parser works in two phases. First, it divides the incoming record (from the




extractor) into positions, or numbered parts, in which each part is separated from theother by the separator token. If no separator token is specified, this step is skipped.Then the parser divides the record into designations, or (name, value) pairs, in whicheach (name, value) pair is two strings joined by the designation token. If nodesignation token is specified, the latter step is skipped.

Consider this example: If the separator token is the regular expression [ ]+, thedesignation token is = (equal sign), and the parser is handed the record:

03/05/06 12:51:06EST Mail name=joe action=login authentication=password

the parser would define six positions and three designations, as shown in Table 1.

Table 1. Positions and designations from the parserPosition/Designation Value

1 03/05/06

2 12:51:06EST

3 Mail

4 name=joe

5 action=login

6 authentication=password

h{'name'} joe

h{'action'} login

h{'authentication'} password

Note: If your incoming record begins with the separator token, position 1 is created,but left empty.

You can use all the defined positions and designations to simplify each parser task.For instance, to create the creationTime attribute, you need only parse position 2.Of course, the entire original record is always available. However, positions anddesignations make each parsing task faster and easier to manage because thesource string is smaller. In many cases, you can use a position or designationdirectly for a CBE value.

Parse the sample log entries

Click Parser again. For convenience, break each daemon.log entry into twopositions using the separator token :[ ]+ (a colon followed by one or morespaces). The daemon.log log entries don't have (name, value) pairs, so thedesignator token is omitted. These settings are shown in Figure 12. Now, save yourwork.

Figure 12. Dividing a record into positions




Set the creationTime

Set the first required field in the CBE: creationTime. The goal is to transform thetime stamp provided with the daemon.log record into a time format compatible withthe XML schema dateTime data type. As a convenience, the adapter canautomatically permute a time format understood by classjava.text.SimpleDateFormat into the XML schema data type.

To set the creationTime field, complete these steps:

1. Expand the parser and select creationTime. This is a required CBEattribute, so select the Required by parent check box.

2. Click the substitution rule associated with creationTime.

3. For Positions, type 1 because position 1 contains the time stamp toextract.

4. For Match, provide the regular expression^(\w{3})\s+(\d{1,2})\s+([\d:]+)\s+.*$. This expressioncaptures the month name as $1, the day of the month as $2, and the timeof day as $3.



http://java.sun.com/j2se/1.4.2/docs/api/java/text/SimpleDateFormat.html


5. For Substitute, supply $1 $2 @YEAR $3 @TIMEZONE.Substitute is used instead of the entire incoming record in the rest of thisspecific parsing task. $1, $2, and $3 came from the previous step.However, because the time stamp doesn't include a year or a time zone,the year and time zone associated with the current context instance,represented by the shorthand @YEAR and @TIMEZONE, respectively, areused instead. Therefore, for the first daemon.log record, the settings inSubstitute yield the string Mar 02 2006 06:27:35 -0700.

6. Ignoring the Substitute extension class field, which allows you toprovide a Java class to do additional substitutions, transform the result ofthe substitution to the right type. You can use ajava.text.SimpleDateFormat format string to do the heavy lifting.Set Time format to MMM dd yyyy hh:mm:ss Z, indicating athree-letter name of the month; a two-digit day of the month; a four-digityear; hours, minutes, and seconds separated by colons; and an RFC 822time zone.

Figure 13 shows the final settings for creationTime. If you save the configurationfile and rerun the adapter, the Formatter Result pane should show a new XMLrecord with attribute creationTime="2006-03-02T13:27:35.000Z".

Figure 13. Parsing the incoming time stamp into the creationTime attribute



http://java.sun.com/j2se/1.4.2/docs/api/java/text/SimpleDateFormat.html#rfc822timezone


Getting the message

The msg attribute is another required CBE attribute. Add this attribute and create theparser task to extract a suitable value:

1. Right-click CommonBaseEvent, then click Add > msg.

2. Click msg, then select the Required by parent check box.

3. Expand msg, then click Substitution Rule.

4. Specify 2 in the Positions field because the message portion of the logentry is located in position 2. (It's everything after the separator token.)

5. For Match, specify a regular expression that selects the entire string. Theregular expression ^(.*)$ captures everything in $1.

6. For Substitute, specify $1.

Figure 14 shows the final settings.

Figure 14. Settings to extract the message




Save the configuration file and click Rerun adapter, found in the Extractor Resultpane. Click Next event and switch to the Formatter Result pane. You should see anew msg attribute that looks like msg="Session from 71.65.224.25".

Find the source

The last mandatory part of a CBE record is the sourceComponentId, used torecord the component (service, system, and so on) that's affected by the event. Inthe instance of daemon.log, the components affected are software services runningon a specific host. The parser's job is to capture and record the specifics.

Right-click CommonBaseEvent once again, and then click Add >sourceComponentId. (Figure 15 shows all the possible attributes and elements youcan add to a CBE.) For brevity, Table 2 shows all the settings required forsourceComponentId. One new setting is Default value. If a match is made by aparsing rule, but no substitute value is provided, the Default value is used.

Figure 15. List of elements and attributes you can add to a CBE record




Table 2. Settings for the sourceComponentIdItem Default

valueRequired byparent

Positions Match Substitute Notes

component Yes 1 ^.* db(\w+)\[.*$

$1 Captures thename of thesoftwareservice,such aspop3ad ormysqld.

componentIdTypeServiceNameYes ^(.*) Indicatesthat thecomponentrecords thename of aservice;ServiceNameis one of theprescribedvalues forthis attribute,




according tothe CBEspecification.

componentTypedaemon Yes ^(.*) Describesthe class ofthecomponent.

location db.linux-mag.comYes ^(.*) Specifies thephysicaladdress thatcorrespondsto thelocation of acomponent.The formatof the valueof thelocation isspecified bythelocationTypeproperty. It isrecommendedthat you usea fullyqualifiedhost namefor thisattribute.Here,because thelog entrydoes notinclude ahost name,one is addedvia thedefaultvalue. Inother cases,you may beable to parsethe hostnamedirectly fromthe log.

locationType Hostname Yes 1 ^(.*) Specifies theformat andmeaning ofthe value inthe locationproperty.TheHostnamekeyword isone of manypossible




keywordsthat you canuse here.

subComponent Yes ^.*\[(\d+)\].*$1 Identifies thespecificdaemonprocess thatthe eventaffects.

If you make all the changes listed in Table 2, and save and rerun the adapter, youshould yield CBE event records that resemble Listing 15. As an additional exercise,add a situation to the CBE. Situations categorize the type of situation thatinitiated the event. For instance, you might create a parser to create aStartSituation whenever the daemon is initially contacted for service or createanother parser to create a RequestSituation when a request is made.

Situations aren't required (hence, Required by parent can be disabled), but youmay find them useful to add granularity to your CBE records. If you create a situationand add a series of possible situation parsers, select the Child choice check box ifprocessing can stop after the first match is made.

Here's a helpful tip for debugging your parsers: If a property is required, but notfound in the incoming record (passed to the parser from the extractor), the FormatterResult pane for that record will be empty. In other words, required properties behavelike logical AND: If one match fails, processing for that record stops. It's often usefulto clear the Required by parent check box to debug rules. Build your rules slowlyand incrementally, and watch the Problem pane for clues.

Section 8. The formatter and organizing the outputter

Now that the parser has yielded properties and values, the new data must beassembled into a CBE instance. That's the role of the formatter.

Emit CBE XML records to a file

The adapter formatter requires no configuration. It's an internal operation thatcreates CBE objects that conform to the CBE V1.0.1 specification.

After the formatter has created CBE objects, it's the job of the outputter to emit themto a file, standard output, another log, a logging agent, or a log analyzer. If youradapter configuration defines multiple contexts, you can use a special formatter toallow multiple contexts to write to a single file.

To keep things simple, emit the CBE XML records to a single file:



http://www.ibm.com/developerworks/autonomic/books/fpy0mst.htm#HDRSIT


1. Click Outputter in the Generic Log Adapter perspective, then chooseSingleFileOutputter for Outputter type.

2. Right-click Outputter, then click Add > property.

3. Click the new property, then set Property name to directory. Set theProperty value to a directory to which you're able to write files. Omit thename of a file. Just specify the path of the directory, omitting the trailingslash.

4. Right-click Outputter again and click Add > property. Set this newProperty name to fileName, and set the Property value to a file name.This file will be created in the directory named by directory.

Change the context instance

In addition to changing the configuration, you must also change the context instanceto use the proper outputter class. To do so, complete these steps:

1. Expand Contexts in the General Log Adapter perspective and expandContext Basic Context Implementation.

2. Click Component Logging Agent Outputter.

3. Change the Name and Description to Single File Outputter.

4. Change the Executable class toorg.eclipse.hyades.logging.adapter.outputters.CBEFileOutputter.

5. Save the configuration file.

Add the SingleFileOutputterType

There is one more important step: For some reason, the Adapter ConfigurationEditor can omit an important element from the outputter definition in theconfiguration file for the adapter. (You can read the relevant thread on thedeveloperWorks Autonomic computing forum's No Output from Outputter.) However,you can quickly add the element to the file manually.

Using your favorite editor, open the file my.adapter. Scroll to the bottom of the fileand look for the following text.

Listing 16. The CBE equivalent of the first record of daemon.log

<cc:Outputterdescription="Single File Outputter"uniqueID="N13725210AFF11DA8000AE8373D52828"



http://www.ibm.com/developerworks/forums/dw_thread.jsp?message=3859826&cat=22&thread=54232&treeDisplayType=threadmode1&forum=307#3859826


type="SingleFileOutputter"><pu:Property propertyName="directory"propertyValue="/home/mstreicher"/>

<pu:Property propertyName="fileName"propertyValue="emitter.log"/>

<op:SingleFileOutputterType directory="/home/mstreicher"fileName="emitter.log"/>

</cc:Outputter>

If the line <op:SingleFileOutputterType... /> is missing, add it, changingthe values of attributes directory and fileName to match the values of thesimilarly names properties. Then save the file.

Section 9. Running the GLA

Your rules-based adapter is now complete. Step through your template log file usingthe controls in the Extractor Result pane and validate its operation. When you'resatisfied that everything is working properly, you can move on to running youradapter using the stand-alone GLA.

Run the adapter using GLA

The GLA uses the settings you created in your adapter to read a log file and producea CBE XML document. Listing 11 shows a small portion of the file my.adapter.

Listing 17. A snippet of the file my.adapater

<adapter:Adapter...<cc:ContextInstance

charset=""continuousOperation="true"description="A context for

daemon.log"isoCountryCode="" isoLanguageCode=""maximumIdleTime="120"pauseInterval="120"timezone="-0700"

uniqueID="N05306B00AFF11DA8000AE8373D52828"year="2006">

<cc:Sensordescription="Read the daemon.log"

uniqueID="N057E8B10AFF11DA8000AE8373D52828"confidenceBufferSize="1024"fileFooterSize="0"maximumBlocking="10"type="SingleFileSensor"><pu:Property

propertyName="directory"propertyValue="/home/mstreicher/java-tptp-gla"

/><pu:Property

propertyName="fileName"propertyValue="daemon.log"

/><sensor:SingleFileSensor




directory="/home/mstreicher/java-tptp-gla"fileName="daemon.log"'

/></cc:Sensor><ex:Extractor

containsLineBreaks="false"description="Divide daemon.log into

individual records"endPattern="$"includeEndPattern="false"includeStartPattern="false"lineBreakSymbol=""replaceLineBreaks="false"startPattern="^"

uniqueID="N05AA7D00AFF11DA8000AE8373D52828"/>

.</adapter:Adapter>

To run the GLA, you must first edit its script to point to where you installed it. Usingyour favorite editor, open the file gla.sh in GenericLogAdapter/bin. (If you followedthe installation instructions verbatim, the file resides in~/java/GenericLogAdapter/bin/gla.sh.) Find the lineGLA_HOME=/home/eclipse/GenericLogAdapter and change the path to pointto the directory that contains your copy of the GLA. Again, if you followed theinstructions verbatim, you would change the line to readGLA_HOME=~/java/GenericLogAdapter. Save the file.

Next, find the file my.adapter in your Eclipse workspace under the directory MyAdapter. On the test system, my.adapter was found in ~/workspace/MyAdapter/my.adapter. To run the adapter, execute gla.sh, providing the path to youradapter file as the only argument:

% ~/java/GenericLogAdapter/bin/gla.sh ~/workspace/My\ Adapter/my.adapter

After a moment, the file emitter.log should appear in your home directory (orwherever you configured your file outputter to create the file).

Section 10. Summary

This tutorial demonstrated how to create a rules-based adapter to convert a typicalLinux log file into a CBE log file. Given a CBE log, you can use a tool such as theAutonomic Computing Toolkit's Log and Trace Analyzer to further process the CBEdata.

Furthermore, if the rules constructs provided by Adapter Configuration Editor aren'tsuitable for your log file, you can integrate your own Java class to parse and emitCBE format. Unlike the rules-based adapter, a static parser (so-named because ituses a Java class instead of rules) only needs a sensor and outputter, both of whichyour Java code provides. You still run the GLA on the final configuration file, but you




must include your Java class in the GLA CLASSPATH, too.

In any case, the Adapter Configuration Editor and the GLA provide a powerfulenvironment in which to analyze the behavior of existing, even legacy applicationsusing modern autonomic computing tools. Simple conversion from any number oflog file formats to CBE requires just a few minutes of work; complex, detailedconversions are easily accomplished with rich rules or your own code.




Resources

Learn

• Read Part 1 and Part 3 of this series by Martin Streicher.

• Learn more about the Eclipse Foundation and its many projects.

• Read "Appendix A. Understanding Common Base Events Specification V1.0.1"of the Autonomic Computing Toolkit Developer's Guide (IBM 2004) forinformation about the Common Base Event format.

• Read a description of the CBE format, produced by the TPTP Log Adapter.

• Read the detailed specification of the Canonical Situation Data Format (PDF).

• Learn how to install the GLA as a stand-alone application with the Stand-aloneGeneric Log Adapter Installation Guide.

• Expand your Eclipse skills by visiting IBM developerWorks' Eclipse projectresources.

• Browse all of the Eclipse content on developerWorks.

• Stay current with developerWorks technical events and webcasts.

• Visit the developerWorks Open source zone for extensive how-to information,tools, and project updates to help you develop with open source technologiesand use them with IBM's products.

Get products and technologies

• Download the entire Eclipse TPTP runtime. Discover all the features of theEclipse TPTP, as well as extensive documentation, tutorials, presentations, andscreencasts that illuminate the capabilities of the Eclipse TPTP.

• Download the Eclipse GLA.

• Read more about the Autonomic Computing Toolkit and download its Log andTrace Analyzer.

• Download Java technology from Sun Microsystems or from IBM.

• Download the freely available, extensible open source Eclipse SDK.

Discuss

• Connect with Eclipse developers and other users in the Eclipse mailing lists andnewsgroups. (You must register to read the newsgroups, but membership isfree, and the registration process is easy.)

• Get involved in the developerWorks community by participating indeveloperWorks blogs.

About the author



http://www.ibm.com/developerworks/edu/os-dw-os-ecl-tptp-i.html

http://www.ibm.com/developerworks/edu/os-dw-os-ecl-tptp3.html

http://eclipse.org/

http://www.ibm.com/developerworks/autonomic/books/fpy0mst.htm#ToC_91

http://www.ibm.com/developerworks/autonomic/books/fpy0mst.htm

http://www.ibm.com/developerworks/autonomic/books/fpy0mst.htm#Header_92

http://www.eclipse.org/tptp/home/downloads/installguide/gla_41/ref/rcbe.html

http://eclipse.org/tptp/home/downloads/installguide/gla_install.html

http://eclipse.org/tptp/home/downloads/installguide/gla_install.html

http://www.ibm.com/developerworks/opensource/top-projects/eclipse.html

http://www.ibm.com/developerworks/opensource/top-projects/eclipse.html

http://www.ibm.com/developerworks/views/opensource/libraryview.jsp?search_by=eclipse

http://www.ibm.com/developerworks/offers/techbriefings/?S_TACT=105AGX03&S_CMP=art

http://www.ibm.com/developerworks/opensource

http://www.eclipse.org/tptp/home/downloads/downloads.php

http://www.eclipse.org/downloads/download.php?file=/tptp/4.1.0.1/TPTP-4.1.0.1/tptp.gla.runtime-TPTP-4.1.0.1.zip

http://www.ibm.com/developerworks/autonomic/overview.html?Open&ca=daw-prod-logandtrace

http://java.sun.com/j2se/downloads/

http://www.ibm.com/developerworks/java/jdk/

http://eclipse.org/downloads/

http://www.eclipse.org/newsgroups/

http://www.eclipse.org/newsgroups/

http://www.ibm.com/developerworks/blogs/


Martin StreicherMartin Streicher is the Editor-in-Chief of Linux Magazine. Martin earned a Master ofScience in Computer Science from Purdue University and has been programmingUNIX-like systems since 1986 in the Pascal, C, Perl, Java, and (most recently) Rubyprogramming languages.




eclipse test & performance tools platform, part 2: monitor ...€¦ · the eclipse test &...

Documents