35833_linux-journal-2011-09
TRANSCRIPT
jQuery | Gauger | Moose | Qt4 Designer | GNU Awk | jEdit
Since 1994: The Original Magazine of the Linux Community
™
PROGRAMMINGDEVELOP GUIswith Qt4 Designerand Eclipse
MULTIPLATFORMDEVELOPMENTUsing GNULibrariesand Tools
USE GAUGERfor PerformanceRegression Testing
WaveMakerfor Rapid
ApplicationDevelopment
ModernDevelopment
with Perland Moose
Make UtilityPrimer
What’s New inGNU Awk 4.0
SEPTEMBER 2011 | ISSUE 209 | www.linuxjournal.com
PLUS:
GETTING STARTED WITH jEDIT
Call iXsystems toll free or visit our website today!1-855-GREP-4-IX | www.iXsystems.comIntel, the Intel logo, Xeon, and Xeon Inside are trademarks or registered trademarks of Intel Corporation in the U.S. and/or other countries.
Pro 2U Appliance:You Are the CloudStorage. Speed. Stability.
With a rock-solid FreeBSD® base, Zettabyte File System (ZFS) support, and a powerful Web GUI, TrueNAS™ Pro pairs easy-to-manage FreeNAS™ software with world-class hardware and support for an unbeatable storage solution. In order to achieve maximum performance, the TrueNAS™ Pro 2U System, equipped with the Intel® Xeon® Processor 5600 Series, supports Fusion-io’s Flash Memory cards and 10 GbE Network Cards. Titan TrueNAS™ Pro 2U Appliances are an excellent storage solution for video streaming, file hosting, virtualization, and more. Paired with optional JBOD expansion units, the TrueNAS™ Pro System offers excellent capacity at an affordable price.
For more information on the TrueNAS™ Pro 2U System, or to request a quote, visit: http://www.iXsystems.com/TrueNAS.
KEY FEATURES: . Supports One or Two Quad-Core or Six- Core, Intel® Xeon® Processor 5600 Series . 12 Hot-Swap Drive Bays - Up to 36TB of Data Storage Capacity* . Periodic Snapshots Feature Allows You to Restore Data from a Previously Generated Snapshot . Remote Replication Allows You to Copy a Snapshot to an Offsite Server, for Maximum Data Security . Up to 4.48TB of Fusion-io Flash Memory . 2 x 1GbE Network interface (Onboard) + Up to 4 Additional 1GbE Ports or Single/ Dual Port 10 GbE Network Cards
JBOD expansion is available on the 2U Pro System
* 2.5” drive options available; please consult with your Account Manager
Expansion Shelves
Available
Create Periodic Snapshot
Clone Snapshot
All Volumes
&✓
✓
✓
✓
EnvironmentallyResponsible:100% Renewable Energy
Solid Technical Foundation:1,000 In-house Developers
Double Security:Your website is simultaneously hosted in 2 locations in our high tech data center!
High-speed Global Network:210 GBit/s Connectivity
No other web host offers more expertise, know-how and quality servicethan 1&1.
®®®
1-855-CA-1AND1 www.1and1.ca
1-877-GO-1AND1 www.1and1.com
WEB HOSTINGTHE NEW STANDARD IN
1&1 DUAL HOSTING
* Offers valid through August 31, 2011. 24 month minimum contract term required for Dual Advanced offer. Set-up fee and other terms and conditions may apply. .com price valid fi rst year only. After fi rst year, standard pricing applies. Visit www.1and1.com for full promotional offer details. Program and pricing specifi cations and availability subject to change without notice. 1&1 and the 1&1 logo are trademarks of 1&1 Internet AG, all other trademarks are the property of their respective owners. © 2011 1&1 Internet, Inc. All rights reserved.
SUMMER SPECIAL: 1&1 DUAL ADVANCED PACKAGE
1 YEAR FREE!*
■ 2 FREE Domains ■ FREE Private Domain Registration■ DNS Management ■ 500 E-mail Accounts ■ 150 GB Web Space■ DNS Management
■ 50 FTP Accounts■ 1&1 SiteAnalytics■ ASP, .NET, AJAX, LINQ, PHP5, Perl, SSI■ 5 Microsoft® SQL Databases■ Mobile Website Optimization Software■ 24/7 Toll-free Customer Support
Need more domains? .com with FREE Private Registration just $4.99/fi rst year.*
OFFER ENDS 08/31/11
US104020100173_US439_206x276_28L.indd 1 05.07.11 17:49
CONTENTS SEPTEMBER 2011Issue 209
4 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
60 Multiplatform GNU DevelopmentMaking a guitar synth work with Rock Band using GNU libraries and tools.
Nathanael Anderson
68 Performance Regression Monitoring with GaugerHow to use Gauger and guidelines for what a suitable development environment for Gauger’s deployment should look like.
Bart Polot and Christian Grothoff
76 mmaann mmaakkee : a Primer on the Make UtilityEver wonder what that Makefile in your project folder does? Here’s a look at the basics of Makefiles and how to manipulate them.
Adrian Hannah
84 Qt4 Designer and EclipseDevelop GUIs quickly and easily.
PJ Radcliffe
FEATURES
PROGRAMMING
LINUX JOURNAL (ISSN 1075-3583) is published monthly by Belltown Media, Inc., 2121 Sage Road, Ste. 310, Houston, TX 77056 USA. Subscription rate is $29.50/year. Subscriptions start with the next issue.
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 5
COLUMNS8 Doc Searls’ From the Editor
Off the Press
32 Reuven M. Lerner’s At the ForgeCoffeeScript and jQuery
40 Dave Taylor’s Work the ShellCalculating Day of the Week, Finally
44 Kyle Rankin’s Hack and /Remotely Wipe a Server
136 Kyle Rankin and Bill Childers’ Tales from the Server RoomUnboxing Day
INDEPTH94 GNU Awk 4.0: Teaching an
Old Bird Some New TricksWhat’s new with version 4.
Arnold Robbins
102 WaveMaker: It’s Like...RAD!WaveMaker—it’s a tsunami of change for rapid application development.
Don Emmack
112 jEdit: a Text Editor and MoreAn intro to this cross-platform text editor.
Adrian Klaver
124 MooseMoose is essentially a language extension for Perl 5 that provides a modern, elegant, fully featured object system.
Henry Van Styn
IN EVERY ISSUE10 Current_Issue.tar.gz14 Letters18 UPFRONT50 New Products54 New Projects139 Advertisers Index141 Marketplace
ON THE COVER• Make Utility Primer, p. 76• What's New in GNU Awk 4, p. 94• WaveMaker for Rapid Application Development, p. 102• Modern Development with Perl and Moose, p. 124• Develop GUIs with Qt4 Designer and Eclipse, p. 84• Multiplatform Development Using GNU Libraries and Tools, p. 60• Use Gauger for Performance Regression Testing, p. 68• Getting Started with jEdit, p. 112
54 GIADA
112 jEDIT
Executive Editor
Senior Editor
Associate Editor
Art Director
Products Editor
Editor Emeritus
Technical Editor
Senior Columnist
Security Editor
Hack Editor
Virtual Editor
Proofreader
Publisher
General Manager
Advertising Sales Representative
Associate Publisher
Webmistress
Accountant
Jill Franklin [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
Geri Gale
Carlie [email protected]
Rebecca [email protected]
Joseph [email protected]
Mark [email protected]
Katherine [email protected]
Candy [email protected]
Contributing EditorsIbrahim Haddad • Robert Love • Zack Brown • Dave Phillips • Marco Fioretti • Ludovic Marcotte
Paul Barry • Paul McKenney • Dave Taylor • Dirk Elmendorf • Justin Ryan
Linux Journal is published by, and is a registered trade name of,Belltown Media, Inc.
PO Box 980985, Houston, TX 77098 USA
Editorial Advisory PanelBrad Abram Baillio • Nick Baronian • Hari Boukis • Steve Case
Kalyana Krishna Chadalavada • Brian Conner • Caleb S. Cullen • Keir DavisMichael Eager • Nick Faltys • Dennis Franklin Frey • Alicia GibbVictor Gregorio • Philip Jacob • Jay Kruizenga • David A. Lane
Steve Marquez • Dave McAllister • Carson McDonald • Craig OdaJeffrey D. Parent • Charnell Pugsley • Thomas Quinlan • Mike RobertsKristin Shoemaker • Chris D. Stark • Patrick Swartz • James Walker
AdvertisingE-MAIL: [email protected]
URL: www.linuxjournal.com/advertisingPHONE: +1 713-344-1956 ext. 2
SubscriptionsE-MAIL: [email protected]
URL: www.linuxjournal.com/subscribePHONE: +1 818-487-2089FAX: +1 818-487-4550
TOLL-FREE: 1-888-66-LINUXMAIL: PO Box 16476, North Hollywood, CA 91615-9911 USA
LINUX is a registered trademark of Linus Torvalds.
They say work smarter, not harder. They must be using our processor.
The next generation of intelligent server processors The Intel® Xeon® processor 5600 series automatically regulates power consumption to combine industry-leading
to your workload. Check out the new intelligent features of the Xeon® 5600 at intel.com/itcenter.
© 2010, Intel Corporation. All rights reserved. Intel, the Intel logo, Intel Sponsors of Tomorrow, Intel Xeon, and Xeon Inside are trademarks of Intel Corporation in the U.S. and other countries.*Other names and brands may be claimed as the property of others.
Intel is not responsible for and has not veri� ed any statements or computer system product-speci� c claims contained herein.
Enterprise Servers for Open Source www.iXsystems.com1-855-GREP-4-IX
iX2216-10G® Xeon® 5600 Series Processors
®
iX1204-10G® Xeon® 5600 Series Processors
®
iX2216-10G®
®
iX1204-10G ®
Servers from iXsystems feature the Intel® Xeon® processor 5600 series.
92936887_IIMPCMLJ070111_IXSystems_LinuxJ.indd 1 5/4/11 11:51 AM
DOC SEARLS
Why all-digital is more liberating than some-digital and some-print.
E ver since I discovered HTML, it’sbeen my preferred format forwriting. Every word of mine that’s
gone into Linux Journal, since I startedin 1996, has been written and deliveredin HTML. That’s because my writing hasbeen normalized to hypertext, and topixels rather than print.What’s different for me this time is
that I’m not paying attention to mymonthly 900-word limit (or less ifimages are involved). While a word limitdoes impose the discipline of brevity,the fact remains that brevity is not theonly virtue of good writing. Yes, it’s agood one to have when your columnappears on the last page of a printmagazine. But when that magazine isno longer confined by the dimensionsof printed pages, you’re free to golonger—or shorter, as the case may be.My case this month is for the all-
digital version of Linux Journal. Yes,we lose a lot, but we stand to gainmuch more. Let me explain.We’ve fought to stay in print ever
since the dot-com crash nearly killed us,11 years ago. Before that crash, we werefat with ads from well-funded startups.When the bust hit, many advertisers
vanished without a trace, owing us literally $millions we never collected.After that crash, getting and keepingadvertisers for a print trade publicationwas much harder. The costs of printingand mailing also went up, and continuedto go up. Meanwhile, Linux succeededin the marketplace and is now the mostwidely used operating system.Yet, while Linux continues to spread,
the population of pure-Linux geeks—thekind who subscribe to Linux Journal—hasremained a core that has grown verylittle. We continue to serve that core.That’s our mission, and we’re sticking toit. The question is, what’s the best way?Today, it’s hard to say print is that bestway, especially with more and more peo-ple spending more and more time read-ing glowing rectangles rather than paper.But, we are by nature and practice a
print magazine, and we have done ourbest to remain one, even as the worldhas changed around us. So I want tocongratulate the publishing side of ourhouse for keeping our print operationgoing, against stupendous odds, and fornever selling out. (And believe me, therewere many offers, mostly from entitiesthat are now gone.) Our team did the
Off the Press
From the Editor
8 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
impossible for as long as it could.Yet, consider this. We also always
have been a digital publication, startingwith the first CD digest of issues in1994. And, digital publishing has donenothing but grow from the beginning.So has advertising in the digital realm,which is inherently limitless.Something else also has started to
happen in digital publishing. It hasbecome easier, and more acceptable,for people to pay for goods that also areavailable for free. There has been muchexperimentation here, and we are amongthe many doing the experimenting. Oneadvantage for us is that we’ve alwayshad paying subscribers. Maybe it’s crazyto think they’ll stick with us after we goall-digital. But, I don’t think so. I’m a bigbeliever in the willingness of people topay for value, provided the means arethere. We have some means today, andwe will have better ones tomorrow,especially if you help us think thosethrough—while also helping us improveour editorial methods and materials.Every magazine has a periodical
heartbeat. Ours always has been monthly.That won’t change. What will change ishow much time passes between whatwe write and when it appears. A pro-duction cycle that took several monthswill now take just weeks. (So for thisissue, I am writing this on August 1stfor a September publication date.) Much
more of our stuff will be current, or asclose to now as we can get.We always will remain a print publica-
tion at heart (and in that respect, we willbe no different from the rest of journal-ism), but we won’t remain contained bythe print medium. That medium, wherenearly all of our contributors grew up,has legacy values (fairness, transparency,credit to sources and so on) that areimportant to bring to a vast new worldthat has too little of them. Again, weexpect you to help us with that.
Linux Journal always has been a publi-cation for the Linux Community. LinuxJournal will now be a publication by theLinux Community as well. This is a verygood thing. Here in the digital world,connection between people and ideasare much more direct and immediate.Understandings are also easily iteratedand improved. Just like code.Maybe Linux doesn’t need Linux
Journal—or anything, other than con-tinued constructive hacking in kernelspace. But I do believe Linux has beenbetter with Linux Journal around than itwould have been without it. Therefore,with Linux Journal in a much moreimprove-able place, we can’t help butmake Linux better in the process.�
Doc Searls is Senior Editor of Linux Journal. He is also a fellowwith the Berkman Center for Internet and Society at HarvardUniversity and the Center for Information Technology and Societyat UC Santa Barbara.
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 9
FROM THE EDITOR
SHAWN POWERS
I f I’ve learned nothing else fromAmerican politics, it’s that it doesn’ttake knowledge or insight on a
topic to have lots to say on the matter.Thankfully, although this issue’sProgramming focus isn’t even close tomy area of expertise, our authors don’thave that shortcoming. The worst youshould have to put up with is me tryingto explain what this issue contains. Feelfree to point and laugh.Kyle Rankin, a fellow sysadmin,
works through an interesting conun-drum this month. You’re all familiarwith programs like DBAN for wipingsensitive data, but what if you needto delete information securely on aserver thousands of miles away? (Or,in the next room if you’re lazy like me.)Kyle shows how to go about takingcare of a seemingly difficult chicken/eggscenario. Kyle also shares a “Tale fromthe Server Room” with Bil l Childersand talks about the joy of UPS delivery—more specifically, when servers areunboxed, sometimes things don’t goquite as planned.If you’re beginning to worry our
Programming issue doesn’t containarticles about programming, fear not.Yes, we try to include a little somethingfor everyone, but this issue focuses onprogramming, and we’ve got tons ofuseful stuff for you. Nathanael Andersonstarts out with an appealing way tolearn multiplatform GNU development:getting a guitar synth to work withRock Band 3. Unfortunately, there’s noprogramming that can make me anybetter at Rock Band, but using a realguitar is a step in the right direction!My friend Adrian Hannah is back
this month with a primer on the Makeutility. For most users, prepackagedapplications are how programs areinstalled. For programmers, or peopleon the bleeding edge, it’s necessary tocompile programs themselves. Adrianshows how to “make” programs fromtheir source code. Sometimes whenyou are on the bleeding edge, you’llnotice that a newer version of anapplication isn’t always better thanthe previous version. Programmersneed to be aware of such things, andBart Polot and Christian Grothoff show
My Language: Dork++
Current_Issue.tar.gz
10 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
CURRENT_ISSUE.TAR.GZ
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 11
us Gauger, a tool that monitors per-formance regression. Sometimes anapplication is slower because it hasmore features, but sometimes it’s justslower because of an erroneous sourcecode change. Gauger helps determinewhen new versions go bad.When I took programming in college,
I started out learning to programcommand-line utilities that did littlemore than solve the problem presentedin the curriculum. If programmingwas a little more interesting back then,I might have stuck with it for longerthan the single semester it was required.My problem was that I wanted to makeGUI programs. PJ Radcliffe shows howto develop GUI interfaces with Qt4Designer and Eclipse. Granted, a fancyGUI controll ing a cruddy underlyingprogram isn’t very useful, but at leastfor me, a cool-looking program issomething I’d l ike to spend time per-fecting. PJ shows how easy it can beto include GUI controls.If GUI programs aren’t for you, that’s
fine too. Adrian Klaver explores jEdit,which is a very powerful and cross-platform text editor. jEdit has featuresthat make programming much easier,and its cross-platform nature meansyou can use a consistent interfaceregardless of the computer you’re stuckusing. Arnold Robbins is a fan of textas well, and he presents GNU Awkversion 4. Awk has been around forever,and although it’s still as useful as it’s
ever been, version 4 offers a few newtricks as well.Of course, we have our regular
columnists teaching about programmingthis month as well. Reuven M. Lernerdiscusses CoffeeScript, a different way to program JavaScript. Dave Taylorfinishes his series on determining theday of the week in a script. Plus, wehave many other programming-relatedarticles as well! Henry Van Styn describeshow to write object-oriented code inPerl, Donald Emmack teaches how touse WaveMaker, and we’ve even includedthe results of a LinuxJournal.com programming survey so you can seewhat your fellow Linux programmersare up to.If you’re a programmer, this issue
likely will be one of your favorites ofthe year. If you’re not a programmer,there still are exciting things to read,and you might find that programmingis more interesting than you originallythought. I know I learned a lot thismonth, and I might have to dust offmy old C++ course book and figureout how to make a GUI version of“Hello World”. Either way, we hopeyou enjoy reading this issue as muchas we enjoyed putting it together.�
Shawn Powers is the Associate Editor for Linux Journal. He’salso the Gadget Guy for LinuxJournal.com, and he has aninteresting collection of vintage Garfield coffee mugs. Don’t lethis silly hairdo fool you, he’s a pretty ordinary guy and can bereached via e-mail at [email protected]. Or, swing bythe #linuxjournal IRC channel on Freenode.net.
Oct. 12, 13, & 14
for BUILDING SUCCESSFUL WEBSITES
Oct. 112, 13, & 14
for BUILDING SUCCESSFUL WEBBBBBBBBBBBBBBBBBBBBBSSSSSSSSSSSSSSSSSSSSSSSSSSSITES
doitwithdrupal.com
Attending Do It With Drupal was absolutely worth it. I
learned more about Drupal in a few days than I would
have in months on my own. Meeting so many talented
and passionate web professionals was exhilarating.
Scott Phillips, Web Developer, Drake University
Jeffrey ZELDMAN
Josh CLARK
Karen MCGRANE
Featuring Awesome Speakers!
Also Jeff Robbins, Angela Byron, Karen Stevenson, Jeff Eaton, Ryan Szrama & more!
Do it with Drupal sessions are designed for you, covering the latest
Drupal information and best practices. But you’ll learn about more
than just Drupal -- choose from sessions on UX, designing mobile
apps, web typography, and more. And the speakers who present are
not just knowledgeable, they’re fun and engaging. Do it with Drupal
provides practical information that you can start using right away!
Use coupon code LJ50 when you register & save $50!
* coupon code expires October 10th
powered
AussieAdmirerI had a cou-ple hours tokill at theshops while Iwas waitingfor new tyreson my carand foundan April edition of
your mag for $13.95 AUD (today’s dateis June 6). The people at the news agencystated it cost that much because theyhad to get it from America, and it wastwo months old because it would costeven more to send via air freight! Aquick check of exchange rates showsthe Aussie dollar paying $1.07 USD,making the price in USD close enoughto $14.95. Would you sell many copiesfor this price, I wonder? It’s a cryingshame that Australian retailers chargethis absolutely stupid markup. The rea-soning I was given is utter garbage—ahundred issues and I could fly them ona seat first-class with free champagneand still show a profit!Enough with the whining. I purchased
the mag anyway and had a very inter-esting read while I waited. Complimentsto your team. This won’t be my lastpurchase; however, I likely won’t be
purchasing retail again any time soon.Word of mouth spread of Linux in
this country can get only so far, and thelack of reasonably priced journals onthe shelves of our shops sparking theinterests of new users is quite an imped-iment to our plans of world domination.—Scott K.
Thank you for picking up an issue,even if the price was painfully high!The cost of international shipping andprinting is one of the reasons we switched(starting with this issue!) to a digital-only format. For the same subscriptionprice as anyone here in the US, you getthe same experience, regardless of yourlocation. We think this recent change willreally level the playing field for internationalsubscribers. Hopefully you agree!—Ed.
Magazine LocationsI’m sitting in a hospital room watchingmy father recover from liver cancersurgery and a fall on the way home.I recently discovered that my subscrip-tion expired, and I’m going to renewit next week. In the meantime, Ithought I’d pick up a copy locally, andthis is why I’m contacting you. I’d likeyou to consider an application foryour Web site that would displaynearby magazine/bookstore locationsthat carry your fine magazine. Perhaps
14 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
letters
an Android app would be nice too.Yes, I can read the on-line version, butI’d really prefer a hard copy. PerhapsShawn Powers could assign this l ittleGoogle smashup to someone? Thankyou for your consideration.—Michael Soibelman
As someone who lives in an areawith no local retailers stocking LinuxJournal, I feel your pain. I’m not surehow to create an app like that myself,but hopefully, your letter will sparksomeone’s interest in doing such athing. As far as assigning it goes, Ialways could pick you if you like. Hope
your dad is doing well.—Ed.
Google MapsI enjoyed the mapping article by MikeDiehl in the April 2011 issue (“FindYourself with the Google Maps API”).Like Mike, I would not be without GoogleMaps. You may not be interested in thecontent (unless you are a train buff), buttake a look at these sites. I think theyare truly awesome and are done by an“amateur” at that. It just goes to showwhat skills are out there, and it makesme very envious: traintimes.org.uk/mapand traintimes.org.uk/map/tube.—Roy Read
[ LETTERS ]
What Day Is It?I’m sending a little feedback to DaveTaylor’s “parsing the cal” output (seeDave’s column in the June–September2011 issues). There’s no need to useregular expressions in the awk script atall, because you can compare numbersdirectly. Below, you’ll find the script youcan call by the following command line:
$ cal | awk -f day.awk 25
# day.awk
BEGIN{
ARGC=1;
getline;getline;
for(i=1;i<=NF;i++) wd[i]=$i
}
{
for(i=1;i<=NF;i++)
if($i==ARGV[1])
print wd[i+($0~/^ /?(7-NF):0)]
}
In the BEGIN block, ARGC=1 preventsit from taking the last argument (25 inthis case) as an input file. Then, thescript fetches the first two lines andstores the weekday names in an array.The rest of the script compares the
argument number with every field inevery line. On a match, the day nameis output, wd[i]. The month does notalways start on a Sunday, so the scripthas to fix the index for lines startingwith a space (condition $0~/^ /). Forthose lines, the first item starts with
index 7-NF. Note that this fix also worksfine for the second line of numbers(which also starts with a space), since7-7 equals 0.You can make the script a one-liner if
you like. It was written in multiple lines forreadability reasons. And, last but not least:great OS, great magazine, keep going.—Eric Miller
Dave Taylor replies: Thanks for yournote. I realized that there was a way tobreak down the input and process itwith a multiline awk script (just as Icould do much of the task more easilyin Perl or, for that matter, a short Cprogram), but my goal with the Workthe Shell column is to force myself tostick with standard Linux shell com-mands and capabilit ies as much aspossible and see what I can accomplish.Sometimes the result is a bit, um,Byzantine and unquestionably ineffi-cient, but the upside is that it’s alwaysinteresting and, I hope, informativeand entertaining reading.
Installfest 2001Installfest goal: adapt seven older desk-top computers for use by fourth-gradeteacher Mike Steins at Shenandoah St.Elementary School and learn how toinstall/configure Linux. (We had twoLinux experts in the group.)Outcome: we got four computers
working with Linux by scavenging partsfrom various other machines.
16 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
[ LETTERS ]
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 17
Although the Linux installfest event ten yearsago may not have been hugely productive (eightpeople @ six hours = four working Linux computers),it got the ball rolling to get computers at theschool. Since then, every teacher has received alaptop, projector and document camera. Theschool has multiple interactive whiteboards, a fullyfunctioning computer lab (actually two—one ismade up of aging computers), digital microscopes,cameras and video cameras, a completely wirelessnetwork with networked printing and storagecapabilities, an in-house server, student e-mailaccounts, and we’re slowly looking to integratetablet devices during the next few years as thetechnology becomes more inexpensive and fundinglevels rise (if that ever happens). So, Linux (I believeit was the Red Hat distribution) laid the red carpetfor technology at our school. Thanks.—Mike Steins
At Your ServiceSUBSCRIPTIONS: Beginning with theSeptember 2011 issue, subscriptions toLinux Journal will be fulfilled digitally andwill be available in a variety of digital formats, including PDF, an on-line digitaledition, and apps for iOS and Androiddevices will be coming soon. Renewing yoursubscription, changing your e-mail addressfor issue delivery, paying your invoice,viewing your account details or other subscription inquiries can be done instantlyon-line: www.linuxjournal.com/subs.Alternatively, within the US and Canada,you may call us toll-free at 1-888-66-LINUX(54689), or internationally at +1-818-487-2089.E-mail us at [email protected] or reachus via postal mail at Linux Journal, PO Box16476, North Hollywood, CA 91615-9911 USA.Please remember to include your completename and address when contacting us.
LETTERS TO THE EDITOR: We welcomeyour letters and encourage you to submitthem at www.linuxjournal.com/contact ormail them to Linux Journal, PO Box 980985,Houston, TX 77098 USA. Letters may beedited for space and clarity.
WRITING FOR US: We always are looking forcontributed articles, tutorials and real-worldstories for the magazine. An author’s guide,a list of topics and due dates can be foundon-line: www.linuxjournal.com/author.
FREE e-NEWSLETTERS: Linux Journaleditors publish newsletters on both aweekly and monthly basis. Receive late-breaking news, technical tips and tricks,an inside look at upcoming issues andlinks to in-depth stories featured onwww.linuxjournal.com. Subscribe for freetoday: www.linuxjournal.com/enewsletters.
ADVERTISING: Linux Journal is a greatresource for readers and advertisers alike.Request a media kit, view our current editorial calendar and advertising due dates,or learn more about other advertising andmarketing opportunities by visiting us on-line: www.linuxjournal.com/advertising.Contact us directly for further information:[email protected] or +1 713-344-1956ext. 2.
Linux installfest at 419 N Vista St., Los Angeles (circa 2001).
Left to right, top row: Danny Olster, Abhijeet Chavan, Christian
Peralta Madera, Mike Steins, Charanjeet Singh, Christian’s Dad.
Bottom row: Chun Wong, Chris Steins.
WRITE LJ A LETTER We love hearing from our readers. Please sendus your comments and feedback via www.linuxjournal.com/contact.
18 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
UPFRONT N E W S + F U N
Google’s Mike Waychison has postedsome patches to organize Google-specificfirmware used in its servers. Thesepatches were met with approval byAlan Cox and Greg Kroah-Hartman,although Google’s servers remainunavailable for purchase by the public.Perhaps that will change soon.
Chris Wright officially is no longeron the stable kernel team, and he hasnot been involved in that project forsome time. Greg Kroah-Hartman willcontinue his effort to release severalshort-term stable versions of each kernel,during the intervals between officialreleases from Linus Torvalds. Long-term stable trees in the 2.6 series aremaintained by Andi Kleen, WillyTarreau, Paul Gortmaker and others.The ancient versioning system, CVS,
still is being used by some kernel devel-opers in spite of the advent of git, andSebastian Andrzej Siewior recentlyposted a patch supporting those devel-opers. His patch took out all the staleCVS $Id$ tags sprinkled throughoutthe kernel that were confusing CVS.Apparently, some developers are syncingthe kernel sources from git and thenfeeding the whole tree into a CVS
repository before working on it—bizarre.But, it’s a great testament to the hardi-ness of CVS after so many years and somany attempts to find something better.A recent bug in SysFS allowed regular
users to overwrite NVRAM. VasiliyKulikov’s patch to close the securityhole had taken more than a month tobe incorporated into the kernel. In lightof that, he’s posted a patch to give thesystem administrator the power tomount the SysFS directory as read-only.This blanket protection would not bein effect at all times, but it would giveadministrators the ability to lock downthat part of the system in the event thata similar bug were discovered. The prob-lem, as Greg Kroah-Hartman points out,is that locking down SysFS may produceother unanticipated problems, and hefeels the right approach simply would beto fix SysFS bugs as they occur, ratherthan add a blanket layer of securityover it. The debate is unresolved at themoment and could play out either way.
Huang Ying recently posted somecode to cause unknown non-maskinginterrupts (NMIs) to crash the kerneland produce a panic report. But, partof the problem is that a wide array of
diff -uWHAT’S NEW IN KERNEL DEVELOPMENT
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 19
[ UPFRONT ]
systems produce unknown NMIs for noreason at all. Huang’s solution was tocreate a whitelist of systems that wereknown not to do this, and his patchwould work only on that whitelisted setof systems. But, Ingo Molnar suggestedusing active event filters to allowunknown NMIs to go through a localizedpolicy decision-making process first, sothe decision to panic the system couldbe made on a per-system basis.Active event filters, as Ingo pointed
out, would allow a certain portion ofthe decision-making process to occurwhile sti l l in kernel space, withouthaving to return to userland. This is key,
because when the system is crashing, itoften is not feasible to pass control overto a user-space program. But in thecase where the active event filtersdetermined that a crash probably wasnot occurring, they could hand controlto a user-space dæmon that wouldmake additional decisions about howto handle the unknown NMI.Active event filters apparently are
tremendously powerful and soon maybe seen in use throughout the kernelas a way of standardizing a number ofdisparate behaviors that currently arehandled in an ad hoc manner.—ZACK BROWN
Digital music and, more recently, digitalvideo and digital books, have changed theway we consume media. Comic books areno different, and with the advent of tabletcomputers, digital comics are becomingmore and more popular. If you don’t havea tablet computer, however, viewing CBR(or their compressed version, CBZ) files isas simple as installing a CBR reader anddownloading your favorite comic.Many comic book readers are available
for Linux. A quick Google search will turnup programs like Comix, ComicMasterand Comical, all of which display digitalcomics quite well. Another search likely
will turn up some free comic resources,like the one shown here: Cory Doctorow’sFuturistic Takes of the Here and Now. Ifyou miss the comic books of your youth,or if you still enjoy them on a regularbasis, you owe it to yourself to check outCBR/CBZ files.—SHAWN POWERS
CBZ, the MP3of Comics
20 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
[ UPFRONT ]
Sometimes, you just want a simple word processor. Yes, thereare many options for word processing, from the awesomeOpenOffice.org to the awesome-for-other-reasons vi. If you’relooking for a happy medium, however, it’s hard to do betterthan AbiWord. When you add the free on-line component,
AbiCollab.net, you even can collaborate with other AbiWord users on documents.AbiWord is available for Linux, Windows and even OS X. You need version 2.8
or higher to use AbiCollab.net, but most distributions include at least version2.8. Check out the Web site at www.abisource.com.—SHAWN POWERS
Non-Linux FOSS
No, I’m not talking browncoats andspaceships. Unfortunately, that ship has sailed. If you’re the musical type,however, installing a Firefly MediaServer is fairly simple. It was renamedfrom mt-daapd, so your distribution stillmight call it that. After a quick install,visit the Web configuration, usually athttp://localhost:3689 with the defaultlogin mt-daapd and password mt-daapd.You can configure your music location,smart playlists and just about everyother aspect of the media server. Thencomes the cool part.On any software or hardware on
your network that supports daap (oftenknown as the iTunes protocol), youshould be able to play your musicremotely. Firefly does a decent job ofscanning your music collection andupdating the clients on the network.
You can add m3u playlists, and Fireflywill serve up playlists as well.I find the best way to listen to music on
XBMC is via daap. It makes configuringplaylists and adding media simple. It’s alsocross-platform, so those folks using actualiTunes can listen to their tunes as well.—SHAWN POWERS
ROCK YOUR WORLD WITH FIREFLY
“The question of whether a computercan think is no more interestingthan the question of whether a submarine can swim.”—E. W. Dijkstra
“One thing a computer can do thatmost humans can’t is be sealed upin a cardboard box and sit in a warehouse.”—Jack Handey (from “Deep Thoughts”,
Saturday Night Live)
“If you have any trouble soundingcondescending, find a UNIX user toshow you how it’s done.”—Scott Adams
“Isn’t it interesting that the same peoplewho laugh at science fiction listen toweather forecasts and economists?”—Kelvin Throop III
“Computer Science is no more aboutcomputers than astronomy is abouttelescopes.”—E. W. Dijkstra
“Even he, to whom most things thatmost people would think were prettysmart were pretty dumb, thought itwas pretty smart.”—Douglas Adams
“It is bad luck to be superstitious.”—Andrew W. Mathis
“The problem with quotes on theInternet is that it is often difficult toverify their authenticity.”—Abraham Lincoln
They Said It
22 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
[ UPFRONT ]
Did you learn all yourLinux console skillsfrom books or fromforums? Or, did youpeek over someone’sshoulder to see thereal action? Once in
a while, we stumble upon new projectsthat deserve some attention, l ikePlayterm (www.playterm.org).What’s the reason for this command-line “peep show”? To spread GNULinux command-line knowledge.You will see a fair amount of on-line
terminal recordings when you enter thissite. The recordings cover several topicsperformed in the shell: tricks, one-liners,guided tutorials and handy utilities.Personally, I found them quite enter-
taining to watch, and it brought meback to the BBS days. It can be educa-tional, and also quite hilarious to seepeople making typos and mistakes.Another interesting Playterm feature
is the embed facility. You can uploadterminal recordings on this site, whichyou then can embed and play on yourblog or Web site. Optionally, you canallow commenting on your recordings,which, of course, will provide interestinghints and tips and other improvements.The Coder of Salvation (Leon van
Kammen) created Playterm because hewas just too curious about what people
were doing in theirterminals. He usedto work for a com-pany where he didextreme program-ming sessions withhis colleaguesthrough the GNU screen -x utility. Inhis experience, it is extremely educa-tional when you work together in oneterminal (and also entertaining). In hisopinion, console-related books andarticles are great, but sometimes it canbe more helpful to see gurus at work.If it were up to him, more command-line projects should feature a terminalplayer on their sites: “Why not? Whyhave only a tar archive on a site?Developers should make more demosto show the world how cool theirutilities are! It hurts me to see so manygreat utilities being overseen by themasses.” Obviously, these are thewords of a true terminal evangelist.Before the big Internet boom, people
used BBSes a lot (en.wikipedia.org/wiki/Bulletin_board_system). Peoplecalled to other people’s BBSes viatheir phone line. The cool thing aboutrunning your own BBS was that youcould create an console “intervention”.By doing this, you could “take over”the terminal session of a given user.In those days, a lot of teaching and
Playterm, Platform of the Gurus
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 23
[ UPFRONT ]
cooperation was done this way.Of course, the Playterm Web site
would not be possible without the GNUand Open Source movement. Thanks alsoto the developers of ttyrec (0xcc.net/ttyrec) and jsttyplay (encryptio.com).Playterm is still beta, but it’s already
fully functional. Currently, we are curi-
ous about how many users we canserve, but in terms of global Linuxknowledge, we are very excited. Atthis point, Playterm.org is a nonprofitproject to serve the community andspread GNU Linux knowledge. Hopefully,it will inspire youngsters to use theshell more often.—LEON VAN KAMMEN
I may not be Steven Spielberg,but every time I see a rerun ofGumby, I’m convinced I couldbe a famous producer. WithLinux, I don’t even have to geta fancy movie set. I can makemy own science-fiction adven-ture film with nothing morethan a Webcam and a streakof bizarre creativity.Stopmotion is a Linux
program designed for creatingstop-motion films. It’s availablefor most distributions andeasily compilable for the rest.Stopmotion is simple in its design, andit allows you either to import a seriesof pre-taken photos or take live stop-action with a Webcam. I find the latterto be slightly easier, as you can see aghost image of the last shot you took,making the slight changes you needeasy to spot.
Recording stop-motion fi lms istedious work, but the end result canbe pretty cool. Check out the homepage,short-linked here: is.gd/stopmotion.If you create an interesting video, senda link to [email protected]. Ifwe get enough, we’ll post them on ourWeb site.—SHAWN POWERS
Roll Your Own Cthulhu Flick
24 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
[ UPFRONT ]
Engineers are some of the heaviestnumber-crunchers around. If you are agrad student, post doc or undergrad, youusually get whatever is lying around asyour work machine. Also, depending onhow inflexible your local IT departmentis, you may be forced to use one of thecommercial operating systems around thesedays. What are lowly students to do whenthey need to do heavy computationalwork? You may be interested in looking atCAELinux (Computer Assisted Engineering,www.caelinux.com). This project providesa live CD that gives you all the open-sourcetools you might need for your engineeringwork. And, because it is a live CD, you canuse it without touching the local drive ofthe machine you are using.Like all live CDs, it has all the standard
Linux desktop tools you should be familiarwith, including Firefox for Web browsing,Evolution for e-mail, and OpenOffice.orgfor word processing, spreadsheets andpresentations. Along with these applications,there are dozens of others to help with allyour number-crunching work. The mostrecent versions are based on Ubuntu, so itshould be a fairly comfortable environmentfor most people. Be aware, however, thatyou can’t use the usual software updatemechanism in Ubuntu. Many of thepackages in CAELinux are compiled fromsource and optimized, so you don’t wantthem being overwritten accidentally by anypackages provided by Ubuntu.
A really well written introduction toCAELinux is available right on thedesktop, called “Getting Started”. Youshould start here if this is your firststep into the world of CAELinux.Last month, I looked at OpenFOAM
in this space. CAELinux includes a fullinstall of OpenFOAM. It also includesanother fluid dynamics program calledSALOME. This program provides a fullgraphical interface that takes you fromforming your problem, to modeling, tocalculation and through to analyzingyour results. This might be a goodchoice for those who are more comfort-able with a GUI. A series of examples onthe desktop are available that provide awalk-through of the program, showingeach of the steps as you go through.As you can see, the tutorials walk
through several common simulations,l ike modeling flow through a pipe.
Computer-Aided Engineering in Linux
Welcome to CAELinux
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 25
[ UPFRONT ]
These can provide great starting pointsfor many people.If your work leans more toward data
analysis, several popular packages are avail-able. For all of you Matlab addicts, there isScilab. Scilab provides the same types offunctions in an environment familiar toMatlab users. There also is Maxima, whichprovides tools more from a mathematicalbackground (for example, analyzing func-tions and doing calculus), as compared toScilab’s approach of working from a matrixbackground (such as looking at data analy-sis). Maxima has several front ends available.The default one in CAELinux is wxMaxima.If you are doing really heavy statisticalanalysis, there is R. The real power of R isthe CRAN repository, and a fair amount isavailable out of the box. R also has severalgraphical front ends. CAELinux providestwo: R Commander and RKWard. If you aredoing work more along the lines of puremathematical analysis, there also is Octave.The default GUI available within CAELinux
is QtOctave. In all of these cases, text-based interfaces also are available, if youare an old-style computer user who prefers
Getting Started CAELinux Tutorials
26 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
[ UPFRONT ]
that kind of thing.Several software packages exist for
applications other than CFD or statistics.If you need to do finite element analysis,there’s elmer. It provides both a text-basedand GUI interface. There also is JavaFoil,available for doing analysis on airfoils andwings. If you are designing electricalcircuits, two packages are available. Electricis a CAD program that helps you lay out acircuit. And, once it is all laid out, you canuse PCB Designer to get it set up so youcan etch a board to make it real.This is all fine and good if you can use
a standard toolset in your work. But, whatif you need computing power for reallycutting-edge research? CAELinux providesthe entire GNU toolset. This means youhave everything you need to go aheadand start developing your own code. Allof the most common scientific and engi-neering libraries, like gsl and LAPACK, areavailable. If you are working on reallylarge problems, MPI and openMP also areavailable. This way, you can develop aparallel programming solution if that iswhat your problem needs.Once you have finished all your calcula-
tions, an important part of data analysisis graphical analysis. There is somethingvisceral and instinctive about actuallyseeing your data represented. To this end,CAELinux provides several packages. Ifyou simply want to plot your data, youcan use programs like grace and LabPlot.If you want to do more complicated dataanalysis, you have programs like G3Data
and OpenDX Data Explorer. These programshave lots of functionality that can beused to look at your data graphically.If you are doing CFD work, severalprograms for visualizing your meshesare available. So, you have your choicebased on what features you need.The last option to look at this month
is using CAELinux in “the cloud”. Cloudcomputing is one of those sexy terms thatgets used a lot in marketing, but it some-times doesn’t really give you anythinguseful. In this case, there really is somethingsubstantial being offered. CAELinuxnow can be run as an application underAmazon Elastic Cloud Computing. You cannow run, on demand, as many nodes asyou like, each having eight cores and 64GBof RAM. For people who don’t have theresources to run their own clusters, butneed more than what a desktop can handle,this can be a very attractive choice. Itdefinitely is worth looking into as a possibleoption. You can find more informationabout EC2 at aws.amazon.com/ec2, andthe CAELinux Web site has a very good setof instructions to get you up and running.As you can see, CAELinux provides a
lot of power and functionality for doingcomputational science. Because it is alive CD, you can run it on essentially any64-bit machine without touching the harddrive. But, you also have the option ofinstalling it on the machine if you areallowed. Download an ISO and startplaying with it to see just how muchwork you can do with it.—JOEY BERNARD
More TFLOPS, Fewer WATTS
GSA ScheduleContract Number:GS-35F-0431N
Configure your next Cluster today!www.microway.com/quickquote508-746-7341
FasTree™ QDR InfiniBand Switches and HCAs
36 Port, 40 Gb/s, Low Cost Fabrics
Compact, Scalable, Modular Architecture
Ideal for Building Expandable Clusters and Fabrics
MPI Link-Checker™ and InfiniScope™ Network Diagnostics
Enhanced GPU Computing with Tesla Fermi
480 Core NVIDIA® Tesla™ Fermi GPUs deliver 1.2 TFLOP single precision & 600 GFLOP double precision performance!
New Tesla C2050 adds 3GB ECC protected memory
New Tesla C2070 adds 6GB ECC protected memory
Tesla Pre-Configured Clusters with S2070 4 GPU servers
WhisperStation - PSC with up to 4 Fermi GPUs
OctoPuter™ with up to 8 Fermi GPUs and 144GB memory
New Processors
12 Core AMD Opterons with quad channel DDR3 memory
8 Core Intel Xeons with quad channel DDR3 memory
Superior bandwidth with faster, wider CPU memory busses
Increased efficiency for memory-bound floating point algorithms
Microway delivers the fastest and greenest floating point throughput in history
Achieve the Optimal Fabric Design for your Specific MPI Application with ProSim™ Fabric Simulator Now you can observe the real time communication coherency of your algorithms. Use this information to evaluate whether your codes have the potential to suffer from congestion. Feeding observed data into our IB fabric queuing-theory simulator lets you examine latency and bi-sectional bandwidth tradeoffs in fabric topologies.
2.5 TFLOPS
5 TFLOPS 10 TFLOPS
45 TFLOPS
FasTree 864 GB/sec Bi-sectional Bandwidth
pC2_Microway.indd 1pC2_Microway.indd 1 7/15/10 9:20:43 AM7/15/10 9:20:43 AM
[ UPFRONT ]
One of our favorite things to do overat LinuxJournal.com is to check thepulse of the Linux community and ourreadership. We do this fairly regularlywith simple polls on our site. Thesegive us valuable insight into yourinterests, and they give us a fun wayto get feedback on a specific question.Sometimes a simple question generatesa tremendous amount of discussion, andeven a little controversy. We recentlyasked readers to choose their favoriteprogramming language, and we receiveda lot of great answers. Check it out atwww.linuxjournal.com/content/whats-your-favorite-programming-language.
We know that for some of you, yourfavorite programming language is such animportant part of your existence that youare understandably quite opinionatedon the subject. So, as you might expect,there were clear leaders and underdogs,as well as passionate supporters of each.Python tends to be the favorite among
our readers, and it has won more thanone Readers’ Choice Award for favoriteprogramming language. Indeed, Pythonhas staying power, as it once againwas the leading choice amongLinuxJournal.com readers with 24% ofthe votes. Not surprisingly, there weremany skeptics among the commenters,
LinuxJournal.com Programming Survey
[ UPFRONT ]
but Python fans did their best to seteveryone straight about Python’s virtues,and one reader offered this sage advice:
In my opinion, a large Python
system is well organized (like any
language), sticks to standards, has
docs (particularly module docs) and
has tests. Consistent, well-written
Python code makes it fairly obvious
what objects/types functions accept
and return, and having useful
standard types (lists, dics, sets, etc.)
encourages people not to make
exotic variations often. Having
module docs that document what
your function does and its inputs
and outputs clarify any questions.
In a close second and third place wereC and C++, respectively. Both had enthu-siastic support in the comments section,and we can infer from some commentsthat “all of the above” (in the case ofC, C++ and Python) might have been apopular answer as well. Many indicateda preference for different languages fordifferent tasks, and we applaud theirversatility and open-mindedness!Java trailed C++ in fourth place, and it
was cited more than once as a preferredlearning and teaching language. Itprobably also is safe to bet that some ofJava’s popularity is due to the increasing
�
30 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
[ UPFRONT ]
demand for Android applications writtenin Java. One anonymous commentergave us a detailed breakdown of howJava fits the bill versus other languages:
� For fun: Forth.
� For teaching (elementary): Logo.
� For teaching (secondary+): Java.
� For desktop applications: C++ (with Qt).
� For Web services: Java (SE or SE +Servlet only).
� For enterprise internal: Java (EE withor without Spring).
� For enterprise external: Java (SE orSE + Servlet and JSP, no Spring).
� For cloud: C++, Java, Scala andPython together.
� For mobile: Java (Android) C++ andQt (native).
� For embedded systems: C++ or Java(real time).
� For device drivers: C++ or C.
� For deep embedded (no MMU): C,Forth or Assembly, as needed.
Although these may not suit every-
one, I have to give credit for such adetailed response.Getting a little less love were Perl,
C# and Ruby. Although these all havedevoted followings, their fans werelargely outnumbered in this poll. I wasa little surprised to see Ruby score only4% of the votes, as I personally knowso many enthusiastic Ruby coders.Haskell and OCaml each got 1% of
the vote, while the catchall “other”made up the remaining 8%. Mostinteresting were the comments describingthe variety of languages our readers useregularly as well as dabble in.PHP always has a few fans, and
although we can argue that PHP belongsin a separate “scripting languages” poll,its fans still showed support. The samecan be said for JavaScript and Bash,both of which got a little love fromour readers. Perhaps we’ll do a favoritescripting language poll soon, but inthe meantime, since PHP, JavaScriptand Bash tend to be part of my dailylife, it is nice to see I am not alone.There were quite a few mentions of
Scala, LISP and Erlang, as well as oldiesbut goodies, FORTRAN and Cobol. Thelatter occasionally were mentioned inthe context of “getting old”, but frankly,do the classics ever go out of style?It is always fun to read the nostalgia
posts that inevitably appear on these sortsof comment threads. When programmersshare their stories, there is usually a men-tion of the language they started with,
and when our readers share stories of thelanguage they were using in 1976 or even1968, it gives us all a glimpse at wherewe’ve been, and how we all got to wherewe are today. Whether you started as amainframe pioneer or a geeky kid typingout rudimentary BASIC on your TI-99/4A(What? Bill Cosby said it was cool!),you’ve likely had a somewhat meanderingjourney made up of more than a fewlanguages to get to the code you writetoday. In my humble opinion, sharingthese stories is one of the best parts ofLinuxJournal.com. I hope you’ll all joinin the fun and check out the current pollnext time you visit LinuxJournal.com.Don’t be shy! Tell us your stories andopinions in the comments. You neverknow who you may inspire. I suspect itmight be me!—KATHERINE DRUCKMAN
Programming LanguageSurvey Results
C 19% (1,661 votes)C++ 17% (1,488 votes)C# 5% (399 votes)Haskell 1% (126 votes)Java 13% (1,118 votes)OCaml 1% (47 votes)Perl 8% (674 votes)Python 24% (2,025 votes)Ruby 4% (361 votes)Other 8% (670 votes)TOTAL VOTES 8,569
32 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
Last month, I wrote about CoffeeScript,a new, small language that compiles intoJavaScript. CoffeeScript has generated agreat deal of buzz and excitement amongWeb developers, including no less thanBrendan Eich, Mozilla’s CTO and theinventor of JavaScript. Ruby on Rails 3.1,which presumably will be released bythe time this column sees print, includesCoffeeScript, and other frameworksmight follow suit in the future.Even if you’re not interested in the
future of JavaScript or in Ruby on Rails,you owe it to yourself to look atCoffeeScript. First, it’s a new andinteresting language, and I’m definitelya believer in learning new languages aspart of my professional development.Second, CoffeeScript’s syntax makes iteasier to do many things that previouslywere difficult, long-winded or just plainugly in JavaScript. Just as a number oflanguages have emerged that compileto the JVM, but that are easier to usein various ways, so too is CoffeeScript
functionality equivalent at the end ofthe day to JavaScript, but with an easiersyntax that’s more appropriate for manymodern applications.But, perhaps the most interesting
part of CoffeeScript is the fact that,ultimately, it’s just another way of writing JavaScript, which means anythingyou can do in JavaScript, you also cando in CoffeeScript. CoffeeScript pro-grams can run on the server, in suchenvironments as node.js, but they alsocan run in the browser, working inconjunction with Web applications.Things become even more interesting if you use a JavaScript framework,such as jQuery, for developing Webapplications—you can benefit fromthe best of both worlds, enjoying thepower and expressiveness of jQuery,along with the terse and readable syntax of CoffeeScript.This month, I describe some ways
that CoffeeScript and jQuery caninteract in a browser-based program.
CoffeeScript and jQueryCoffeeScript is a better way to write JavaScript, but it integratesjust fine with libraries like jQuery.
REUVEN M.LERNER
AT THE FORGE
CCOOLLUUMMNNSS
Even if you don’t decide to adoptCoffeeScript in your own programs,it’s worth playing with the languageto get the hang of things.
Starting OffI’m going to assume you already haveinstalled CoffeeScript, as well as anysupport files, such as a mode for youreditor. Create a bare-bones HTML file,as shown in Listing 1, and a stylesheet(coffeescript.css), in the same directory,similar to what’s shown in Listing 2.Notice how in the HTML file, I includetwo JavaScript files:
<script
src="http://ajax.googleapis.com/ajax/libs/
�jquery/1.4.2/jquery.min.js"></script>
<script src="app.js"></script>
The first probably is recognizable asthe Google-hosted version of a minifiedversion of jQuery. But the second file,app.js, is the target of the CoffeeScriptcompilation—that is, you’re not goingto write app.js directly. Rather, you’regoing to write CoffeeScript that compilesinto JavaScript.You do this by creating (in the same
directory as the HTML file, unless you want to change the paths in the<script> tag) a CoffeeScript program,named app.coffee. Just to test things, I created a very simple CoffeeScript
program that uses the standard (andannoying!) alert dialog to say “hello”:
alert "hello"
Save this file as app.coffee. On thecommand line, you then want to tellCoffeeScript to compile app.coffee intoapp.js. (Otherwise, it’ll try to executeyour program, which not only will meanthat the resulting JavaScript isn’t available
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 33
AT THE FORGE
CCOOLLUUMMNNSS
Listing 1. coffeescript.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="content-type"
�content="text/html; charset=utf-8" />
<link rel="stylesheet" type="text/css"
�href="coffeescript.css" />
<title>CoffeeScript</title>
<script src="http://ajax.googleapis.com/
�ajax/libs/jquery/1.4.2/
�jquery.min.js"></script>
<script src="app.js"></script>
</head>
<body>
<h1>Headline</h1>
<p>Paragraph 1</p>
<p>Paragraph 2</p>
<p>Paragraph 3</p>
</body>
</html>
for your Web page, but it also will resultin an error if you try to access the DOM,which isn’t available outside a browsercontext.) You can do this with:
coffee --compile app.coffee
The problem with this approach is thatyou need to recompile your CoffeeScriptprogram every time you change it. A bet-ter solution probably is to tell CoffeeScriptto watch the file and compile it everytime a change is detected:
coffee --compile --watch app.coffee
Just after running this, the compilerwill run over app.coffee, producingapp.js. When you load your Web page,app.js will run, and you should havean alert saying “hello”.
Functions and ObjectsIt’s important to remember that althoughCoffeeScript certainly is a different syntaxfrom JavaScript, it is fundamentallythe same language. This means anyfunction or object that you can accessfrom JavaScript can be accessed fromCoffeeScript, with the same name. True,CoffeeScript does offer some shortcutsand syntactic sugar; those basic JavaScriptobjects are still around and kicking.That’s why you could invoke the “alert”function in app.coffee—it’s not thatCoffeeScript has defined a new function,but rather that you’re using the samebuilt-in JavaScript function.This means if you load jQuery in the
same document as a program written inCoffeeScript, you can use jQuery fromwithin CoffeeScript. What does that mean?Well, it means you can access the jQueryobject directly, often abbreviated as $. Forexample, let’s change app.coffee so that ittells you what version of jQuery you’reusing, normally available via $().jquery.You also can do this in CoffeeScript:
alert $().jquery
Let’s do something a bit more excitingnow, using jQuery’s capabilities for easilychanging elements in the DOM based onevents that take place. For example, youcan add the “large” class to all of theparagraph elements in your document.
34 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
AT THE FORGE
CCOOLLUUMMNNSS
Listing 2. coffeescript.css
p.large {
font-size: 30px;
}
p.medium {
font-size: 20px;
}
p.small {
font-size: 10px;
}
In JavaScript, you would do this with:
$("p").addClass("large");
In CoffeeScript, you can use thesame code as above. But, becauseCoffeeScript allows you (like in Ruby)to remove most of the parentheses,you end up with this:
$("p").addClass "large"
Notice how the original jQuery selectorhas remained the same, as has themethod you’re call ing on each of theselected DOM elements. What haschanged is the way you invoke themethod; you no longer need to putparentheses around it.There is a problem with this though.
It will execute immediately upon beingloaded. The problem is that just becausethe JavaScript is executing, it doesn’tmean the HTML all has been loaded orrendered onto the screen. Now, you canget around this in traditional jQuery byputting all of your code inside a call to$(document).ready(), as in:
$(document).ready(
function () {
// Event handlers go here
}
);
You can do the same thing, but in lessspace (of course) using CoffeeScript:
$(document).ready ->
($ "p").addClass "large"
As you can see, CoffeeScript’s syntaxis cleaner and trimmer, without nearly asmany curly braces and parentheses. Youstart off with the same invocation of $with the “document” parameter, andthen invoke the “ready” method on thatobject. You then need to pass a functionto “ready”, which you do by defininga new, anonymous method withCoffeeScript’s -> symbol, cleverly dubbed“dashrocket” in the PeepCode screencastabout CoffeeScript.In other words, you’ve wrapped your
original invocation of “addClass” andfriends inside a function that’s invokedwhen the document is ready. But, you’vecut the number of lines of code in half,without sacrificing readability. Now, let’sdo something a bit more exciting, namelychange the size each time you click on aparagraph. In order to do that, you’ll needto use one of jQuery’s event handlers—specifically, you’ll use the “click” handler,which you set by invoking a selector, the“click” method, and then passing thename of a function. For example, if all youwant to do is display an alert dialog when
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 35
AT THE FORGE
CCOOLLUUMMNNSS
a paragraph is clicked, you can do it withthe following CoffeeScript:
$(document).ready ->
changeSize = ->
alert("changing size!")
$("p").addClass "large"
$("p").click changeSize
Note how I’ve defined two functionshere: an anonymous function for$(document).ready and another functionto which I give the name changeSize.But, of course, you want to do somethinga bit more complex than display an alertdialog; you want to change the size.When changeSize is fired, you want toknow which paragraph to change. Anevent handler always is passed “this”,an all-too-common word in JavaScriptthat confuses many people.One way to get the sizes to rotate is
shown in Listing 3, app.coffee. Basically,your callback function starts off byassigning a local variable, “text”. If thiswere JavaScript, “text” would not be alocal variable, but rather a global one,because you used neither the “var”keyword nor another object (for example,myObject.text). In CoffeeScript, variablesare local, which means you cannot pollutethe global namespace accidentally.
Listing 3 shows a basic use of if/then/elseblocks. Notice there isn’t any need forbraces, begin/end statements or othermarkers. Python programmers will see this(rightly) as a vindication of semanticallysignificant whitespace. I just like the factthat well-indented code is easy to read,
36 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
AT THE FORGE
CCOOLLUUMMNNSS
Listing 3. app.coffee
$(document).ready ->
changeSize = ->
text = $(this)
if text.hasClass "small"
text.removeClass "small"
text.addClass "medium"
else if text.hasClass "medium"
text.removeClass "medium"
text.addClass "large"
else if text.hasClass "large"
text.removeClass "large"
text.addClass "small"
else
text.addClass "large"
true
($ "p").addClass "large"
($ "p").live 'click', changeSize
and that CoffeeScript enforces this on me.You also can see that with rare
exception, you’ve managed to get ridof the parentheses that JavaScript wouldrequire, in favor of terse, clean syntax.You’re still using the same jQuery meth-ods, but you’re doing so in a way that Ifind easier to read.
You then take the changeSize functionand attach it to an event:
($ "p").click changeSize
It might look a bit strange to have the
parentheses around the call to $ "p",which in standard jQuery would look like:
$("p")
CoffeeScript tries to get rid of asmany parentheses as possible, but thereare times when the ambiguity wouldmakes things too difficult for its parser.In such circumstances, you can useparentheses to make things easier.
As you can see from the aboveexample, CoffeeScript makes all ofjQuery’s functions available. No matter
AT THE FORGE
COLUMNS
lj209_September2011_digitalBU:lj_template_current060704.qxt 8/7/11 11:19 PM Page 37
what you might want to do to the textor HTML of your document, you canuse CoffeeScript to do it—adding and removing (and querying) nodes,adding and removing (and querying)attributes, changing text, invokingmenus or anything else you can doin JavaScript. Having jQuery aroundmeans you can make use of its syntaxand abstractions, a potentially killercombination. Indeed, a number ofblog postings (including several men-tioned in the Resources section for thisarticle) indicate that the combinationof CoffeeScript and jQuery is a popularand effective one.
ConclusionjQuery is a popular framework for client-side Web development, providing a largenumber of abstractions and conveniencefunctions for querying and modifying theDOM. CoffeeScript is a language thatmakes it easier to write in JavaScript, bysimplifying the syntax, removing some ofthe most common problems that peoplehave with the language, and providingeasier ways to work with strings, arraysand hashes. But at the end of the day,both jQuery and CoffeeScript are toolsfor working with JavaScript, whichmeans there’s full interoperabilitybetween them. Although the examplesin this column are simple, they demon-strate that it’s easy to get started with
CoffeeScript and even to integrate it intoan existing application. My guess is thatCoffeeScript has a very bright futureand, I should add, deservedly so.�
Reuven M. Lerner is a longtime Web developer, architectand trainer. He is a PhD candidate in learning sciences atNorthwestern University, researching the design and analysisof collaborative on-line communities. Reuven lives with hiswife and three children in Modi’in, Israel.
38 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
AT THE FORGE
CCOOLLUUMMNNSS
Resources
The home page for CoffeeScript, including docu-mentation, quick references, FAQs and annotatedsource code, is at jashkenas.github.com/coffee-script.There is an active and growing community ofCoffeeScript users, with an IRC channel (#coffeescript) and Wiki at GitHub.
A good introduction to CoffeeScript is this presentation written by Jacques Crocker: coffeescript-seattlejs.heroku.com.
PeepCode (peepcode.com), which makes excellentscreencasts on a variety of subjects, has one aboutCoffeeScript that I learned from and enjoyed.
There are many blog postings about CoffeeScript andjQuery. Stefan Buhrmester wrote a nice description ofusing jQuery with CoffeeScript: buhrmi.tumblr.com/post/5371876452/how-coffeescript-makes-jquery-more-fun-than-ever. And, Aaron Russell describeshis experience combining CoffeeScript with jQuery:aaronrussell.co.uk/articles/using-coffeescript-with-jquery.
Finally, the Pragmatic Programmers have released(at the time of this writing) an excellent pre-release“beta book”, written by active CoffeeScript userTrevor Burnham. If you’re interested in learning moreabout this interesting little language, I highly recom-mend this book. It’s aimed mostly at beginners, butgiven the limited number of advanced CoffeeScriptprogrammers out there, that shouldn’t bother you.
40 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
As with many of the challenges wetackle in this column, the latest project hassprawled across more issues than I everexpected when I first received the queryfrom a reader. The question seems reason-ably simple: given a month, day numberand day of the week, calculate the mostrecent year that matches those criteria.There are some obscure and complex
formulas for doing just this, but instead, Idecided it’d be interesting basically to loopbackward from the current year for themonth in question, parsing and analyzingthe output of the handy cal program.The real challenge has been that the
cal program never really was designedto produce easily parsed output, sofiguring out the day of the week (DOW,as we’ve been abbreviating it) involvesbasically counting the number of leadingspaces or otherwise compensating foran average month where the first daystarts mid-week, not neatly on Sunday.An algorithmic-friendly version of cal
would have output where days prior tothe first day of the month would be out-put optionally as zeros or underscores,
making this oodles easier. But it isn’t, sowe have to compensate.
Figuring the Day of the WeekLast month, we wrapped up with ashell function that expected the day,month and year as arguments andreturned the day of the week of thatparticular date in that month on thatyear. In other words, 16 May, 2011,occurs on a Monday:
May 2011
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31
The actual return value of the func-tion in this instance is 2, so 1 = Sunday,2 = Monday, and so on.Given the desired day of the week that
the user specifies and a simple way todecrement the year until we hit a matchcoupled with the function already shown,
Calculating Dayof the Week, FinallyDave wraps up the script and leaves us with the problem of Leap Year.
DAVE TAYLOR
WORK THE SHELL
CCOOLLUUMMNNSS
it should be relatively easy to assembleall the pieces and create—finally—thescript that details when a specific datewas on a specific day of the week.I won’t republish all the code from
previous months (the completed script is83 lines long), but here’s the most salientportion at the end, the section that stepsback year by year to figure out whichone has a matching calendar entry:
echo Looking for $weekday, $day, $month \($monthnum\) \
starting in $mostrecent
# now we need to loop backwards through years until a match
year=$mostrecent
DOW=-1 # start with a dead value
while [ $DOW -ne $desiredDOW ]
do
figureDOW $day $monthnum $year
# echo "> $day $month occurred on a $DOW in $year"
year=$(( $year - 1 ))
done
echo "Got it! $day $month occurred on a $weekday
�most recently in ${year}:"
cal $month $year
Notice that when we find a match, wenot only print out what year had that
WORK THE SHELL
CCOOLLUUMMNNSS
Develop. Scale.Deploy.Full root access on your own virtual server for as little as $19.95/mo
www.linode.com or 609-593-7103
evDl root access on Ful
.poleevyour own virtual server for as little as $19.95/mol root access on
.yloepDyour own virtual server for as little as $19.95/mo
cSyour own virtual server for as little as $19.95/mo
e.layour own virtual server for as little as $19.95/mo
.edolin.www 935-906r omoc 0317-93
date on the specified day of the week,but we also print out the calendar forthat month as a visual confirmation.A few sample runs illustrate:
$ whatyear Friday February 9
Got it! 9 feb occurred on a fri
�most recently in 2006:
February 2006
Su Mo Tu We Th Fr Sa
1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28
$ whatyear wed aug 3
Got it! 3 aug occurred on a wed
�most recently in 2004:
August 2004
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31
Since we convert the day of the weekname and the month name to all lower-case and then truncate anything afterthe first three letters, you can see that“Friday” and “wed” both work, which is anice side benefit. Applications with more
flexible input options obviously are greatlypreferred and make everyone’s life easier.
Something’s Still BrokenOne date breaks the script because itdoesn’t occur every year: February 29.Here’s the problem in a nutshell:
$ cal feb 2010
February 2010
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28
When we try to find a match for “29”on this calendar, there’s no matching out-put, and the conditional tests we have inthe script can’t handle the empty string.It’s not pretty:
$ whatyear mon feb 29
./whatyear.sh: line 21: [: -eq: unary
�operator expected
./whatyear.sh: line 72: [: -ne: unary
�operator expected
Got it! 29 feb occurred on a mon
�most recently in 2010:
February 2010
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6
7 8 9 10 11 12 13
42 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
WORK THE SHELL
CCOOLLUUMMNNSS
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28
You know, if we had these ugly “[” testerror messages but the end result was cor-rect, I probably could live with it, but youcan see that it’s matched on a Februarythat doesn’t even have a 29th day—lame.
However, fixing it might be moretrouble than it’s worth, and it certainlywill cause us to sprawl into a subse-quent column. Instead, I encourage youto grab the entire source code library
from ftp.linuxjournal.com/pub/lj/listings/issue209/11090.tgz, and explorehow to fix it yourself. Yes, I am punting!
Next month, I’l l start on a new shellscripting challenge, and as usual, Iencourage you to send me a quicke-mail note with some ideas you haveon what would be compelling for us todevelop or any particularly interestingscripting problems you’re facing.�
Dave Taylor has been hacking shell scripts for a really long time,30 years. He’s the author of the popular Wicked Cool ShellScripts and can be found on Twitter as @DaveTaylor and moregenerally at www.DaveTaylorOnline.com.
WORK THE SHELL
COLUMNS
RACKMOUNT SERVERSSERVERSRACKMOUNT SERVERS
lj209_September2011_digitalBU:lj_template_current060704.qxt 8/7/11 11:22 PM Page 43
RACKMOUNT SERVERS
44 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
In many ways, I feel sorry for peoplestuck with proprietary operating systems.When something goes wrong or if theyhave a problem to solve, the solutioneither is obvious, requires buying specialsoftware or is impossible. With Linux, I’vealways felt that I was limited only by myown programming and problem-solvingabilities, no matter what problem pre-sented itself. Throughout the years thatLinux has been my primary OS, I’ve runinto quite a few challenging and strangeproblems, such as how to hot-migratefrom a two-disk RAID 1 to a three-diskRAID 5, or more often, how to somehowrepair a system I had horribly broken.
The ProblemRecently, I ran into an interesting challengewhen I had to decommission an oldserver. The server had quite a bit ofsensitive data on it, so I also had toerase everything on the machine securely.Finally, when I was done completelywiping away all traces of data, I had topower off the machine. This is a relatively
simple request when the server is underyour desk: boot a rescue disk, use a toollike shred to wipe the data on all thehard drives, then press the power button.When the server is in a remote datacenter, it’s a little more challenging: usea remote console to reboot into a rescuedisk, wipe the server, then remotely pullthe power using some networked PDU.When, like me, you have to wipe a serverthousands of miles away with no remoteconsole, no remote power, no remotehelp and only an SSH connection, youstart scratching your head.
Why Would You Ever Do This?At this point, some of you might beasking: “Why would you ever need to dothis?” It turns out there are a few differ-ent reasons both legitimate and shady:
1. You have broken hardware. Thiscould be a server with a brokenvideo card, a malfunctioning KVM orremote serial console, or some otherproblem where physical hardware
Remotely Wipe a ServerWhat would you do if you had to erase all the files securely on aserver thousands of miles away?
KYLE RANKIN
HACK AND /
CCOOLLUUMMNNSS
access just doesn’t work.
2. You are locked out from your server.This could happen, for instance, if youcolocate your server in a data centerbut stop paying your bills or somehowhave a falling out with the provider.They revoke your physical access toyour server, but you need to remove allthe sensitive files while the machine isstill available over the network.
3. You have a bad consulting client.Perhaps you are a responsible andtalented sysadmin who sets up aserver for a client in good faith onlyto have that client refuse to pay youonce the server is on-line. You wantto remove your work securely, theclient won’t return your calls, yet youstill have SSH access to the machine.
4. You bought a cloud server with inad-equate tools. It is very popular thesedays to host your server environmentin the cloud; however, one downsideis that many cloud providers cut costsby giving you limited access to man-agement of your cloud instance. Doyou really trust that when you termi-nate a server instance it is securelyerased? Do you get access to toolsthat would let you boot a rescuedisk on your cloud instance? Insome cases, about the only remote
management you have for a cloudserver might be your SSH connection.
5. You are an evil, malicious hacker whowants to cover his tracks. Yes, this isthe least legitimate and most shadyreason to wipe a server remotely, butI figured I should mention it in theinterest of completeness.
6. It’s a challenge. Some people climbmountains, others run marathons,still others try to wipe serversremotely over SSH. You could justbe a person who likes to pushthings to the limit, and this soundslike an interesting challenge.
How Would You Ever Do This?Now that you have worked through thereasons you might need to know how towipe a server remotely, let’s talk abouthow you actually would do it. First, andmost important, there are no redos!When you write random bits to a rawdisk device, especially over SSH, you haveonly one shot to get it right. When I waspreparing this process, I tested my proce-dure multiple times on virtual machines tomake sure my steps were sound. I’m gladI did, as it took a few times to get all thesteps right, confirm my assumptions andget the commands in the correct order.What makes this challenge tricky is the
fact that you will write randomly over
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 45
HACK AND /
CCOOLLUUMMNNSS
the very filesystem you are logged in to.What happens if you overwrite the sshdand shred files while you are running shredand logged in over SSH? More important,what happens when you overwrite thekernel? The main principle that will makethis procedure work is the fact that Linuxlikes to cache files to RAM whenever itcan. As long as you can make sure every-thing you need is stored in RAM, youcan overwrite the filesystem as muchas you want. The trick is just identifyingeverything you need to store in RAM.
Always Have a Plan BSo, I mentioned there was no redo to thisprocedure, but that doesn’t mean youcan’t set up some sort of safety net foryourself. Although I knew that once Ilaunched the shred command it would runcompletely from RAM, what I had to figureout was what commands I would need torun after shred. Even commands like lswon’t work if there’s no filesystem to read.So that I would have some sort of backupplan, I took advantage of the /dev/shmramdisk that all modern Linux systemsmake available. This is a directory that anyuser on the system can write to, and allfiles will be stored completely in RAM.Because I wasn’t sure whether com-
mands like echo (which I would needlater) would work after I had shreddedthe hard drive, I copied it to /dev/shmalong with any other files I thought I
would need. If you have the space, whynot copy all of /bin, /sbin and /lib if youcan. Finally, I knew I would need accessto the /proc filesystem to power offthe server. I assumed I still would haveaccess to /proc even if I had overwrittenthe root filesystem, but I wasn’t 100%certain, so just to be safe, I became root(you can’t assume sudo will work later)and mounted an extra copy of /procunder /dev/shm as the root user:
$ sudo -s
# mkdir /dev/shm/proc
# mount -t proc proc /dev/shm/proc
It turns out I ultimately didn’t needany of these precautions, but it doesn’thurt to be prepared.
It’s Clobbering TimeNow is the point of no return. Just to besafe, I changed to the /dev/shm directoryso my current working directory wouldbe on a ramdisk. Then, I unmounted anyunnecessary mountpoints (like /home)and ran the shred command below onevery nonroot drive on the system. In mycase, I used software RAID, so I also tookthe extra step of hot-removing all butone drive from any RAID array andshredded them separately. Finally, I wasleft with just my root filesystem storedon /dev/sda, so I took a deep breath andtyped the following command:
46 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
HACK AND /
CCOOLLUUMMNNSS
# shred -n2 -z -v /dev/sda
This command writes random bits to/dev/sda for two complete passes (-n2)then does a final pass with zeros so thedrive looks perfectly clean (-z) withverbose output so I can see what’s goingon (-v). Of course, adjust the -n argumentto your particular level of paranoia—twopasses was fine for me. I have to admit,there’s something satisfying and strangeabout overwriting the root filesystemwhile you are still logged in.Once the shred process completed, I
had a completely empty filesystem. Itwas weird—commands like ls gave odderrors, and I knew I was isolated in my/dev/shm jail. All that was left was toshut down the server, but how do youdo that when /sbin/shutdown is erased?No problem, you might say, just kill PID1, since if you kill init, it will halt thesystem. That would work if, say, the killprogram still were around. In this case,the only way I had left to shut down thesystem was via the /proc interface. The/proc directory is a special filesystem thatallows you access to processes and kernelinformation, and it resides entirely inRAM, so my little shred stunt didn’t wipeit out. To halt the machine, just enablethe sysrq interface in the kernel, andthen send the right command to sysrq:
# echo 1 > /proc/sys/kernel/sysrq
# echo o > /proc/sysrq-trigger
If the halt command doesn’t work, orif you just want to reboot the machineinstead, you would type:
# echo b > /proc/sysrq-trigger
Now you might be asking yourself, didn’tI overwrite the echo command? After all,/bin/echo is on the root filesystem. It turnsout I didn’t even have to rely on my copyof the command under /dev/shm—echo isone of the programs that are built in tothe bash shell. When you execute echo,bash executes the version that is built into itself, and because I already was insidea bash shell, the executable ran from RAM.Once you run the last echo command,the kernel instantly will halt. Any remotepings or other commands will stop, andthe system will be powered off.As a final note, I want to say that even
if you don’t think you’ll ever need to go tosuch lengths to wipe a server, I think thisprocedure is such fun that you should atleast try it in a disposable virtual machine.Shred the system and see which commandsstill work and which ones don’t. As an extrachallenge, see if you can get commands torun within /dev/shm.�
Kyle Rankin is a Sr. Systems Administrator in the San FranciscoBay Area and the author of a number of books, including TheOfficial Ubuntu Server Book, Knoppix Hacks and Ubuntu Hacks.He is currently the president of the North Bay Linux Users’ Group.
48 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
HACK AND /
CCOOLLUUMMNNSS
High Performance Computing, Low Latency, Networks, DataCenters, Cost Savings – the largest meeting of High Performance
Computing in New York in 2011.
This HPC networking opportunity will assemble 800 Wall Street ITprofessionals at one time and one place in New York in September 2011.
This show will cover High Performance Computing, High FrequencyTrading, Low Latency, Networks and Switch Solutions, Data Centers,Virtualization, Grid, Blade, Cluster, overcoming Legacy systems.
Our Show is an efficient one-day showcase and networkingopportunity.Register in advance for the full conference program whichincludes general sessions, drill down sessions, an industry luncheon,coffee breaks, exclusive viewing times in the exhibits, and more. Save$100. $295 in advance. $395 on site.
Don’t have time for the full Conference? Attend the free Show.Register in advance at: www.flaggmgmt.com/hpc
Show & Conference: Flagg Management Inc353 Lexington Ave, NY10016(212) 286 0333 [email protected]
SAVETHE DATE
Wall Street IT speakers and Gold Sponsors will lead drill-down sessionsin the Grand Ballroom program.
Show Hours: Mon, Sept 19 8:00 - 4:00Conference Hours: 8:30 - 4:50
This Show is a networking opportunity for the entire IT community.
www.flaggmgmt.com/hpc
8th Annual
HIGH PERFORMANCE COMPUTINGON WALL STREET Show and ConferenceSeptember 19, 2011 (Monday) Roosevelt Hotel, NYC Madison Ave and 45th St, next to Grand Central Station
2011
Global InvestmentTechnology
Register today online. See HPC, Low Latency, Networks, Data Centers,Speed, Cost Savings. Wall Street markets will assemble at the 2011HPC Sept. 19 show to see these new systems live on the show floor.
Sponsors
Visit: www.flaggmgmt.com/hpc
��������(+/1����&-.#%��#2,/.������������������������#'&��
50 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
NEW PRODUCTS
DeLorme’s inReachWe’ll be expecting letters to the editor from Antarcticaonce the more intrepid among you get DeLorme’s newinReach, a personal communicator that delivers two-way communication beyond the reach of cell-phonesignals and one-way satell ite systems. The IridiumCommunications-based inReach offers “pole to pole”two-way satellite text messaging, delivery confirmations,SOS capabilities, remote tracking and an Android smart-phone interface. The device’s core communicationscomponent is the Iridium 9602 short-burst datatransceiver, which utilizes the company’s far-reachingsatell ite network. The GPS-enabled inReach can be used by itself or paired witheither an Android smartphone or a DeLorme Earthmate PN-60w. With the standaloneinReach, users can send pre-loaded text messages to designated recipients andactivate remote tracking, allowing others to follow one’s travels on-line via a“bread crumb” trail. When paired with an Android or the DeLorme EarthmatePN-60w, users enjoy full-featured, two-way text messaging to and from e-mailaddresses and cell phones, as well as the ability to post to Facebook and Twitter.www.delorme.com
Mellanox’s FDR 56Gb/s InfiniBand SolutionsMellanox recently unveiled a complete end-to-endsolution for FDR 56Gb/s InfiniBand consisting of adaptercards, switch systems, software and cables, a feat thatthe firm calls an industry first. The solution consists ofMellanox’s ConnectX-3 FDR 56Gb/s InfiniBand adapters,
SX-6000 series switch systems, Unified Fabric Manager (UFM), Mellanox OS, softwareaccelerators and FDR copper and fiber cables. As a package, it delivers “the highestlevel of networking performance while reducing system power”, according to thecompany. The combination enables cost-effective networking topologies for HPC,financial services, database, Web 2.0, virtualized data centers and cloud computing.www.mellanox.com
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 51
NEW PRODUCTS
Emphase’s S3 Series Compact SATA modulesEmphase’s new S3 Series l ine of compact SATAmodules—that is, the CFast, SATA Flash Module,and mSATA—offers big performance in a small
package, combining performance, dependability and longevity with extremelylow power consumption and the most compact footprint. When space is anissue, this team of SLC NAND solid-state devices is an ideal solution. Transferspeeds have been increased to 120R/100W; capacities range from 1GB–32GB,with a 64GB capacity in the pipeline. The quick-and-rugged CFast S3 is thesolution for data mobility or where hot-swap functionality is a must. The SATAFlash Module S3 weighs in at less than 10 grams and plugs directly, verticallyor horizontally, in to a board’s SATA port. The mSATA S3 is ideal for a low-profileembedded system, integrating high performance and capacity at less than 4mmthick. All modules now integrate TransferSAFE technology to weather-inconsistentpower scenarios and operate off as l ittle as 0.5 Watts, well below the averageof 3 Watts.www.emphase.com
Acquia’s CommonsHere’s a recipe for a kick-booty social software solution: downloadDrupal, stir in Acquia’s updated open-source Commons 2.0, moldcarefully with your creative vision and skil l and voilà—you’ve gotyourself an engaging community Web site. Commons melds rapidlyevolving social Web features, including activity streams, social net-working, blogs, wikis, badges and events together with enterprise-class analytics, support and management services. The result is a“prepackaged open-source alternative to proprietary social businesssolutions” that gives companies extensive freedom to extend andadapt their community sites to meet unique business needs. Version 2.0 addsfeatures such as increased style flexibil ity, access to Acquia Cloud, increasedconfiguration options, more support resources and complete design and themingcontrol, including prebuilt Commons themes. A number of service packages areavailable from Acquia.www.acquia.com
52 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
NEW PRODUCTS
David Kennedy, Jim O’Gorman, DevonKearns and Mati Aharoni’s Metasploit: A Penetration Tester’s Guide (No Starch)You may be familiar with the the Metasploit Framework, atoolset that makes discovering, exploiting and sharing vulner-abil it ies quick and relatively painless. The challenge for secu-rity professionals is that the documentation is lacking, and it can be hard tograsp for first-time users. Metasploit: A Penetration Tester’s Guide from NoStarch Press intends to fi l l this gap by teaching how to harness the Framework,use its many features and interact with the vibrant community of Metasploitcontributors. Readers will learn to find and exploit unmaintained, misconfiguredand unpatched systems; perform reconnaissance and find valuable informationabout targets; bypass antivirus technologies and circumvent security controls;integrate Nmap, NeXpose and Nessus with Metasploit to automate discovery;use the Meterpreter shell to launch further attacks from inside the network;harness standalone Metasploit util it ies, third-party tools and plugins; and learnhow to write Meterpreter post-exploitation modules and scripts.www.nostarch.com
Jason Andress and Steve Winterfeld’s Cyber Warfare: Techniques, Tactics andTools for Security Practitioners (Syngress)Jason Andress and Steve Winterfeld’s new book Cyber Warfare:Techniques, Tactics and Tools for Security Practitioners exploresthe battlefields, participants and the tools and techniques used
during today’s digital conflicts. The concepts discussed in this book will givethose involved in information security at all levels a better idea of how cyberconflicts are carried out now, how they will change in the future and how todetect and defend against espionage, hacktivism, insider threats and non-stateactors l ike organized criminals and terrorists. The authors provide concreteexamples and real-world guidance on how to identify and defend a network againstmalicious attacks, dive deeply into relevant technical and factual informationfrom an insider’s point of view, and outline the ethics, laws and consequencesof cyber war and how computer criminal law may change as a result.www.syngress.com
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 53
NEW PRODUCTS
Likewise OpenWith Likewise Open, recently upgraded to version 6.1, admins canstandardize on Microsoft Active Directory in their enterprise networkswithout losing the flexibility to choose non-Microsoft operating systems.The open-source Likewise Open allows single sign-on for criticalenterprise applications, such as Apache Tomcat, IBM WebSphere,Oracle WebLogic and JBoss Application Server. Newly open-sourcedintegrations include the Kerberos/NTLM JAAS login module andSPNEGO Kerberos/NTLM servlet fi lter, which can be integrated withany servlet spec2.3-compliant application server. Customers wishing toextend event management, auditing and reporting for compliance withSOX, PCI-DSS, HIPPA and other industry standards to these integrationscan do so by upgrading to Likewise Enterprise.www.likewise.com
QLogic’s InfiniBand Fabric SuiteThe new 7.0 release of QLogic’s InfiniBand Fabric Suite(IFS)—a fabric management software package thatenables users to optimize fabric performance andcommunications efficiency for HPC clusters—is nowavailable. IFS 7.0’s leading new feature is the integra-tion of vFabric QoS (Quality of Service) with work-flow schedulers from AdaptiveComputing and Platform Computing, making it possible for HPC users to set apriority and QoS level as the message-passing interface application is beingscheduled. This reduces management overhead, simplifies cluster schedulingand optimizes use of the fabric. Other noteworthy features include an enhancedFabric Viewer with Fabric Dashboard, improved static routing performance, animproved Congestion Control Algorithm and support for both NVIDIA GPUs andRed Hat Enterprise Linux 5.6/6.1. The result of this feature set, says QLogic, isan ability for HPC customers to obtain maximum performance and efficiencyfrom their cluster investments while simplifying management.www.qlogic.com
Please send information about releases of Linux-related products to [email protected] or New Products c/o Linux Journal, PO Box 980985, Houston, TX 77098. Submissions are edited for length and content.
Giada—Hard-Core Live Loopingwww.monocasual.com/giadaLive DJs chasing a simplistic butattractive application are going to loveGiada. At the same time, Giada alsocovers something I’ve been wanting fora year now, which is a way to triggerindividual samples from a computerkeyboard simply in real time. Accordingto the documentation:
Giada is a free, minimal, hard-core
audio tool for DJs and live perform-
ers. Load up to 32 samples, choose
to play them in single mode (drum
machine) or loop mode (sequencer),
and start the show with your com-
puter keyboard as a controller. Giada
aims to be a compact and portable
virtual device for Linux and Windows
for production use and live sets.
Installation Using Giada is prettyeasy, but its ease of use comes at aprice. It’s a freeware binary. This is thefirst freeware program I’ve covered,but don’t worry, I won’t make a habitof it.The project’s Web site consists of a
(well-designed) single page, withdownload links for both the Windowsand Linux versions of Giada.As far as l ibrary requirements go,
the documentation states:
Giada is based upon RtAudio and
FLTK (GUI). They are both statically
linked, but the former needs
libjack.so to provide JACK’s features.
In a modern Linux-based OS, you
should be able to run Giada without
any further installation or hack.
54 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
NEW PROJECTS
Fresh from the Labs
The Giada hard-core looping program forreal-time DJ performances with a GUIthat’s sleekly minimal.
Giada in its full-flight recording mode letsyou layer a live performance piece by piece.
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 55
Regarding binaries, the manual alsonoted: “This software is compiled forx86 processors; we sti l l don’t knowwhat happens if you run it under a64-bit OS/environment; try it and tellus your experience.” Yes, I’m on 64-bitLinux, and it runs just fine.Once you have the dependencies out
of the way, download the latest tarballand extract it. Personally, I found I couldjust open the new folder, click on thebinary, and it worked. For those wantingmore control, open a terminal in thenew folder, and enter the command:
$ ./giada_lin
Usage Once you’re inside, using Giadais actually pretty easy. Although I wasrather confused at first glance, a quick bitof “RTFM” shows that its methodology isvery basic, but you need to understand afew things from the outset.First, this isn’t for programming songs
over some sort of grid, suchas the way that programslike Fruity Loops operate.Giada is for playing l ive.All of your actions take placein real time, as you performwhat is essentially a live DJset (even if it’s only in yourbedroom to an audience ofone). So trust me, you’ll want topractice before using it in public.Second, Giada is designed to be run
by your keyboard, and by that I mean
the thing on which you type, and notsomething that resembles a piano.Before I explain the three modes of
operation, let’s first load some samples sowe have something to play with. In termsof format, Giada likes only 44KHz .wavfiles. A good starting point is Hydrogen’sdrumkits. If you have Hydrogen installed(and if you don’t, at least install thedrumkits), look for .wavs under/usr/share/hydrogen/data/drumkits.Kicks, snares and a cymbal or two—hi-hats in particular—are the best startingpoint. With these, you can lay down abasic beat and then layer other samplesover the top to make a song.To load these samples, click on the
long and wide buttons that say, “-- nosample --”, and choose your .wav filefrom the file browser. Now, if you lookto the left of each sample, a keyboardcharacter is shown; with this, you switchsamples on and off. Try pressing it now,and nothing will play, but fear not. I
discuss Giada’s three running modesbelow, and it all will make sense.
Oneshot mode: this is the mostbasic way of operating Giada. Press a
NEW PROJECTS
First, this isn’t for programmingsongs over some sort of grid,such as the way that programslike Fruity Loops operate.Giada is for playing live.
keyboard button, and that button’ssample will play. However, first you mustturn on this mode, as well as turn upthis sample’s volume. Starting with thevolume, the empty circle to the immedi-ate right is actually a volume knob.Clicking and dragging inside the circleturns up the volume. However, unlessyou’ve pressed the Play button, therestill will be no sound; you have toenable Oneshot mode.The next control to the right, with the
small circle inside the square, is the keyto operating Giada. Click the button,and you’ll have a choice of loopingmodes, or Oneshot “basic”, “press” and“retrig”. Choose basic, press the key,and at last, a sound plays!With this basic mode, you press a key
and a sample plays until it’s finished—pretty basic. But, you also can interruptthe sample by pressing the key again.With the “press” option, you have tohold down the key to play the sample,and as soon as you release it, thesample stops. The “retrig” option (andthis is the functionality I’ve been chasing)plays a sample upon pressing a key,but pressing again restarts the samplewhether or not the sample has finished.You even can keep thrashing away atthe key for instant response, which ishandy for playing hi-hat notes or rippingup a waveform.
Loop mode: this is the second mode,and perhaps the most conventional.When Giada is actually playing, choosing
either “Loop . basic” or “Loop . once”plays a sample on the next bar along. Inorder to use this, press the Play buttonnear the top-left corner, then press eachsample’s button to activate/deactivateit on the next bar. The “basic” optionsimply keeps the sample looping untilyou turn it off manually; the “once”option plays a sample until it’s finished,and then starts it again at the beginningof the next bar.
Recording mode: this is the pièce derésistance. Basic loops can be turnedon and off willy-nil ly; however, theOneshot samples start turning thisinto a real live performance. As thebar moves along each of its counts,every time you play a note, that noteis repeated every bar that follows.Using this method, you genuinely can layer an entire song, creating newbeats on the fly. Be wary, however.Every note you play is a commitment,and you’ll be stuck with that noterepeating for the rest of the song. Get it right, and you’ll have peopledancing. Get it wrong, and you’ll looktotally lame and ruin the party.Nevertheless, some tools are available
to help out the mere mortals amongus. The beat bar will be immediatelyobvious, because it’s the only movingthing on screen. Use this and its(default) four boxes to guide yourcounting. Over to the right, the boxthat’s marked “off” is for quantizingyour music. For the uninitiated, this
56 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
NEW PROJECTS
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 57
aligns your notes to even places on amusical grid, removing the element ofhuman error: “1b” is the most severe,making each note land on a whole count;“8b” is the least severe, allowing youto make much more intricate music.If you look farther right, you’ll see
a tempo and beat number, set to adefault BPM of 120 and a time signa-ture of 4/4. You can turn these up ordown, allowing for strange feels, suchas 7/4 @ 72 BPM (less dance-friendly,but much more trippy).Still, this early software does have its
limitations. First, it’s freeware. In thisday and age? Why? Ech! Second, thereweren’t any panning controls as far as I could tell. Any stereo imaging you’llhave to do beforehand, manually.Nevertheless, this program is incredibly
cool. It allows you to output to JACK,which makes it more powerful, and justlook at it. It’s a techno-minimalist’s wetdream! Giada has an amazing economyof space and features in its design that’squite deceptive. I actually thought thiswas going to be a very short reviewwhen I started. Giada is a must-havefor any electronic musician.
LinkChecker—Web Site Testinglinkchecker.sourceforge.netBroken links are a serious pain in thebackside for Webmasters, and keepingtrack of every individual link becomes solaborious, most Webmasters simply giveup on the idea. Thankfully, there’s a way
to automate the process with LinkChecker.According to its Freshmeat entry:
With LinkChecker, you can check
HTML documents and Web sites for
broken links. It features recursion,
robots.txt exclusion protocol support,
HTTP proxy support, i18n support,
multithreading, regular-expression
filtering rules for links and
user/password checking for authorized
pages. Output can be colored or
normal text, HTML, SQL, CSV or a
sitemap graph in DOT, GML or XML
format. Supported link types are
HTTP/1.1 and 1.0, HTTPS, FTP, mailto:,
news:, nntp:, Telnet and local files.
Installation The Web site has pack-ages for Windows, OS X and Debian(yes, it actually specifies Debian), and the
NEW PROJECTS
LinkChecker makes Web site maintenancethat much easier by scanning your Website for broken links.
obligatory source. The Debian packagesare available for just about every archi-tecture on the planet, and they workedimmediately with my Kubuntu installation.Installing the .deb package is mucheasier, so you may want to run with that;however, unless I missed something, youget only the command-line version.For those who don’t have a Debian-
based system or want to use the GUIversion, here’s a very compressed versionof the instructions.In terms of dependencies, you’ll
obviously need gcc, as well as Python>= 2.6, including its -dev package, aswell as the Qt development tools, whichare named qt4-dev-tools on my system.There also are a bunch of optionall ibrary dependencies for extendedfunctionality, such as bash completion,syntax checks and so on; see the manualfor more information on these.Once you have the dependencies out
of the way, grab the latest tarball fromthe Web site, extract it, and open aterminal in the new folder. Enter thefollowing commands:
$ make -C doc/html
(The above generates the Qt help files.)Then:
$ python setup.py sdist --manifest-only
$ python setup.py build
If your distro uses sudo:
$ sudo python setup.py install
If your distro uses root:
$ su
# python setup.py install
To run the command-line version:
$ linkchecker
To run the GUI version:
$ linkchecker-gui
Usage Actually using LinkChecker isa simple affair. If you’re running thecommand-line version, enter:
$ linkchecker http://{website url}
Of course, LinkChecker also can scan
58 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
NEW PROJECTS
The effect of ten years’ neglect on a Website: here I’m running the console versionagainst my old Tomb Raider page.
local files, but unless the page startswith www, remember to put the preced-ing http:// before on-line pages, or itautomatically scans local files instead.Once inside, enter the URL in the bar
at the top and press Enter.Whether in the GUI or console ver-
sion, LinkChecker gradually makes itsway through all the pages of a givenWeb site, outputting any broken links orwarnings in the process. Depending onthe Web page, the output can be prettyverbose, so console users might considerpiping the output for larger pages.
Once the scan has finished, a readoutis provided with the number of valid andinvalid URLs, as well as various statisticshaving to do with URLs and content.In the end, LinkChecker is a very
simple project that serves its purposebeautifully. Its ease of use and multi-platform nature also make everyday usagemuch more likely. Any serious Webmastershould check out this project.�
John Knight is a 27-year-old, drumming- and bass-obsessed
maniac, studying Psychology at Edith Cowan University in Western
Australia. He usually can be found playing a kick-drum far too much.
NEW PROJECTS
BREWING SOMETHING FRESH, INNOVATIVE OR MIND-BENDING? Send e-mail to [email protected].
60 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
MULTIPLATFORMGNU DEVELOPMENTGet a guitar synth working
with Rock Band 3.NATHANAEL ANDERSON
In my ideal world, mixing games and music
would result in music games that use real
instruments. Harmonix’s Rock Band series is
the closest mainstream realization of this lofty
ideal, except for one major issue. I couldn’t plug
in my guitar and play pro guitar mode songs
out of the box.
Now, before you think this is an impossible
task, one important fact should be noted. I play
a guitar synthesizer. What this means is my
guitar has a hexaphonic pickup that reads and
processes every string’s signal individually and
connects with a 13-pin cable to a guitar processor
with midi out. This means my signal is already
digital and, therefore, does not require any
additional A/D algorithms in my software.
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 61
Custom-built guitar synth, built by the author.
The HardwareI have a few guitars with different 13-pininterfaces that can connect to an AxonAX-100 that has its midi out running tothe midi in port on an M-Audio UNO,which gets connected to my Linux boxrunning g2ghpro. The midi out port onthe UNO then runs into a PS3 Rock Bandadapter and into my PS3. A guitar hexa-phonic pickup with 13-pin midi out isrequired to run g2ghpro. The followinglists are of tested hardware that will work:Pickups:
� Roland GK-3a.
� Roland GK-2a.
� Roland GK-3b.
� Graph Tech LB-63 (any Graph Techpiezo bridge will work if using thehexpander module).
� Godin Synth Access guitars.
Guitars with built-in 13-pin capability:
� Godin LGX-SA.
� Godin Freeway-SA.
� Godin LGXT.
� Roland Ready Fender stratocaster.
� Brian Moore i8.13.
Guitar-to-midi converters:
� Axon Ax-100.
� Roland VG-99.
� Roland GR-55.
� Roland GR-20.
Of the listed hardware, I use a RolandVG-99, Axon Ax-100, Godin LGX-SA,Godin LGX with Roland GK-3a and custom-built Ibanez S540 with GraphTech LB-63 and hexpander.I’ve also tested g2ghpro with a Roland
GK-3a pickup running into a Roland VG-99.Others have reported using a Roland GK-3awith GR-20 and GK-3b with GR-55 on abass guitar as well. G2ghpro currently is theonly solution for using a real bass guitarin the game, as no official bass guitar con-trollers have been released at this point.
The Original ControllerHarmonix created two guitar controllersfor Rock Band 3 pro guitar mode. Thefirst controller released was the Mustang,which is a “button” controller, with 102buttons on the frets and six strings overthe body of the guitar. When I wrotethe initial version of g2ghpro, theFender Squier Pro Strat wasn’t on themarket yet, which is the other proguitar controller that works with thegame. So I had midi dumps only fromthe Mustang to work with.
62 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE Multiplatform GNU Development
MidiIn order to understand how the Mustangworked, I first had to understand whatthe Mustang dumps meant and relatecontroller actions to messages sent.This required a refresher on the midistandard. Midi has 16 separate chan-nels, and changing the sending channelis done by adding the value of thechannel minus one to the message typevalue. My first example is a midi “noteon” message, which has a decimal valueof 144 in the first byte for channel 0.To send the same message on channel6, add 5 to 144.
Example DataAction: held fret one of low E and pickedlow E:
TIMESTAMP IN PORT STATUS DATA1 DATA2 CHAN NOTE EVENT
0023E843 1 -- 95 29 00 6 F 2 Note Off
0023E843 1 -- 95 29 7D 6 F 2 Note On
0023E847 1 -- F0 Buffer: 8 Bytes System Exclusive
SYSX: F0 08 40 0A 05 06 7D F7
The dumps I had found came from aWindows program called Midi Ox, whichshowed the data in hex. The midi specifi-cation shows data in binary, and I wasused to seeing this data with aseqdumpin decimal. I converted all the examples Ihad been provided with into decimal so Icould understand their behavior. The midispec states that note on-and-off eventscontain 3 bytes of data. The first byte isevent type plus channel; the second is
note number, and the third is velocity.From experience, I’ve seen that manydevices send a velocity event of 0 insteadof an actual note off event, which is whatthe above shows. So, the result after ithas been converted to decimal is (note:velocity zero below is really note off):
Type Channel Note Velocity
(Note On 149) 5 41 00
(Note On 149) 5 41 125
SysEx is short for System Exclusivemessages, and they are free-form messages to send data that doesn’t fitinto the predefined midi message types.Initially, I tried to treat the data fromthe dumps as a normal midi controller,and I ignored the SysEx data in the
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 63
From left to right: custom-built guitarsynth, custom-built rack for VG-99, AxonAX-100 and Fantom-XR.
dump, which I later realized is why Ididn’t have any code that made thegame react. All game functions reactto SysEx messages, not note events. Thisis why a guitar synthesizer cannot beplugged in to Rock Band and just work.At this point, I requested more dumps,where different frets were pressed downwith the same string pressed.I converted all the dumps I had to
decimal and compared SysEx messagesto note messages and found a correlation.Here’s the resulting structure of themessages (displayed in decimal):
Part 1 2 3 4 5 6 7 8
Sample 240 8 64 10 1 1 43 247
� Part 1: starting byte of a SysEx message.
� Part 2, 3, 4: identifiers that this is aSysEx message used by the Mustang.
� Part 5: message type (1 = set fretposition, 5 = play string).
� Part 6: midi channel (string on theinstrument).
� Part 7: midi note number.
� Part 8: end SysEx message.
The SoftwareTo explore the message format, I puttogether a quick program for sending acombination of note events and SysEx
messages to the game. I know that theguitar synthesizer hardware required touse the software I was writing isn’t verycommon, so I want to be proactive inremoving any limitations to people usingit. I’ve done midi and C++ programmingwith ALSA under Linux before, but nevermidi on Windows or OS X, and I wantedto be able to support all three to makethe software more accessible.From past experience with RtMidi, I
knew it was written in portable C++, whilesupporting Windows, Linux and OS X.The home page for RtMidi provides
detailed, easy-to-read documentation,with examples for many basic midi tasks.Copy-and-paste examples are providedthat give a base from which to startworking, along with ready-to-compiledemos provided in the tests directoryin the RtMidi source code.A good place to start with RtMidi is
the bundled code in the tests directory.I started by modifying midiout.cpp andtried sending different SysEx messagesbased on my data dumps until finally Iended up with the following:
std::vector<unsigned char> sysExMessage;
sysExMessage.push_back( 240);
sysExMessage.push_back( 8 );
sysExMessage.push_back( 64 );
sysExMessage.push_back( 10 );
sysExMessage.push_back( 1); // 1 sets fret position,
// 5 to play the current string
sysExMessage.push_back( channel + 1 ); // channel
sysExMessage.push_back( note );
64 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE Multiplatform GNU Development
sysExMessage.push_back( 247 );
midiout->sendMessage( &sysExMessage );
With the above SysEx message, I wasable to toggle fret position and stringsplayed in the game. The actual logic tomake this work was less than 100 lines.The full code is available in the Subversionrepository for game2midi in g2ghpro.cpp.RtMidi currently has issues processing
active sensing messages that came frommy Roland gear, which the author isaware of. A flag is provided to filterout active sensing messages. SettingignoreTypes to true on your midi inputobject’s third parameter will work aroundthe issue until it is resolved—for example:
midiin->ignoreTypes( false, false, true);
The main missing feature of RtMidi,as far as the Linux pro audio world isconcerned, is no jack-midi support.The RtMidi documentation listed
compiler flags for all three operatingsystems to link the required libraries, soall that was left for me to do was figureout how to compile under Windows.
Supporting Other Operating SystemsI hadn’t touched a Windows developmentIDE in more than ten years, and I wantedto keep the same code base for all threeoperating systems. Somewhere duringthe past few years, I heard mention ofMinGW (Minimalist GNU for Windows).As I am familiar developing in a
GNU/Linux environment, this soundedlike what I needed. To bring your Linuxdev environment to Windows, usemingw-get-inst, and do a full installation.This will provide you with the MinGWShell, bundled with many standardGNU tools, including SSH. Next, installTortoiseSVN, which is a Subversion clientthat integrates with the Windows shell.Checkout and commit actions areaccessed by right-clicking on folders inExplorer to keep files in sync. The MinGWshell allows for changing drives’ letterslike a standard DOS shell with cd C:.The next problem is how to build the
code based on operating system. Let’slook at two options: Makefiles andautotools. First, let’s look at basicMakefile-based builds and comparethe differences by platform:
# Makefile.linux
all:
mkdir -p deps
g++ -DHAVE_CONFIG_H -I. -I.. -g -O2 -D__LINUX_ALSASEQ__
�-g -O2 -MT midiio.o -MD -MP -MF deps/RtMidi.Tpo
�-c -o RtMidi.o RtMidi.cpp
g++ -DHAVE_CONFIG_H -I. -I.. -g -O2 -D__LINUX_ALSASEQ__
�-g -O2 -MT -midiio.o -MD -MP -MF deps/midiio.TPO
�-c -o midiio.o midiio.cpp
g++ -g -O2 -o midiio RtMidi.o midiio.o -lasound
# Makefile.mingw
all:
mkdir -p deps
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 65
g++ -DHAVE_CONFIG_H -I. -I.. -g -O2 -D__WINDOWS_MM__
�-g -O2 -MT RtMidi.o -MD -MP -MF deps/RtMidi.Tpo
�-c -o RtMidi.o RtMidi.cpp
g++ -DHAVE_CONFIG_H -I. -I.. -g -O2 -D__WINDOWS_MM__
�-g -O2 -MT -midiio.o -MD -MP -MF deps/midiio.TPO
�-c -o midiio.o midiio.cpp
g++ -g -Wl,--enable-auto-import -O2 -o midiio RtMidi.o
�midiio.o -lwinmm
The library I used, rtmidi, requires thatthe platform be defined, so for Linux,define -D_ _LINUX_ALSASEQ_ _, and forWindows, define -D_ _WINDOWS_ _MM_ _.The last step in each is the linkingphase, where you specify systemlibraries to link to the binary. To enablea clean build under Windows, I had toadd -Wl,--enable-auto-import, sofunctions would be auto-imported.Now, let’s look at autotools-based
builds. Usually when an autotools-basedbuild is committed to version control,only non-generated files are committed,which includes configure.ac, Makefile.amand src/Makefile.am. The standardpractice is to create an autogen.shscript that will call the files to generateconfigure, Makefile and other requiredfiles from *.ac and *.am files:
# Autogen.sh
#!/bin/sh
aclocal
autoreconf
automake --add-missing --copy
autoreconf
libtoolize -f --automake
configure.ac (Listing 1) containshost-based auto-detection and is whereto specify which host-based libraries tocheck for and link against.Full code for these examples can
be downloaded from the game2midiproject’s Subversion repository in thebasic-midi-io-example folder.
ConclusionUsing GNU libraries and tools can helpreduce the time and effort required insupporting multiple platforms. I hopethis article encourages you to consideradding multiplatform support to a current or future open-source project.�
Nathanael Anderson has been a UNIX systems administrator forfive years. Family, coding and all things guitar keep him active.Feel free to contact him on his blog at wirelessdreamer.com.
66 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE Multiplatform GNU Development
Resources
Official Midi Specification:www.midi.org/techspecs/midimessages.php
Posts of Mustang Controller Dumps: www.rockband.com/forums/showthread.php?t=207792&page=1
Official RtMidi Home Page:www.music.mcgill.ca/~gary/rtmidi
MinGW: www.mingw.org
Mingw Download: sourceforge.net/projects/mingw/files/Automated%20MinGW%20Installer/mingw-get-inst/mingw-get-inst-20110316/mingw-get-inst-20110316.exe/download
TortiseSVN: tortoisesvn.tigris.org
game2midi Home Page: game2midi.sourceforge.net
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 67
Listing 1. configure.ac#configure.ac
dnl - dnl represents a comment in automake config files
dnl here we specify the
AC_INIT(midiio,0.1)
AM_INIT_AUTOMAKE(midiio, 0.1)
AC_PROG_CXX
AC_LANG_C
dnl Checks for programs.
AC_PROG_AWK
AC_PROG_CC
dnl Check for headers
AC_CHECK_HEADERS(unistd.h)
dnl Checks for typedefs, structures, and compiler characteristics.
AC_TYPE_SIZE_T
dnl Detect OS we're building on
dnl this next line is required to be able to read the host value
AC_CANONICAL_HOST
dnl Use the value here to add support for other operating systems
echo "Host Value: '${host}'"
case "${host}" in
*-mingw32*)
dnl specify Windows specific compiler flags
�and linker options
compile_target=win
CPPFLAGS="$CPPFLAGS -D_ _WINDOWS_MM_ _"
LIBS="$LIBS -lwinmm"
;;
*linux-gnu)
dnl specify Linux specific compiler flags
�and linker options
compile_target=linux
dnl Check for ALSA
AC_CHECK_LIB(asound, snd_seq_event_output_direct,
�alsalib=yes,alsalib=no)
AC_CHECK_HEADERS(alsa/asoundlib.h,alsaheader=yes,
�alsaheader=no)
if test "$alsalib" = "yes"; then
if test "$alsaheader" = "yes"; then
LIBS="$LIBS -lasound"
else
AC_MSG_ERROR([** Coulnd't find ALSA
�header file sys/asoundlib.h **])
fi
else
AC_MSG_ERROR([** Couldn't find ALSA library
�libasound. **])
fi
CPPFLAGS="$CPPFLAGS -D_ _LINUX_ALSASEQ_ _"
;;
esac
AC_HEADER_STDC
AM_CONFIG_HEADER(config.h)
AC_OUTPUT(Makefile src/Makefile)
# Makefile.am
# Here we specify we have files in the source directory to process
AUTOMAKE_OPTIONS = foreign
SUBDIRS = src
# src/Makefile.am
# Here we define there are 2 programs we're compiling
bin_PROGRAMS = midiio midiout
# and here we define what we put together to make the final programs
midiout_SOURCES = midiout.cpp RtMidi.cpp
midiio_SOURCES = midiio.cpp RtMidi.cpp
68 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
R egression testing is a well-established technique to detectboth the introduction of new bugs
and the re-introduction of old bugs.However, most regression tests focusexclusively on correctness while ignoringperformance. For applications withperformance requirements, developersrun benchmarks to profile their code inorder to determine and resolve bottle-necks. However, unlike regression tests,benchmarks typically are not executedand re-validated for every revision. As aresult, performance regressions sometimesare not detected quickly enough.Compared to correctness issues,
performance regressions can be harderto spot. An individual absolute perfor-
mance score rarely is meaningful;detecting a performance regressionrequires relating measurements to previous results on the same platform.Furthermore, small changes in externalcircumstances (for example, other pro-cesses running at the same time) cancause fluctuations in measurementsthat then should not be flagged asproblematic; this makes it difficult to sethard thresholds for performance scores.Also, good measurements often takesignificantly longer than correctnesstests. Performance improvements inone area may cause regressions in others, causing system architects some-times to consider multiple metrics at thesame time. Finally, performance can be
Performance Regression
Monitoring with Gauger
Introducing Gauger, a lightweight tool for visualizingperformance changes that occur as software evolves.
platform-specific. This can make itnecessary to perform performanceevaluations on a range of systems.The Gauger package described in this
article provides developers with a sim-ple, free software tool to track systemperformance over time. Gauger islightweight, language-independent andportable. Gauger collects any number ofperformance values from multiple hostsand visualizes their development overtime (Figure 1). In order to use Gauger,developers need to add the necessaryinstrumentation to their code to obtaina performance measurement and thensubmit it to Gauger with the gaugerfunction call. The gauger function argu-ments are the description of the value, acategory, the value itself and a unit ofmeasurement. Gauger’s Web interfacethen allows visitors to group metrics by
category or by execution host and adjustthe visualized revision range or the sizeof the plot. Gauger is ready to be usedwith many programming languages andrevision control systems, and it is easilyexpandable to accommodate new ones.Gauger’s Web interface can be used
to analyze the collected performancedata in various ways. It can combinedifferent metrics in a single plot and offersa color-coded guide to help visitorsselect only unit-wise compatible metrics.Gauger also allows users to normalizethe data in order to mask differences inabsolute performance between differentexecution hosts. If multiple measure-ments were taken for the same revision,Gauger will show the average andstandard deviation as long as only asingle metric is plotted. For larger projects with many metrics or execution
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 69
Figure 1. Gauger in action: this screenshot shows performance measurements obtainedand visualized by Gauger for the GNUnet Project over the course of a few revisions.
hosts, Gauger offers a search feature tolocate the desired plots. An additionalinstant search keeps the menus free ofirrelevant items.Finally, should further fine-tuning
be needed (for example, for use inpresentations), Gauger can be used toretrieve the gnuplot source of anyplot. The generated gnuplot sourceincludes the plotted data.
Gauger’s ArchitectureGauger uses a traditional client-serverarchitecture, where the clients reportperformance measurements to a centralserver. This architecture allows machinesbehind NAT or with otherwise restrictedInternet access to provide performancemeasurements to Gauger.All of the performance-monitoring
machines to be used with Gaugershould install the Gauger client, andthe software to be tested should be
integrated with the appropriate lan-guage bindings. Language bindingsare designed to be transparent and(except for a few extra system calls)have no negative effects in case theGauger client is not installed on themachine. Thus, it is safe to committhe language bindings to a projectrepository. As long as the (tiny) lan-guage bindings are included, integrat-ing Gauger will not disrupt operationson systems where the Gauger client is not installed.The Gauger server runs the data
collection and visualization part. Datais logged through a RESTful API andsaved in human-readable plain-textfi les. The primary job of the server isto provide a dynamic Web interface to visualize and analyze the collecteddata. All the communication betweenGauger clients and the Gauger serveris done in standard HTTP(S) requests
70 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE Performance Regression Monitoring with Gauger
Figure 2. Gauger architecture: the Gauger server is responsible for authentication andreceives performance data from the clients. The results are stored in a simple text formatin a local directory. PHP is used to generate the Web site.
so that only port 80 (or 443) needs tobe open (Figure 2).
InstallationEach Gauger client installationrequires a local Python (> or = than2.6) interpreter. For the Gauger server,a Web server installation with PHP andgnuplot is required.The provided install.sh script can be
used to install the client, install theserver code into an appropriate loca-tion and generate an updated Apacheconfiguration. The script prompts forkey configuration options, such as theinstallation path and the desired URLat which the Gauger server shouldrun. Installations that do not useApache currently require manuallyconfiguring the Web server.
ConfigurationEach part of Gauger uses a simple configu-ration file. The Gauger client configurationfile contains the remote server URL,followed by the user name and password.Here’s a sample configuration:
https://gnunet.org/gauger/ username password
The Gauger server configuration fi le
contains the directory where data andauthentication information are stored.Listing 1 shows a sample server con-figuration. When the auto-add featureis enabled, new hosts can be added bylogging in to the Web site using afresh hostname and password.
Integrating GaugerAdding a single simple call at theplaces where performance measure-ments are obtained typically is allthat’s required to integrate Gaugerwith existing projects. This call thenstarts the Gauger client process,which, if installed and configured correctly, submits the performance
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 71
Listing 1. A few basic configurationoptions and a list of clients with theirpasswords’ hashes are needed for theGauger server configuration file.
[config]
data = "/var/lib/gauger"
salt = "makemyhashesunique"
auto_add = 1
[hosts]
debian-amd64-grothoff="da39a..."
Adding a single simple call at the placeswhere performance measurements are
obtained typically is all that’s required to integrate Gauger with existing projects.
measurement to the server. On systemswhere the Gauger client is not installed,the call fails silently so as not to disruptnormal operations in any way. Thesyntax of the Gauger client command-line tool is as follows:
$ gauger [-i ID] -c CATEGORY -n NAME \
-u UNIT -d DATA
Here, NAME is the name of the met-ric, and DATA is any floating-pointvalue that represents the performancemeasurement. UNIT is a string thatdescribes the unit of the value, forexample, ms/request. CATEGORY is astring used to group multiple perfor-mance metrics by subsystem. We
recommend using the name of thesubsystem or module here, especiallyfor larger projects.
Revision NumbersGauger can autodetect the currentrevision of the project if the bench-mark is executed in a directory thatwas checked out from a supportedVersion Control System (VCS). Thesupported VCSes are Subversion, Git,hg, bazaar, monotone and GNU arch.For distributed VCSes that do not pro-vide an ascending revision numberingsystem (l ike Git), Gauger uses thenumber of commits to the repository.In this case, all execution hosts usedfor benchmarking should use the sameorigin to keep the data consistent. Ifthe project uses an unsupported VCSor if the benchmark is executed out-side the tree, Gauger needs to knowwhich (revision) number to use for the
72 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE Performance Regression Monitoring with Gauger
Listing 2. The GAUGER macro makes iteasy to integrate Gauger with C code.Note that the code does not need to belinked against any additional libraries(other than libc).
#include <gauger.h>
#include <time.h>
int main() {
time_t start = time (NULL);
do_test ();
time_t delay = time (NULL) - start;
GAUGER ("subsystem", "execution time for f",
delay, "s");
return 0;
}
Listing 3. A simple static method call,leading to a single line of Java code, canbe used to invoke Gauger from Java.
import static org.gnunet.gauger.Gauger.gauger;
class HelloGauger {
public static void main(String[] args) {
gauger ("subsystem", "Speed",
42 /* value */, "kb/s");
}
}
x-axis. The --id ID option is used tosupply the revision number in this case.Some projects may want to use aninternal version number or a timestampinstead of a revision number generatedby their VCS. The only restrictionimposed on the numbers used is thatGauger’s regression monitoringassumes a l inear progression of devel-opment. For projects with multiplebranches under active development,different Gauger servers should be setup for each branch.
Language BindingsGauger ships with bindings for variouslanguages (see, for example, Listings2, 3, 4 and 5) to make integrationeasy. The resulting binaries do notdepend on a Gauger client installationon the target system. The bindingsshould be integrated with the mainproject and, as mentioned before,simply do nothing when invoked if the Gauger client is not installed ornot configured properly.The JavaScript binding is unusual.
Because JavaScript cannot access thelocal fi lesystem to read the configura-
tion fi le, the login data must bestored in a cookie in advance. Thelogin page on the Gauger Web site,which usually is used to set up newaccounts for execution hosts, can beused to set the respective login cookiein the browser. Access to the sourcecode’s repository also is not possiblefrom JavaScript, so the revision numbermust be supplied explicitly to theGAUGER call. Typically, the current revision number is obtained on theserver side. For example, PHP codecan be used to obtain the numberfrom the VCS, or on-commit trigger
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 73
Listing 4. Example Code for InvokingGauger from Python
from gauger import GAUGER
GAUGER("CAT", "NAME", 42, "kb/s")
Listing 5. The browser must be regis-tered with the Gauger Web site beforeGauger can be invoked from JavaScript.Once the login cookie is set, the maindifference from other languages is thatthe JavaScript code must supply itsrevision number explicitly.
[...]
<script type="text/javascript"
src="http://www.example.com/gauger.js">
</script>
[...]
var rev = $Revision$;
/* or */
var rev = <?php echo get_revision() ?>;
var performance = do_test();
GAUGER ('CAT', 'NAME',
performance, 'kb/s', rev)" />
[...]
functions provided by the VCS couldbe used to insert the number into thesource code.
Selecting Proper UnitsGauger provides developers completefreedom with respect to the names,values and units of the performancemetrics to be monitored. So, how doyou choose a good unit? Naturally, partof the unit always is dictated by thekind of performance characteristic youare interested in—for example, executiontime (seconds) or space consumption(bytes). However, generally it’s a goodidea always to relate this basic unit tothe amount of work performed as partof the unit given to Gauger.For example, suppose a benchmark
measures the execution time for 100GET requests. In this case, it is betterto choose a name “GET request perfor-mance” with unit “requests/second”(and log the execution time divided by100) instead of the name “Time for100 GET requests” with unit “seconds”.The reason for this is it’s quite possiblethe benchmark will be adjusted in thefuture—for example, to run 1,000 GETrequests. If performance is logged as“requests/second”, such a changewould then not require any changes to
the name of the tracked metric. As aresult, the performance regressionanalysis can continue to track the metricin the same curve.Additionally, Gauger allows different
results to be compared by adding newmetrics to existing plots. If the newmetric uses the same unit as the oldone, they will use the same y-axis;otherwise, the new one will be plottedagainst a second y-axis on the rightside of the plot. This limits the numberof units per plot to two. We recom-mend using the same units whereapplicable (for example, no mixing of“seconds” and “milliseconds”) to makecomparative analysis easier.
Related ProjectsGauger does not include for supportfor actually automatically running the benchmarks on various systems.However, this is not a drawback,because an excellent tool for that purpose exists in the form of Buildbot.Buildbot typically is used for regressiontesting and, hence, is by itself not suitable for identifying performanceregressions. Nevertheless, Buildbotrequires a similar network setup—thatis, clients that run the tests connect toa public server. This makes it trivial to
74 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE Performance Regression Monitoring with Gauger
Gauger provides developers complete freedomwith respect to the names, values and units ofthe performance metrics to be monitored.
combine a Buildbot setup with Gauger.Buildbot is used to execute regressionand performance tests, and Gaugervisualizes the development of performancemetrics over time.Another tool for monitoring perfor-
mance is Munin. Like Gauger, Muninallows users to specify which perfor-mance measurements should be created.In contrast to Gauger where executionhosts push data to the server, theMunin server periodically pulls all participating systems for a performancescore. As a result, NATed systems arenot easily supported. Also, becauseMunin stores the data indexed by timeand not revision number, and giventhat software performance may differwidely between different platforms,not all systems may have performancescores ready at fixed time intervals.Although Munin is not a good fit forperformance regression analysis fordevelopers, it likely is a better fit forsystem administrators who need tomonitor system performance.
ConclusionGauger offers a l ightweight and language-independent approach forintegrating performance regressiontesting with existing development pro-cesses for projects using a wide rangeof version control systems. With Gauger,performance regressions are detectedearly, providing users with softwarethat finally is improving consistently in
both correctness and performance.�
Bart Polot is working on his PhD as a researcher at theTechnische Universität München. His research interestsinclude security, networking, routing and botnets.
Christian Grothoff is an Emmy-Noether research groupleader at the Technische Universität München. His researchinterests include compilers, programming languages,software engineering, networking and security.
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 75
Resources
Gauger: freshmeat.net/projects/gauger
Buildbot: buildbot.org
Munin: munin-monitoring.org
76 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
In a compiled language, the makefileis arguably the most important partof any programming project. Tocompile your project, you first haveto compile each source file into an
object file, which in turn needs to belinked with system libraries into thefinal executable fi le. Each commandcan have a considerable number ofarguments added in. That’s a lot oftyping and a lot of potential for mistakes.The more source files you have, themore complex the compilation processbecomes, unless you use makefiles.Most Linux users have at least a cursoryknowledge of make and makefiles(because that’s how we build softwarepackages for our systems), but notmuch more than that. Most developersprobably don’t have too much in-depthexperience with makefiles, becausemost Integrated DevelopmentEnvironments (IDEs) have the capability
of managing makefiles for them.Although this is convenient most ofthe time, knowing more about how makeworks and what goes into makefilescan help you troubleshoot compilationerrors down the road.According to make’s man page,
“The purpose of the make utility is todetermine automatically which piecesof a large program need to be recom-piled, and issue the commands torecompile them.” Essentially, make isused to determine efficiently (andwithout user error) which portions ofthe source code have been updatedsince the last compilation and recom-pile them. It can be used for more thanjust compiling programs. Because itisn’t limited to any particular language,you can use it for anything you can comeup with that relates to the modifieddate of a group of files.Running make is a straightforward
man makeA PRIMER ON THE MAKE UTILITYIn the modern world of Integrated Development Environments, weforget what really goes into compiling a large code project. Thisarticle should be a refresher on (or teach for the first time) the
basics of makefiles, the most underrated part of any code project.
Adrian Hannah
process. The more convoluted portionof using make is constructing themakefile. The makefile is a file thatconsists of a series of rules that definethe dependencies of your project.These rules govern the behavior ofmake during execution.
Rules and TargetsEach rule in the makefile is an inde-pendent series of commands that areexecuted in order to build a target.Make does not necessarily run eachrule in order. Make will run throughthe rules recursively, building eachtarget in turn, based on modification.Rules are formatted like this:
target: dependency list ...
commands
...
The target is typically the name ofa fi le, but it can be a phony target(discussed later in this article). Thedependency l ist is a space-separatedlist of files that designate whether the
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 77
IMPORTANT:Command lines must be indented with tab
characters; spaces cause funky errors.
This has been a design flaw in make for
decades. Empty lines must sti l l have a
tab character or else make will throw a fit.
THE BASICS:� Comments start with a pound sign (#).
� Continuation of a line is denoted by a
back slash (\).
� Lines containing equal signs (=) are
variable definitions.
� Each command line typically is executed in
a separate Bourne shell—that is, sh1.
� To execute more than one command line
in the same shell, type them on the same
line, separated by semicolons. Use a \ to
continue the line if necessary.
Listing 1. Example Makefile
CC=g++
CFLAGS=-c -Wall
LDFLAGS=
SOURCES=main.cpp hello.cpp factorial.cpp
OBJECTS=$(SOURCES:.cpp=.o)
EXECUTABLE=hello
all: $(SOURCES) $(EXECUTABLE)
$(EXECUTABLE): $(OBJECTS)
$(CC) $(LDFLAGS) $(OBJECTS) -o $@
.cpp.o:
$(CC) $(CFLAGS) $< -o $@
target needs to be rebuilt. The com-mands can be any shell command, solong as the target is up to date at theend of them. It is imperative that youindent the commands with a tab char-acter and not spaces. This is a designflaw in make that has yet to be fixed,and it will cause some strange andobscure errors should you use spacesinstead of tabs in your makefile.When make encounters a rule, it first
checks the files listed in the dependencylist to ensure that they haven’t changed.If one of them has, make looks throughthe makefile for the rule containingthat file as the target. This recursioncontinues until a rule is found whereall the dependencies are unchanged orrebuilt (or have no further dependen-cies), and then make executes the listedcommands for that rule before returningto the previous rule, and so on, untilthe root rule has been satisfied and itscommands run.You may use pattern-matching char-
acters to describe dependencies in thedependency list or in commands, butthey may not be used in the target.
Phony TargetsPhony targets (also called dummy orpseudo-targets) are not real files; theysimply are aliases within the makefile.As I mentioned before, you can specifytargets from the command line, and thisis precisely what phony targets are usedfor. If you’re familiar with the process of
using make to build applications onyour system, you’re familiar with makeinstall (which installs the applicationafter compiling the source) or makeclean (which cleans up the temporaryfiles created while compiling thesource). These are two examples ofphony targets. Obviously, there are no“install” or “clean” files in the project;they’re just aliases to a set of commandsset aside to complete some task notdependent on the modification timeof any particular fi le in the project.Here is an example of using a “clean”phony target:
clean:
-rm *.o my_bin_file
Special TargetsSome special targets are built in tomake. These special targets hold specialmeaning, and they modify the way makebehaves during execution:
.PHONY — this target signifies whichother targets are phony targets. If atarget is listed as a dependency of.PHONY, the check to ensure that thetarget file was updated is not per-formed. This is useful if at any timeyour project actually produces a filenamed the same as a phony target;this check always will fail when execut-ing your phony target.
.SUFFIXES — the dependency list ofthis target is a list of the established filesuffixes for this project. This is helpful
78 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE man make: a Primer on the Make Utility
when you are using suffix rules (discussedlater in this article).
.DEFAULT — if you have a bunch oftargets that use the same set of com-mands, you may consider using the.DEFAULT target. It is used to specify thecommands to be executed when no ruleis found for a target.
.PRECIOUS — all dependencies of the.PRECIOUS target are preserved shouldmake be killed or interrupted.
.INTERMEDIATE — specifies whichtargets are intermediate, or temporary,files. Upon completion, make will deleteall intermediate files before terminating.
.SECONDARY — this target is similarto .INTERMEDIATE, except that thesefiles will not be deleted automaticallyupon completion. If no dependenciesare specified, all fi les are consideredsecondary.
.SECONDEXPANSION — after theinitial read-in phase, anything listedafter this target will be expanded fora second time. So, for example:
.SECONDEXPANSION:
ONEVAR = onefile
TWOVAR = twofile
myfile: $(ONEVAR) $$(TWOVAR)
will expand to:
.SECONDEXPANSION:
ONEVAR = onefile
TWOVAR = twofile
myfile: onefile $(TWOVAR)
after the initial read-in phase, butbecause I specified .SECONDEXPANSION,it will expand everything following asecond time:
.SECONDEXPANSION:
ONEVAR = onefile
TWOVAR = twofile
myfile: onefile twofile
I’m not going to elaborate on thishere, because this is a rather complexsubject and outside the scope of thisarticle, but you can find all sorts of.SECONDEXPANSION goodness out thereon the Internet and in the GNU manual.
.DELETE_ON_ERROR — this targetwill cause make to delete a target if ithas changed and any of the associatedcommands exit with a nonzero status.
.IGNORE — if an error is encounteredwhile building a target list as a depen-dency of .IGNORE, it is ignored. If thereare no dependencies to .IGNORE, makewill ignore errors for all targets.
.LOW_RESOLUTION_TIME — forsome reason or another, if you havefiles that will have a low-resolutiontimestamp (missing the subsecondportion), this target allows you to desig-nate those files. If a file is listed as adependency of .LOW_RESOLUTION_TIME,make will compare times only to thenearest second between the target andits dependencies.
.SILENT — this is a legacy targetthat causes the command’s output to
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 79
be suppressed. It is suggested that youuse Command Echoing (discussed inthe Command Special Characters sec-tion) or by using the -s flag on thecommand line.
.EXPORT_ALL_VARIABLES — tellsmake to export all variables to any childprocesses created.
.NOTPARALLEL — although makecan run simultaneous jobs in order tocomplete a task faster, specifying thistarget in the makefile will force maketo run serially.
.ONESHELL — by default, make willinvoke a new shell for each command itruns. This target causes make to use oneshell per rule.
.POSIX — with this target, make isforced to conform to POSIX standardswhile running.
VariablesIn other versions of make, variablesare called macros, but in the GNU
version (which is the version you likelyare using), they are referred to as variables, which I personally feel is amore appropriate title. Nomenclatureaside, variables are a convenient wayto store information that may be used multiple times throughout themakefile. It becomes abundantly clearthe first time you write a makefile and then realize that you forgot acommand flag for your compiler in all 58 rules you wrote. If I had usedvariables to designate my compilerflags, I’d have had to change it onlyonce instead of 58 times. Lessonlearned. Set these at the beginning of your makefile before any rules.Simply use:
VARNAME = information stored in the variable
to set the variable, and do use $(VARNAME) to invoke it throughoutthe makefile. Any shell variables that
80 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE man make: a Primer on the Make Utility
PREDEFINED VARIABLES� $? — evaluates to the l ist of compo-
nents that are younger than the current
target. Can be used only in description
fi le command l ines.
� $@ — evaluates to the current
target name. Can be used only in
description fi le command l ines.
� $$@ — also evaluates to the current
target name. However, it can be
used only on dependency l ines.
� $< — the name of the related fi le
that caused the action (the precur-
sor to the target). This is only for
suffix rules.
� $* — the shared prefix of the target
and dependent—only for suffix rules.
existed prior to calling make will existwithin make as variables and, thus, areinvoked the same way as variables. Youcan specify a variable from the com-mand line as well. Simply add it to theend of your make command, and it willbe used within the make execution.If, at some point, you need to alter
the data stored in a variable tem-porarily, there is a very simple way tosubstitute in this new data withoutoverwriting the variable. It’s doneusing the following format:
$(VARNAME:find=replace)
where find is the substring you are try-ing to find, and replace is the string toreplace it with. So, for instance:
LETTERS = abcxyz xyzabc xyz
print:
echo $(LETTERS:xyz=def)
will produce the output abcdef xyzabc def.
Suffix RulesIn certain situations, you will find thatthe rules for a certain fi le type areidentical except for the fi lename. Forinstance, a lot of times in a C project,you will see rules l ike this:
file.o: file.c
cc -O -Wall file.c
because for every .c fi le, you need tomake the intermediate .o fi le, so thatthe end binary then can be built.Suffix rules are a way of minimizingthe amount of time you spend writingout rules and the number of rules inyour makefile. In order to use suffixrules, you need to tell make which fi lesuffixes are considered significant(suffix rules won’t work unless the
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 81
� CC — the name of the compiler.
� DEBUG — the debugging flag. This is
-g in both g++ and cxx. The purpose
of the flag is to include debugging
information into the executable, so
that you can use util it ies l ike gdb to
debug the code.
� LFLAGS — the flags used in l inking.
As it turns out, you don’t need any
special flags for l inking. The option
listed is -Wall, which tells the compiler
to print all warnings. But, that’s fine.
We can use that.
� CFLAGS — the flags used in compil-
ing and creating object fi les. This
includes both -Wall and -c. The -c
option is needed to create object
fi les—that is, .o fi les.
COMMON VARIABLES FOR C++ PROGRAMMING
suffix is defined this way), then writethe generic rule for the suffixes. Inthe case described above, you woulddo this:
.SUFFIXES: .o .c
.c.o:
cc -O -Wall $<
You may note that in the case ofsuffix rules, the dependency suffixgoes before the target suffix, which isa reversal from the normal order in amakefile. You also will see that youuse $< in the command, which evalu-ates to the .c fi lename associated withthe .o fi le that triggered the rule.There are a couple predefined vari-ables l ike this that are used exclusivelyfor suffix rules:
� $< — evaluates to the componentthat is being used to make the target—that is, fi le.c.
� $* — evaluates to the fi lenamepart (without any suffix) of thecomponent that is being used tomake the target—that is, fi le.
Note that the $? variable cannotoccur in suffix rules, but the $@ variablestil l wil l work.
Command Special CharactersCertain characters can be used in
conjunction with commands to alterthe behavior of make or the com-mand. If you’re familiar with shellscripting, you’l l recognize that \ sig-nifies a l ine continuation. That is tosay, using \ means that the commandisn’t finished and continues on thenext l ine. Nobody likes looking at amessy fi le, and using this character at the end of a l ine helps keep yourmakefile clean and pretty. If a rule has more than one command, use asemicolon to separate commands. Youcan start a command with a hyphen,and make will ignore any errors thatoccur from the command. If you wantto suppress the output of a commandduring execution, start the commandwith an at sign (@).Using these symbols will allow you
to make a more usable and readablemakefile.
DirectivesSometimes, you need more controlover how the makefile is read andexecuted. Directives are designedexactly for that purpose.From defining, overriding or export-
ing variables to importing othermakefiles, these directives are whatmake a more robust makefile possible.The most useful of the directives arethe conditional directives though.Conditional directives allow you to
define multiple versions of a commandbased on preexisting conditions. For
82 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE man make: a Primer on the Make Utility
example, say you have a set oflibraries you want included in yourbinary only if the compiler used is gcc:
libs_for_gcc = -lgnu
normal_libs =
foo: $(objects)
ifeq ($(CC),gcc)
$(CC) -o foo $(objects) $(libs_for_gcc)
else
$(CC) -o foo $(objects) $(normal_libs)
endif
In this example, you use ifeq tocheck if CC equals gcc and if it does,use the gcc libraries; otherwise, usethe generic libraries.This is just a small, basic sampling
of the things you can do with makeand makefiles. There are so many morecomplex and interesting things youcan do, you just have to dig aroundto find them!�
Adrian Hannah has spent the past 15 years bashing keyboards tomake computers do what he tells them. He currently is workingas a system administrator for the federal government. He is a jackof all trades and a master of none. He spends all his waking hourson the Linux Journal IRC channel, on Twitter (@codemoney2841)and talking to random chat bots on the Internet.
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 83
Resources
GNU make comes with most Linux distributions by default,but it can be found on the main GNU FTP server:ftp.gnu.org/gnu/make (via HTTP) andftp.gnu.org/gnu/make (via FTP). It also can be found on the GNU mirrors at www.gnu.org/prep/ftp.html.
Documentation for make is available on-line atwww.gnu.org/software/make/manual, as is documentation for most GNU software. You also can find more information about make by running info makeor man make, or by looking at /usr/doc/make/,/usr/local/doc/make/ or similar directories on your system.A brief summary is available by running make --help.
Conditional directives allow you to definemultiple versions of a command based onpreexisting conditions.
84 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
Q t4 Designer is one of the mostpowerful GUI builders on themarket. If you look at the many
Qt4 Designer tutorials, they are all verycode-heavy, which really puts off newprogrammers. After some searching, Iput together a few simple methods thatmake GUI development easy, even fornovice C++ programmers, and forexperts, it will speed up development.This article shows how to use a GUI
builder based on Qt4 Designer andEclipse. Unlike other tutorials, this one is not code-heavy; instead, it describesseveral simple methods that will enableyou to develop a GUI in a visual mannerwith a minimum of C++ code. If you arenew to C++ and GUI building, this tuto-rial will make it possible for you to builda GUI. If you are an expert, these meth-ods at least will help speed your devel-
opment. The development tools mustrun under the KDE desktop, althoughthe final program can be run on aGNOME desktop as well.GUI interfaces are a key part of most
operating systems, and with Linux, weare spoiled for choice. So many goodtools are available; see the WikipediaIDE comparison in the Resources sectionof this article for an excellent summary. I teach some 300 first- and second-yearengineers about Linux programmingincluding GUI programming. They needto use C++, and the options tend to benarrow. The two IDEs that look mostapplicable are Code Blocks and the combination of Eclipse CDT and Qt4Designer. Qt4 Designer clearly is themost powerful GUI builder with the bestdocumentation, and it’s free if you arecreating GPL or LGPL code.
Designer
You don’t have to be a code guru to develop GUIs. PJ RADCLIFFE
and
EclipseQt4
Installing the SoftwareThe simplest option is to obtain a liveDVD with all the necessary tools installed.See interestingbytes.wordpress.com fora Mint-based live DVD with these toolsplus many other development tools.To install the software yourself, work
through the following steps:First, ensure you have the gcc compiler
toolchain installed. From a commandline, type g++ --version. If this fails,use your package manager to install gcc,g++ and libstdc.Next, you need the GUI builder Qt4
Designer and the documentation systemQt4 Assistant. You can find them in yourLinux repository and install them withyour package manager. In the unlikelyevent that Qt4 is not available, try theNokia Qt download site (see Resources).You also need Eclipse with CDT
(C/C++ Development Tools) installed.There are several options here:
� Install Eclipse CDT from your packagemanager (easiest option).
� If you already have Eclipse installedbut not the C/C++ developmenttools, start Eclipse and select themenu option Help→ Install Newsoftware. Select your main repositorysite and filter on “CDT”. Select theCDT package and install.
� The latest version of Eclipse at thetime of this writing is Helios, which
may not be in your distribution’srepository. It has a nice all-in-onepackage of Linux Tools that includescode coverage and profiling. TheEclipse Helios site has a downloadand installation instructions (seeResources). Note the wiki link fromthat page, which has some veryuseful tutorials.
Finally, you need the Qt4 integrationplugins for Eclipse. The Nokia Website has instructions and the download(see Resources).
Creating a GUI ProjectMost Qt4 tutorials show how to writecode to build a form. The combinationof Qt4 Designer and Eclipse meansyou can do this in a purely visual manner, which is faster and easier. To build your own GUI, work throughthe following steps:
� Start Eclipse using the Linux Menuoption Development→Eclipse. If anyprojects are open, select the ProjectExplorer in the left-hand panel,right-click on the project folder,then select Close Project.
� From the Eclipse menu, selectFile→New→Project.
� Find the Qt option, select the sub-option Qt GUI project, andthen click Next.
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 85
� On the next panel, provide a projectname (let’s use demo_1 for thisexample), and click Next (not Finish).
� The next panel will set up the classand filenames. You can change thename of the class where you placeyour code. The bottom list boxlabeled UI Type is the most importantfield. It determines the type of GUIyou will create. A Widget is a basicpanel with no special functions; aQdialog provides simple interactionwith a user, and a QmainWindow is apanel with menus and other features.For this example, selectQmainWindow, then click Next.
� This panel allows you to include extraQt modules—for example, a network
interface or SQL interface. For thisexample, the extra modules aren’tneeded, so just click Finish.
� You will be asked to accept Qt perspectives; click yes.
You now should have a GUI projectready to add visual components. To seethe appropriate files, as shown in Figure1, expand the project in the ProjectExplorer panel to the left and double-clickon demo_1.ui.The compile and execution of the GUI
you have just created may have a hiccup,especially the first time you run it. Youshould be able to select from the Eclipsemenu File→Save All, Project→BuildAll, and then Run→Run. If Run→Runis grayed-out, try the following:
86 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE Qt4 Designer and Eclipse
Figure 1.Empty MainPage
� If demo_1.cpp is not already in theeditor window, then in the ProjectExplorer panel on the left, double-click demo_1.cpp to get the code inthe editor window.
� Add a space anywhere, and thenfrom the Eclipse menu, selectFile→Save All.
� Select Project→Build All, thenRun→Run. If you are asked for adebugger configuration, choosegdb/mi.
� Your GUI now should appear. Closeit by clicking the close icon on thetop right.
Now, it’s time to add visual compo-nents to your bare form. On the left-hand panel, select the Qt tab to seethe visual components. Click and dragacross two Push Buttons, an LCDNumber, a Text Edit box, a HorizontalSlider and a Progress Bar to get a formsimilar to Figure 2.You can see the properties of any
visual component by clicking on a
component and then examining theproperty editor in the right-hand panel.The properties are shown as a hierarchywith the parent class first and then thechild classes. You may need to scrolldown to find the property you want tochange. In Figure 2, a button has beenselected and the property editor showsits properties. The property called “text”has been changed to MyButton, whichis not the name of the button, just themessage displayed. It’s worth spendingsome time looking at all the propertiesyou can alter. Try also right-clickingon visual components to see what youcan change. For example, right-clickon the text edit box and select ChangeHTML. Type in some text, click OK,and that text is now displayed in thetext edit box.To create a menu for your form,
click at the top left where it says “TypeHere”, and type “File”. Your form nowhas a menu option “File”.Click on this again, and add a sub-
option “Save”. After finishing thistutorial, it’s worth starting a new projectand playing with a range of visualcomponents, right-clicking on each and
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 87
Figure 2.VisualComponents
changing the properties.Qt Designer has the concept of signals
and slots. A class can emit a signalwhen it has done something—for exam-ple, when a button is clicked. A classcan accept a signal in a slot—for exam-ple, the LCD display has a slot to accepta new value. You can connect visualcomponents directly together usingsignals. From the Eclipse menu, selectQt Designer→Editor Mode→Signals andSlots. Click on the slider, drag the result-ing red line to the progress bar, andrelease the mouse. In the panel thatpops up, select sliderMoved in the leftbox and setValue in the right box, thenclick OK. It’s important to know this editmode, because it’s a quick way to dis-cover all the signals and slots a visualcomponent can support. To see whatsignals a button can generate, drag themouse from the button to the form and
release the mouse. A window now popsup that names all the signals a pushbutton can emit. Close this window bypressing cancel to delete the link. Now,try dragging from the form to the but-ton to see all the slots the button cansupport. Try doing the same to the LCDdisplay (see Resources for more detailson slots and signals).To run your GUI, first change the
editor mode back to the normal widgetview with Qt Designer→Editor Mode→Widgets. Then, compile and run usingthe Eclipse menu options File→Save All,Project→Build Project and Run→Run.If all goes well, you should get aresult l ike what’s shown in Figure 4.Try moving the slider, clicking on thebuttons (which do nothing), and thenclose the window.I don’t have space to cover layout
tools and spacers, which ensure thatyour layout expands and contracts as auser resizes the GUI window. See thelarge e-book from Blanchette andSummerfield, the short Layout Tutorial
88 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE Qt4 Designer and Eclipse
Figure 3. Wired Signal
Figure 4. First Run
and the Qt Overview listed in theResources section for more information.Qt Assistant, discussed later in thisarticle, also has an excellent tutorial.
Adding Your Own CodeOther Qt4 tutorials become code-heavyat this point and lose people who arenot experienced C++ programmers.Here, I explain how to link your codeto the GUI interface in a simple andpain-free manner with minimal C++.Adding code to a GUI project is not
like writing code for a command-lineprogram. Your code must be placed inthe class created for you, in the exampledemo_1.cpp, not a main() routine.You should alter only demo_1.cpp,demo_1.h or demo_1.ui. The other filesare generated automatically and will beoverwritten on the next build.Your code can be started only when
a GUI event happens and it is linked toyour code, or when a timer you havecreated times out. Furthermore, yourcode must do its job quickly and return.If your code delays, the entire applica-tion freezes and stops responding touser stimuli. These factors require you todesign your code differently from howyou would design a command-line pro-gram. When you write a GUI program,plan for all of your code to respond toGUI events and timers. The GUI packagehandles the rest for you.Most GUI systems have a library of
useful classes, and it’s very much worth
learning how to use those classes, asthey can save you a lot of time and pro-vide features you could not implementotherwise. The library with Qt Designeris particularly powerful and worthmastering (more on that later).Now, let’s add the links between your
own code and the visual components.Click the demo_1.h tab, and add thesingle #include line at the top of Listing1. Next, add the declaration for timer,the public slots section and functionsthat are called when signals are gener-ated. These slots are all of the formon_objectName_action(). The available“action” can be discovered from what
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 89
Listing 1. Adding Slots to demo_1.h
#include <QtGui>
public:
demo_1(QWidget *parent = 0);
~demo_1();
private:
Ui::demo_1Class ui;
//--- add the following lines.
QTimer timer ;
public slots:
void on_pushButton_clicked();
void on_pushButton_2_clicked();
void on_actionSave_triggered() ;
void timer_tick() ;
};
you did to create Figure 3. The first twoslots are called when the buttons areclicked, the next when the form’s menuitem Save is selected, and the final onewhen your own timer based on Qtimerticks. In total, you must add seven linesof code to the existing file.Now, you can link the timer to the
LCD display. In the demo_1.cpp file,modify the constructor to add the threelines of timer setup, as shown in Listing2. First, the timer is created, and thenthe timer tick signal is connected to
your own routinetimer_tick().Finally, the timer is set up to tick at1,000 milliseconds(one second).Listing 3 shows
the code to react tothe first push but-ton. You need toadd all the lines, asthis member func-tion is new. The firsttwo lines of codechange the text inthe button and thefont type. The nextfew lines change thecolor. This is wellbeyond what a sim-ple application willneed to do, but itshows a little moreof what is possible.
Note that the properties of any visualcomponent can be changed using the formatui.ComponentName->member_function().As you begin to write more complexapplications, you will need to find thesemember functions. The simplest way isto start a separate application called QtAssistant, by going to Development→Qt4 Assistant from the Linux menu. Asshown in Figure 5, click the Index taband enter the name of the visual element.A detailed description of the element willbe displayed, including all useful member
90 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE Qt4 Designer and Eclipse
Listing 2. Constructor with Timer Start
demo_1::demo_1(QWidget *parent)
: QMainWindow(parent)
{ ui.setupUi(this);
//--- add the following lines.
timer = new QTimer(this);
connect(timer, SIGNAL(timeout()), this, SLOT(timer_tick()));
timer->start(1000);
}
Listing 3. First Push Button Code
void demo_1::on_pushButton_clicked()
{ //--- comment out or remove whatever is not needed.
ui.pushButton->setText("New Name") ;
ui.pushButton->setFont(QFont("Courier",10,QFont::Bold));
QPalette palette;
palette.setColor(ui.pushButton->backgroundRole(), Qt::red);
palette.setColor(ui.pushButton->foregroundRole(),Qt::blue);
ui.pushButton->setPalette(palette);
}
functions. If you are serious about master-ing Qt Designer, it’s worth spending sometime working through the Qt Assistantdocuments. In particular, click theContents tab, and open the QtDesigner Manual, whichincludes a wealth of usefultutorials and examples.The second push button also
must have its member functioncreated (Listing 4). Again, thecode shown here is more com-plex than a simple applicationneeds to be, but it shows avery useful feature: calling acommand-line program.Let’s work through the
code. Note how the text boxon-screen is cleared. Theui.textEdit gives you accessto the visual component, andthe member function clear()
is called. QbyteArray is anadvanced C++ string typeobject. The QProcess objectenables you to run a command-line program. First, the workingdirectory is set, and then thecommand started. If it returnswithin 300 milliseconds, theresult is written into thetextEdit box; otherwise, thecommand is terminated, andthe default error message isprinted. Note that the returnedtext from the command line mayhave nonprinting characters,
such as line feeds. To solve this problem,look up the QbyteArray member functionsimplified() in Qt Assistant.To respond to the menu on the
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 91
Figure 5. Qt Assistant
Listing 4. Second Push Button
void demo_1::on_pushButton_2_clicked()
{ ui.textEdit->clear() ;
QByteArray command_line, work_dir("/tmp"),
result ;
command_line = "ls /home" ;
result="Nothing happened." ;
QProcess shell(this) ;
shell.setWorkingDirectory( work_dir) ;
shell.start(command_line) ;
if ( shell.waitForFinished(300)) //ms timeout
result = shell.readAllStandardOutput() ;
ui.textEdit->append(result) ;
}
window, add:
void demo_1::on_actionSave_triggered()
{ ui.textEdit->clear() ;
ui.textEdit->append("Menu item Save just triggered.") ;
}
To increment the LCD display everysecond, add:
void demo_1::timer_tick()
{ ui.lcdNumber->display( ui.lcdNumber->intValue() + 1) ;
}
Now, let’s run your program. Fromthe Eclipse menu, go to File→Save All,Project→Build Project and, finally,Run→Run. The LCD timer should beincrementing every second. Try clickingon the buttons and the form menu tomake sure they perform as expected.
What Next?What happens next is up to you. Ifyou have trouble creating the exampledescribed here, you can download all of itfrom www.pjradcliffe.wordpress.com,then click on the “Other UsefulResources” page. Place the new direc-tory inside the workspace directory(/home/user/workspace), then start Eclipse.Browsing through Qt Assistant is a good
way to discover the useful member func-tions of visual components and the otherpowerful classes, such as QbyteArray. QtAssistant has many tutorials and examples,and the Web is a great source of informa-
tion, as Qt has a vibrant user community.It won’t take you long to make those visualcomponents perform just as you wish.
ConclusionAfter reading this article, you should beable to create a GUI using Eclipse and Qt4Designer, mostly by visual manipulation ofvisual components plus a little C++ code.The example given here shows how to callyour own code when visual componentsare activated (as with a button click), howto start your own code with a timer tick,and how to call command-line programsand read back their responses. You can doa lot with just those functions.�
Dr PJ Radcliffe is a senior lecturer at RMIT University inAustralia. He once was an ardent Windows programmer, butthen he discovered Linux, which he now teaches along withthe control of hardware using Linux. If you are interested inthese topics, see www.pjradcliffe.wordpress.com.
92 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
FEATURE Qt4 Designer and Eclipse
Resources
Wikipedia Article on IDE Comparison: en.wikipedia.org/wiki/Comparison_of_integrated_development_environments.
Qt4 download site from Nokia: qt.nokia.com/downloads.
Helios Eclipse Linux tools download and install instructions:www.eclipse.org/linuxtools.
Qt4 integration for Eclipse: qt.nokia.com/developer/eclipse-integration.
Good designer information (see the “signals and slots” link in particular): doc.qt.nokia.com/4.6/designer-manual.html.
A 700-page reference for Qt Designer from Blanchette andSummerfield (definitely worth obtaining although many examplestake a code-heavy approach): search for “c-gui-programming-with-qt-4-2ndedition.pdf” to find a convenient upload source.
Layout basics: thelins.se/learnqt/2009/05/qt-layouts-the-basics.
Good overview of key Qt4 visual design features:web.mit.edu/qt-dynamic/www/qt4-designer.html.
94 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
GNU Awk (gawk) is one of thoseprograms that has been available“since forever”, which many peoplenever think about. But, it’s a standardand important part of just about everyGNU/Linux distribution. In fact, it hasbeen available since even before GCC!During the past year and a half or
so, gawk has undergone a quiet revo-lution, culminating in the release ofgawk 4.0. Although not yet released atthe time of this writing, work is inprogress and moving forward. By thetime you read this article, gawk 4.0will be a fact and not just a promisingcode base in the Git repository.
A Little HistoryThe awk language was developed byAl Aho, Peter J. Weinberger and BrianKernighan, then at Bell Labs (hence thename A.W.K.). It first was released in
1978 with V7 UNIX. It offered the pattern-action programming paradigm,powerful regular expression matching,associative arrays, conventional opera-tors and control structures, and a mod-est array of built-in numeric and stringfunctions. It was only minimally docu-mented. (So minimally, in fact, that Iremember being terribly confused afterreading the short paper on awk, anddeciding to avoid it!) Nonetheless, theUNIX world accepted it and used it;true UNIX wizards were comfortablewriting even large scripts in it.Circa 1985, the authors started
beefing up the language, adding user-defined functions, C-compatible opera-tor precedence, more built-in func-tions, dynamic regular expressions anda few other minor features. Moreimportant though, they then proceededto write a book about the new version
GNU Awk 4.0:Teaching an Old BirdSome New TricksWhat’s new and nifty in gawk 4.0, with a little history andbackground along the way. ARNOLD ROBBINS
INDEPTH
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 95
of awk (The AWK ProgrammingLanguage), which was published inlate 1987. This version became availableto the world with the UNIX System VRelease 3.2.I bought the book, figuring that now
was my chance to learn awk. It was(and remains) a great book. Having aninterest in programming languages andan interest in contributing to the worldat large, I decided to see whether theGNU project had a version of awk.Indeed, it did, but it implemented onlyold awk (and poorly, at that). Beingsingle at the time, I decided to getinvolved and see if I could work tomake gawk compatible with new awk.(And, thus, the course of historychanged, forever.)As early as 1988, the GNU developers
were corresponding with Brian Kernighanand other awk implementers to makesure that the awk semantics were con-sistent across implementations. System VRelease 4, in 1989, brought a few newfeatures for new awk (the -v option,the ENVIRON array, the tolower() andtoupper() built-in functions) and thefirst POSIX standard (circa 1992) intro-duced the CONVFMT variable.Starting in December 1993, Brian
Kernighan was able to release thecode to new awk; it continues to beavailable (see Resources) and seesminor bug fixes from time to time.
GNU AwkGNU Awk was first written around1986 by Jay Rubin and Paul Finlason,with some help from Richard Stallman.It barely implemented the original awklanguage, was buggy and not particu-larly fast. It worked by building a parsetree representation of the programand then recursively evaluating theparse tree for each input record.When I got involved in late 1987,
David Trueman already had volun-teered to upgrade it to new awk, andI joined the effort, contributing codefixes and doing serious work on thedocumentation. We worked togetheruntil around 1994, when I became thesole maintainer.Along the way, gawk acquired full
compliance with new awk, includingPOSIX, and it improved in code quality,speed and new features. Throughoutthe course of more than 20 years though,the basic design remained the same:build the parse tree and recursivelyevaluate it for each input record.In 2003, out of the blue, a gentle-
man named John Haque contactedme. He had rewritten the gawk inter-nals to use a byte-code interpreter andprovided an awk-level debugger forawk programs. This was a startlinginnovation. I worked with him to gethis version to the point where it wasstable and passed the test suite, but I
INDEPTH
96 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
did not integrate his changes, becausethey were major, and I wanted to under-stand them better.Bad move: John disappeared in early
2004, and the code languished, unused.Finally, in fall 2009, I got a volunteer(Stephen Davies) to start bringing thelast version of the byte-code gawk thatI had into the present. He and I hadthings working, pretty much, and I evenannounced a test release to the world.Again, out of the blue, John resur-
faced in early 2010 and joined theeffort to make the byte-code gawkviable. This moved things into highgear, and we made a lot of progress.As I write this, the byte-code versionhas been merged with my “new features” branch of the code. This isthe basis for gawk 4.0.If you don’t yet have gawk 4.0, see
Resources for information on where todownload the source and how to buildit; building from source is very easy.
New Stuff in gawk 4.0With all the background out of the way,let’s look at the cool stuff. Due to spaceconsiderations, this is just a quick tour;
see the documentation (listed inResources) for details.
New InternalsThe most significant new feature is thatthe gawk internals have been completelyredone. The parser now builds a linkedlist of “instructions”. Each instructioncontains a code indicating what it is anda few members with needed informa-tion, such as the next instruction andwhich instruction to jump to if a jump isneeded. This list then is interpreted foreach record by a big switch statementrunning inside a for loop that traversesthe list. Data for operations are pushedand popped off a runtime stack.This implementation performs no
worse than the original recursive evalu-ator, and in many cases, it performsbetter. But what’s really cool is thatJohn added an awk-level debugger!Since 1978 when awk was first intro-
duced into the world, the only debug-ging tool was the print statement.Now, gawk has a full debugger, withbreakpoints, watchpoints, stepping bystatement or instruction, the ability tostep into and out of functions, and
INDEPTH
The most significant new feature is that the gawk internals have beencompletely redone.
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 97
many other features.The debugger is a separately compiled
program named dgawk. It is a l ine-oriented debugger modeled after GDB(the GNU Debugger). If you’re familiarwith GDB, it will be very easy to learnthe gawk debugger. In addition, thedebugger is fully documented in thegawk.texi file in the gawk distribution.
New Language-Level FeaturesAt the language level, there are severalnew features.1. gawk now provides a built-in file
inclusion mechanism. Lines that beginwith @include and have a filename indouble quotes cause gawk to includethat file, using the same path search-ing mechanism as the -f option.Nested includes are supported, andgawk will not include the same filetwice. This effectively obsoletes theigawk script that has come with gawkfor many years.2. New patterns named BEGINFILE
and ENDFILE provide “hooks” intogawk’s automatic “read a record andprocess it” loop. The action forBEGINFILE is called before the firstrecord is read from each input file.Normally, when a file cannot beopened, gawk exits with a fatal error(such as if you provide a directory onthe command line). When a programhas a BEGINFILE pattern, instead,
gawk sets the ERRNO variable to a stringindicating the problem, so that you cantell if the file is problematic. If it is,use the nextfile keyword to just skipit. ENDFILE actions let you do easyper-file cleanup actions.3. You now can call a function
indirectly. By setting a variable to thename of the function you wish to calland using special syntax, gawk will“indirect” through the variable andcall the desired function:
function f1(a, b) { .... }
function f2(c, d) { .... }
{ fun = "f1"; @fun(2, 3) # calls f1()
fun = "f2"; @fun(4, 5) } # calls f2()
4. gawk now sports true multidi-mensional arrays! Regular awk simu-lates multidimensional arrays (a[x, y])using string concatenation of theindex values. gawk now providesmultidimensional arrays (a[x][y]) but doesnot require that arrays be rectangular(as in C or other compiled languages).Code like this is valid:
a[1] = 1
a[2][1] = 21
It is up to the programmer to trackthe type stored at any given index:scalar or array. Subarrays can be passed
INDEPTH
98 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
to functions, as long as the functionknows what to expect.5. The switch/case statement is
enabled by default. gawk has hadswitch/case for a long time, but it hadto be enabled at build time, and thedefault was not to do so; now it’senabled automatically.6. gawk now supports defining fields
based on field content, instead ofbased on the separators betweenfields. A new variable, FPAT, is used.When you assign a string containing aregular expression to FPAT, gawkbegins splitting fields such that eachfield is the text that matched FPAT.(Normal field splitting is based on thetext in between fields matching theregular expression in FS.) This is usefulfor many kinds of data where FS-basedmatching just doesn’t work.The new patsplit() built-in function
provides access to this functionality forstrings besides the input record. It isthe analogue of awk’s regular split()function. Additionally, patsplit() letsyou capture the text of the separatorsbetween fields.7. Standard awk provides only one-
way pipelines, either to or from anotherprocess. gawk provides a notation foropening a two-way pipeline to a co-process. gawk uses the same notationwith special, internally recognized file-names, to provide TCP/IP communication
over sockets. This feature has beenavailable for a long time.gawk 4.0 enhances the networking by
providing explicit filenames to indicateIPv4 or IPv6 connections. Filenames areof the form /inet4/protocol/local-portt/remote-host/remote-port or /inet6/protocol/local-port/remote-host/remote-port. Plain/inet/protocol/local-port/remote-host/remote-port is what gawk supplied upto now and continues to be supported:it now means “use the system default”.Most likely, this will continue to be IPv4for many years.8. gawk now provides a short (sin-
gle-letter) option for every long optionthat it has. This finally makes it possibleto use almost every feature from a !#script. It does somewhat bloat themanual page. (gawk has too manyoptions, but that’s a different problem;nonetheless, I did remove a few redun-dant long options.)9. Interval expressions now are available
by default. An interval expression is anenhanced regular expression syntax,such as (foo|bar){2,4}, which matchesanywhere from two to four occurrencesof either foo or bar. The part betweenthe curly braces is the interval expres-sion. POSIX added them to awk manyyears ago for compatibility with egrep’sregular expressions. But most awksdidn’t implement them. For historicalcompatibility, gawk’s default was to
INDEPTH
disable them, unless running in POSIXmode. Today, compatibility with POSIXhas gained enough importance forenough users that interval expressionsnow are available by default.10. Finally, for this release, the code
has been reviewed and cleaned up.gawk now requires a full C 89 environ-ment to compile and run. It will notwork with K&R compilers or if _ _STDC_ _is defined but less than 1. The code formany obsolete and unsupported systemshas been removed completely. This slightlydecreases the size of the distribution,
but mainly it reduces useless clutter inthe source. The documentation also hasbeen reviewed and cleaned up.
Source Code ManagementFor many years, I was the only one withaccess to gawk source while it was beingworked on. Circa 2006, I made both thestable and development versions availablevia CVS from savannah.gnu.org. Thiswas a good move; it gave the usercommunity access to all my bug fixesand to my development code base.In late 2010, I moved to git. I am
INDEPTH
SUBSCRIBE TODAY!WWW.LINUXJOURNAL.COM/SUBSCRIBE
100 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
expecting greater productivity fromusing git and better ease of use forthe user community. And, it’s nice tobe using 21st-century tools.
Future WorkSome further interesting developmentremains to be done.1. The XMLGawk Project (see
Resources) is a fork of gawk based on3.1.6 that provides better facilities forloading dynamic extensions and severalvery interesting extensions to go withthose features. These should be mergedinto the main gawk code base anddistribution, respectively.2. Although gawk has had the ability
to load extensions dynamically for manyyears, the API has not been stable oreasy to use. I have designed an API forC functions that can be called from anawk program that is considerably better,but I have not implemented it yet. Thisshould be done.3. Currently, the gawk distribution
builds three separate executables: regulargawk, pgawk (for profiling awk programs)and dgawk for debugging them. Thenew internals enable the possibility ofmaking just one executable that couldperform all three functions (based oncommand-line options). This should sim-plify the build process and definitely willreduce the total installation “footprint”.4. The documentation could use further
cleanup. Some of the examples causethe documentation to show its age. (Whouses dial-up BBS systems anymore?)
AcknowledgementsThanks to Brian Kernighan, StephenDavies and John Haque for reviewingthis article.�
Arnold Robbins is a programmer, technical author, husband andfather. A native of Atlanta, Georgia, he and his family have beenliving in Israel since 1997, where he now works writing softwarefor a very large semiconductor manufacturing company. He hasbeen involved with GNU Awk since 1987(!). In his non-copiousspare time, he maintains gawk and its documentation, amongother activities. Arnold is also the author or co-author of a numberof UNIX- and Linux-related books from O’Reilly and PrenticeHall, which he hopes that all readers of this article will now runout and buy. For more information, see www.skeeve.com.
INDEPTH
Resources
Gawk Home Page at the FSF:www.gnu.org/software/gawk
Gawk Project Home Page at Savannah, with Links and Instructions for Using Git: savannah.gnu.org/projects/gawk
Gawk Download Directory: ftp.gnu.org/gnu/gawk
Gawk Documentation:www.gnu.org/software/gawk/manual
Installation Instructions:www.gnu.org/software/gawk/manual/html_node/Installation.html#Installation
Brian Kernighan’s “one true awk”:www.cs.princeton.edu/~bwk/btl.mirror
The XMLGawk Download Page: sourceforge.net/projects/xmlgawk
BBBBBBBBBBBBBBBBBBBBROROROROROROROOROROROROROOOOOROOUGUGUGUGUGUGUGUGUGUGGHTHTHTHTHTHTHTTHTHTTHTHTHHHTHTHTHTHTT T T T TT TTTTT TT T TTT TTTOOOO OO OO O OOOOOO OOOOOO YOYOYOYOYOYOYOYOYOYOYOYOOYOYOYOYOYY U U UU U U U U UUUUUUUUU BYBYBYBYBYBYBYBYBYBYYBYYBYBYBBY
GGGGGGGGGGGGGGGGGGUEUEUEUEUEUEUEEEUEUEEUEUEUUESTSTSTSTSTSTSTSTSTSTSTTSTTSSSSSSSSSSS EEEEEEEEEEEEEEEEEEEEEEEEEEXHXHXHXHXHXHXHXHXHXHXHHXHXHHXXHHHHIBIBIBBIBIBIBBBBBBBIBBBBBBBIBIBBBBBBIBBITITTITITITITITITITTTTITTTITITTITITTTTI ORORORORORORORORORORORORORORORROROOROOROROOOOOOROO SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS LLLLLLLLLLLLLLLLLLLEAEAEAEAEAEAEAEAEAEAEAEAAEAEAEAEAEAEAAAEAEAE RNRNRNRNRNRNRNRNRRNRNRNRNRNRRNRNNRNRNRRRRNRRNNINININNININNININININNNNINININNNINNNNNNNNI GGGGGGGGGGGGGGGGGGGGGGGSESESESESESESESESESESESESESSEESEESEESESESESESS SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSIOOIOIOIOIOIOIOIOIOIIOOIOIOIOIIIIOOOOONSNSNSNSNSNSNSNSNSSSNSNSNSNSNSNSNSNSNSNNSSSNN
LLLLLLLLLLLLLLLLLLLLLLLLLIIGIGIGIGIGGIGIGGIGGIGIGIGIGIGGGGGIGGGIGGGGIGGGHTHTHTHTHTHTHTHTHTHTHTHTHTHTHTTHTHTHTHTHTHTHTHTNININININININININNININNINININININNIINNININNN NGNGNGNGNGNGNGNGNGNGNNGNGNGNGNGNGGNGGNGNGNGGNGNNGGGNGGTATATATATATATATATATATATAATATAATAATATATTAATATT LKLKLKLKLKLKLKLKLKLKKLKLKLLKLKLKLKLKLKLLLKLKLKKLKKSSSSSSSSSSSSSSSSSSSSSSSS
MMMMMMMMMMMMMMMMININININININININININNNNINND-D-D-D-D-D-D-D-D-D-DDD-DD-D-BLBLBLBLBLBLBLBLBLBLBLBLBLLBBLLBLBBB OWOWOWOWOWOWOWOWOWOWOWWOOOWOWOWOOWOWWOWWINININININININININNIINNNININI GGGGGGGGGGGGGGGG
SSSSKEKEKEKEKEKEKEKEKEKEKEKKEKKKKEKEKKEKEKEKEKKKKKEK YNYNYNYNYNYNYNYYNYNYNYYYNYNYNYYYNOTOTOTOTOTOTOTOTOTOTOTOOTOTOTOTOTOTOTOTOTOOTOTOTOTTESESESESESESESESEEESESESESESESESESSSSSESESEE
TTTTTTTTTTTTTTTTTTTTTTEXEEXEXEXEXXEXXEXEXEXXXXEEEEEEXEXEXEXEEEEXEXEEXEXASASASASASASASASASASASASASASAASASAS--------STSTSTSTSTSTSTSTSTSTSTSTSTSSTTSTSTSSSSTSTTSTTYLYLYYLYLYLYLYLYLLYLYLYLYLYYLYLLLY EEEEEEEEEEEEEEEE
BABABBABABABABABABBABABABABABABABABABBBBAB SHSHSHSHSHSHSHSHSHSHSHSHSHSHSHHHHSHSS ESESESESESESESESESESESESESEESESESSEEEEES
Learn strategies and tactics to automate your operations, free up resources, and conquer the opposition!
Rapid application development(RAD) is a trendy data-processing idea toshorten application software develop-ment time. In the truest sense, RADshould include user-friendly develop-ment tools and seamless access betweenend users and actual development soft-ware. Small-business users lust forsoftware applications that fit this descrip-tion. Commercial tools, like MicrosoftAccess and AlphaSoftware, share greatsuccess in the business community.Both of these applications give small-
business users and IT departments asimple way to develop applications forinternal use quickly, with little effort.Usually, a small Web-based applicationcan be created in only a day or two.Rapid application development com-bined with a treasure trove of profes-sionally presented training videos andother learning resources keep theseproducts in the forefront of technologyand embedded in the business world.Thus, a product like AlphaSoftware
frequently is the cornerstone tool forsmall-business users to deploy Web toolsfor on-the-fly needs. Although some ofthese homespun software applicationsare downright ugly (from an aestheticand design standpoint), nonetheless,they are critical business tools. In fact,their importance is so profound thatsome small businesses will not evenconsider a Linux environment becausefew tools exist that closely match thefeature set of AlphaSoftware or similarcommercial products.Certainly, good arguments exist for
the robust programming tools Linuxoffers. Ruby gains much attention asa powerhouse tool, although Ruby isa “foreign language” to most small-business owners. So, traditional open-source tools seem to lack the absolutesimplicity necessary for acceptance bymany business users. Further, Linuxcoding tools are largely unfamiliar,which reduces the pool of people whoare able to assist businesses develop
102 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
INDEPTH
WaveMaker: It’sLike...RAD!Database stuck in Windows? Take RADical action and tryWaveMaker Community edition. DONALD EMMACK
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 103
new applications. Alternatively, considerhow easy it is to find a local IT guy whocan create an Access database with fullform control in only a few days!Now, WaveMaker provides a strong
Linux contender, properly positioned tobattle for the interest of the businessuser. WaveMaker clearly targets a useraudience with quick Web applicationdevelopment with a minimal learningcurve. The product is available in both acommunity edition and an enhanced betaversion poised for more commercial situa-tions (wavemaker.com/downloads).
Meet WaveMakerFrom a marketing perspective, WaveMakeris a robust tool for rapid applicationdevelopment of Web-based applications.You can peruse the WaveMaker site(www.wavemaker.com) for completedetails of its promotional position.From a small-business viewpoint,WaveMaker meets the needs andrequirements for rapid applicationdevelopment, ease of use and theability to work across multiple plat-forms—including Linux.The two editions of WaveMaker
appear separated by the need to integrate with mainline applications.WaveMaker’s community edition is limit-ed to single-instance use, while the fulledition enables unlimited users andinteraction with commercial databases
like Oracle, IBM DB2 or Microsoft SQLServer. Conversely, the community version supports the use of MySQL,PostgreSQL or HSQLDB. These twoalternatives give small business usersenough options for unique operatingenvironments.
The ChallengeWaveMaker seems well suited to com-pete for Linux business-user interest.A tough sell, most business ownersneed a real-l ife example before theyconsider open-source systems. Thus,this article intends to demonstratehow WaveMaker could migrate a legacy database from the proprietarysoftware world into the communityedition of WaveMaker. This simple test is only a baseline, but it depictsWaveMaker’s potential. In short, Ishow how WaveMaker can measure up to the tall demands of the businessworld and also re-affirm Linux as aviable choice for daily operations.The example test here includes a
sample business database originallydeveloped with AlphaSoftware. Theparameters of this test limit the environ-ment to a small local Web applicationwith only one table. It also demonstratesdata export in a common format (comma-separated or Excel) file and MySQL dataimport. Plus, MySQL becomes the platformfor the new application development.
INDEPTH
104 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
This process looks for potential issuesthat may arise when users cull datafrom a commercial application.In summary, the test steps are as follows:
1. Prepare the data in a common format.
2. Install WaveMaker community edition.
3. Use WaveMaker to auto-create a Webform for data entry and update.
Server and WaveMaker PreparationFor novice users, WaveMaker’s Web sitefully describes how to prepare youroperating environment to support thetool. The one key decision point iswhether to use the default HSQLDBdatabase. As part of installation,WaveMaker includes the componentsnecessary to use HSQLDB. In this test,let’s use the popular MySQL as thedatabase engine for the application.First, let’s use a Debian distribution
to install MySQL. Let’s also usephpMyAdmin to assist with databasecreation and data manipulation. If youwant to follow along, check your distri-bution’s instructions to install MySQLand phpMyAdmin, and make sure theyare running before you continue. Next,download WaveMaker communityedition and follow the installation instruc-tions here: dev.wavemaker.com/wiki/bin/wmdoc_6.3/Install.
Preparation for the TestAfter installation, let’s plan a simpledata export process to move into thenew environment. My research discov-ered that AlphaSoftware includes a niftyexport routine to send the data from itsown database to other formats. I testedthe export process with both text andMicrosoft Excel formats successfully. Toassist WaveMaker with field generation,I included field names during export.With the sample data in Excel format,use phpMyAdmin through the browserat http://localhost/phpmyadmin (or similar)to manage the MySQL experience.To minimize errors with data import,
a database and table were created inadvance. In Figure 1, a new databasecalled “linuxjournal” with a tablenamed “sheet” already is in place.Next, let’s use the import tool to pop-ulate the new “linuxjournal” database.Afterward, examine the database’sstructure to make sure the primary keyvalue and index are appropriate forthe test environment (Figure 2). In this example, I avoided foreign keys to minimize opportunities for errors.WaveMaker includes functions fordatabase creation; however, the inter-face is not as familiar as phpMyAdmin.Further, using WaveMaker to buildyour data structure is an advancedtopic. Therefore, the simpler approachis to stay with phpMyAdmin.
INDEPTH
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 105
WaveMaker LiveI used Linux Mint (a Debian distribution)for this test. After installing WaveMaker,the icon for startup did not appear onthe desktop or in the application menu.To work around this problem, navigateto /opt/{wavemaker sub-directory}/bin/.Then, execute ./wavemaker.sh, andobserve the new WaveMaker applicationpresented on your desktop as shown in
Figure 3. From here, press Start to beginthe process, and the default browser
INDEPTH
Figure 1. Oncelogged in tophpMyAdmin,create a testdatabase called“linuxjournal”,and import test data withfield names.
Figure 2. Verifythat the indexesand keys are correct for theimported data.
Figure 3. To the point, WaveMakerresponds with a minimal window allowingyou to start or stop the application.
106 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
opens to the application’s home screen(Figure 4).From this point, click Create a New
Project, give it a name, and thenWaveMaker drafts the workspace for theapplication within the browser (Figure5). This is the primary control centerfor development. While the look andfeel may seem unfamiliar, WaveMakerreduces training time through intuitivecommand labels and software compo-nent tree organization.
Establishing Database ConnectionOnce WaveMaker is up and running, itmust establish database connectivity tocontinue. Next, select Services and thenDatabase Services to define the connec-tion to “linuxjournal”. Now, WaveMakerneeds the right information to connect tothe database (Figure 6). With MySQL, thefirst step is to change the initial drop-down list from the default HSQLDB toMySQL. Next, input the database userid,password and exact database name inthe fields provided. Click Test Connectionat the bottom of the window to ensureconnection. Once successful, selectImport to continue the data importprocess, and WaveMaker responds withthe elements of the MySQL database ina new window (Figure 7). Stop here andevaluate your structure. It’s important to
INDEPTH
Figure 4. At first, WaveMaker gives you theopportunity to create projects, open existingones or just work through a tutorial.
Figure 5. A clean slate or “Palette” is thehome screen of WaveMaker.
Figure 6. WaveMaker’s Template toEstablish Database Connectivity
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 107
validate keys and indexes. Also note thefield names of your data to ensure theyare listed properly.
Application DesignWith a predominantly GUI interface,WaveMaker now is ready to create aWeb-based application. A pivotal benefitfor the small-business user, this interfacerelies on simple drag and drop for manyoperations. Automatic processes assistwith form creation, data placementand manipulation. Plus, error correctionis quite simple and capitalizes on thefamiliar Edit and Undo routine wheneveryou change your mind!First, cl ick on Canvas and Palette
as shown in Figure 8. The link to the“linuxjournal” database previously cre-ated now is available in WaveMaker’stree view. Now, let’s create a simpleentry and update application—in onestep. With the mouse, drag thedatabase onto the blank area of the
working screen. WaveMaker automagi-cally populates the working area ofthe Canvas with the sample data from“linuxjournal”. It’s remarkable to watchWaveMaker create editable (bold) fieldsfor the data on the bottom of the Canvas.
INDEPTH
Figure 7.Afterimport, yourdatabasestructure isdisplayed.
Figure 8.Navigatingto theLocation of YourImportedTable(Called“Sheet”)
108 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
Figure 9 shows the results.In essence, data from the MySQL
table is displayed on top, and a detailsection (with editable fields) is createdbelow. At the same instant, WaveMakercreates three buttons for data manipu-lation: New, Update and Delete (Figure10). The current view displays theskeleton of the application, completeand test-ready. Select Test from thetop window, and a new browser windowwith the application displays in testmode. Choose Run, and WaveMakersaves and then runs the application in a new browser window, as shown
in Figure 11.Now, work with this mini-program
to verify the success of the transition,exit the application and then re-visitphpMyAdmin to see the elements prop-erly added, changed and deleted.
Pinch YourselfWaveMaker just completed this testmigration in approximately 15–20minutes! Certainly more complex datamigrations will require a concertedeffort to ensure proper foreign keys andprecise index setup. Yet, the viabil ityof this principle is proven for a smallapplication migration.In this test, WaveMaker performs
quite well. Building the applicationand running it on the same machineis a bit sluggish, but this will changesubstantially with respect to hardwarespecifications and overall configuration.
INDEPTH
Figure 9. Afterdrag and drop,WaveMakercreates the listof data on thetop of thescreen, andthe editablefields are listedbelow. Red textwas added bythe author.
Figure 10. In the bottom right, WaveMakerautomatically provides New, Update andDelete buttons for the application.
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 109
No Parking!This test purposely excluded necessaryelements to make many applicationsreally suitable for business.Nevertheless, adding buttons, screensand tabs is very straightforward. Forexample, user login logos and multipletabbed displays often are staples ofeven the most rudimentary businessapplications. WaveMaker includesmany nicely packaged tools thatenable users to drag and drop addi-tional features immediately into thelive Palette.WaveMaker builds these other essen-
tial elements with a host of “widgets”along the application’s left panel(Figure 12). Users take advantage ofthis noncode environment and simplydrag and drop new features into thePalette. For example, a handy calendar(Figure 13) or simple page navigation
(Figure 14) require no direct code. Plus,login security and roles are handled
INDEPTH
Figure 12. A Sample of Widgets Includedwith WaveMaker
Figure 11.Live Operationof the SampleData Migrationin Less Than20 Minutes
110 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
efficiently in the Services and Securitytabs of the application—once again, asimple user interface.
The TakeawayFor business users, WaveMaker excitesthe potential of open-source possibil i-ties. In recent years, the strongholdsof proprietary operating systems usu-ally included finance and point ofsale. Because few comparable business-class applications exist, the impetus to move into the open-source world is sometimes difficult. It’s also clearthat mainstream business users mayconsider Linux when the core applica-tions of their business can be migratedwith ease.Often overlooked, small database
development served as a type of
commercial software lynchpin.Thus, business operators are oftenlocked in the proprietary realm.As a former AlphaSoftware user,I too see the impediments tochange. Plus, applications likeAlphaSoftware give developers astandardized tool to develop anddeploy executable software inaddition to Web applications. Thisis a market segment not directlymarketed by WaveMaker. As withmost software, familiarity of inter-face and technical support are allweighty factors in the decision.
WaveMaker seems l ike a bold stepand immediately gathers the atten-tion of business users. It is a truecompetitor to the commercial Webapplication development world anddeserves attention by the businesscommunity. VMware’s recentlyannounced acquisit ion of WaveMaker(dev.wavemaker.com/blog/2011/03/08/wavemaker-springs-to-vmware)gives this application a boost inrecognition and in business users’confidence in its future stability. Gogive WaveMaker a try!�
Don Emmack, a Change Management consultant, assistsgovernment and business users with infrastructure and opera-tional issues. A former Sr. Vice President for an internationalconsulting firm, his work includes domestic and internationalclientele. Don is an early adopter of new technology, yet remainshopelessly addicted to his 1980s-era fax machine!
INDEPTH
Figure 13. The Included Calendar Widget
Figure 14. Simple Creation of Data Navigation Buttons
jEdit is a cross-platform text editorwritten in Java. The current stable ver-sion at time of this writing is 4.3.2,and it’s available at jedit.org. Besidesthe cross-platform capabilities, jEditoffers other features, such as asophisticated plugin system, syntaxhighlighting for 130 languages, abuilt-in macro language and extensiveencoding support. I wrote this articleusing jEdit, and I demonstrate some ofits features here, especially some ofthe plugins I have found useful.Before I start, jEdit is a GUI text editor
of some heft. It is not a replacementfor using vi on the command line toedit a configuration fi le on a remoteserver. It does serve well in handlingmany fi les simultaneously with visualfeedback and with the benefits of a GUIinterface. To put it another way, I usevi or jEdit depending on the need.Installation is fairly easy; just go
to the download page and grab theinstaller jar. Be sure to check out the
compatibility link if you have a non-Sun(Oracle) or Apple version of Java. Frompersonal experience, I have not hadsuccess running jEdit on gcj. Assumingyou have a compatible version of Java,use the following to install:
java -jar jedit4.3.2install.jar
This launches an installer programthat guides you through the process.For the sake of reference, jEdit keepsits configuration files, on Linux anyway,in ~/.jedit/. I mention this because Ikeep that directory synced between mylaptop and my desktop machines. As a result, I have a consistent workingenvironment between the two.At its heart, jEdit is a just a text editor,
although it’s a text editor with a lot ofoptions. You can make these optionsglobal or apply them on a per-bufferbasis. You can reach the options via theUtilities menu item. The global optionsstick between editing sessions, but the
112 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
INDEPTH
jEdit: a Text Editorand MoreGetting started with this extensible GUI text editor.ADRIAN KLAVER
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 113
buffer options do not, unless you usethe buffer-local method. This consistsof embedding colon-separated hints tojEdit in the file. jEdit checks the firstor last ten lines for these hints. As anexample, to specify an indentation of2, use spaces for tabs and “hard” wrap,the embedded hints would be:
:identSize=2:noTabs=True:wrap=hard:
jEdit checks anywhere in those lines,so you can place the hints behindcomment symbols.Also note that jEdit supports mode-
specific settings, where a mode is afile type, such as Python (*.py), C (*.c),HTML (*.html) and so on. The variousmodes come with default settings, butthey can be overridden. One of the keybenefits is that the mode system pullsin file-type-specific syntax highlighting.Other options are available for theeditor’s layout. As you can see in thescreenshots for this article, I tend torun jEdit with two buffers open, splitvertically and with line-numberingenabled. The ability to look at the begin-ning and end of file at the same time,especially source code, is invaluable.You can use jEdit in a great number
of ways. Watching me enter text,although it has its moments, is notterribly inspiring. So to keep thingsinteresting, here I demonstrate some
of the plugins I have found useful.Plugins are code that scratches anitch. The base jEdit program does alot, but it does not cover the universethat is text editing, or other choresfor that matter.jEdit has a macro system (not cov-
ered in this article), so you can whipup your own solutions to problems orscope out the plugins available and notre-invent the wheel. So, before gettinginto the plugins themselves, here’s anoverlook at the plugin system itself.They can be found at plugins.jedit.orgor via the Plugins item on the menubar. Click on the Plugin Manager itemand then the Install tab for a l ist ofavailable plugins. Clicking on an itemshows a description at the bottom ofthe page. Check the box of any plug-in(s) you want to install, and then clickInstall. If the plugin has dependencies,they also will be installed.So, where to start with the plugins?
Let’s go from less-involved to more-involved, beginning with one suggestedto me by a member of the Bell inghamLinux User Group: WhiteSpace. It doeswhat it says—tracks whitespace. I haveit set up to show trailing whitespaceand, additionally, to eliminate anysuch whitespace when I save. I alsohave it show tabs and modify themaccording to my jEdit soft tab setting.This setting, when enabled, converts
INDEPTH
114 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
tabs into a defined number of spaces.WhiteSpace uses the setting to convert preexisting tabs into spaces or vice versa. All of the above helpswhen I work in Python code, keepingthat pesky whitespace in order. Thisalso is valuable when writing for Linux Journal, which requires thatwriters use spaces not tabs. See Figure 1 for WhiteSpace in action on a Python fi le. From the screenshot,you can see one way to set it up. Theother way is to go to Plugins→PluginOptions→WhiteSpace. This is howmost of the plugins work, althoughyou will find there often are differ-ences in options available between
the two locations.Another plugin I use quite often is
JDiff. As the name implies, it shows thediff between files. Of course, you coulduse the command line to do the samething. The benefit of the plugin is thegraphical presentation it provides.Figure 2 shows the dual-diff modeusing this article as the fi les. Fromhere, you can create a diff output. Youalso can walk through the diffs andapply them from one side to another.The JDiff plugin has a dockable
component that allows you to dril ldown into the lines of the fi les fordifferences (Figure 3). In the dual-diffscreenshot (Figure 2), you can see
INDEPTH
Figure 1.WhiteSpacePlugin Settingsand at Work
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 115
INDEPTH
Figure 2. JDiff Plugin inDual-Diff Mode
Figure 3. JDiff DockableShowing LineDifferences
116 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
another plugin at work, VoxSpell. Theunderlining is the spell-checker atwork. If you look at the file, you cansee that plugin/plugins go from beingunderlined to not underlined. I right-clicked on the words and added themto the dictionary as acceptable, atleast for the purposes of this article.Note that VoxSpell has a dependencyon the Spell Check plugin. It also usesquite a bit of memory, so that may bean issue.Next is a chicken-and-egg problem.
I ran across a reference to the SQL plug-in for jEdit. In the course of install ing
it, I found it had a dependency on theProject Viewer plugin, which meant Ihad to learn how to use ProjectViewer in order to use the SQL plugin.It turns out that was a good thing. Infact, this article was written usingProject Viewer. First, I wil l cover theSQL plugin and later expand on theProject Viewer plugin.SQL allows you to work with SQL
databases from within the editor.Setting things up to use the plugin is a two-step process. First, you needto do the general setup in the SQLoptions dialog. Go to the menu, then
INDEPTH
Figure 4. Adding SQL Server to a Project
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 117
Plugins→Plugins Options→SQL. Go to the JDBC page and use the AddElement button to indicate the path(s)to the JDBC drivers you want to makeknown to the program. At the time of this writing, SQL can work withOracle, MySQL, PostgreSQL, Firebird,DB2, Progress, MS SQL Server 2000,Sybase and Teradata, assuming youhave the requisite JDBC drivers. Thisprevious step “registers” the databaseso it can be used in the next step.The next step is to configure a spe-
cific database (or maybe more thanone) with a project. This is where
Project Viewer comes in. You use it tocreate the project (more detail on thatlater). For now, I will use the projectthat is this article. From the projectpane, right-click on the project nameand select properties. This will lead toa series of dialogs that allow you tofil l in the needed information (Figure4). In this case, I am using the Pagilademo database for Postgres. From ajEdit buffer, you now have access tothe database (Figure 5). There is quitea bit going on there, so let’s take it astep at a time.Just above the buffers is the SQL
INDEPTH
Figure 5. SQL Plugin Returning a Result Set
118 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
toolbar. First, above the left buffer is a Database: drop-down list with thepreviously configured database selected.To the right of that are four buttons:the first is Execute selection, the sec-ond is Execute buffer, the third isLoad object, and the last is Repeat lastquery. To the right of the last buttonis the Preprocessors drop-down. Forthis example, I am using the Variablesubstitution preprocessor. This can beseen in the “actor_id > ?” expressionin the SQL statement in the rightbuffer. (As a side note, notice the SQLsyntax highlighting prompted by the
use of the *.sql extension.) To continue,I have selected the statement I wantto run and then clicked the Executeselection button. Because I have vari-able substitution in force, an inputbox was presented (not shown) for meto enter the value for actor_id, in thiscase 35. The result is presented in aseparate window. From the result set,it is possible to save the data as CSV ortab format or as INSERT statements.You also can show/hide columns.Additionally, it’s possible to run multi-ple statements at once (Figure 6). Thisis a somewhat contrived example, but
INDEPTH
Figure 6. SQL Plugin Running Multiple Statements
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 119
it does show what’s possible. The previous feature allows me to createdatabase DDL fi les and run them fromwithin the editor.The last feature is SqlVFS (Sql
Virtual File System). This allows you to browse the selected database as a fi lesystem. To get there, go toFile→Open→Commands→Plugins→Show databases. Figure 7 shows whatyou get. Note that although Data says0 bytes, double-clicking on it gives aresult set from the table.Project Viewer is a plugin to make
handling a group of related fi les (a
project) easier. For demonstration purposes, I’m using the files that makeup this article. Project Viewer creates a docked button below the menu bar.Click it, and a drop-down appears withAll Projects listed. Click this, and a window opens. Right-click on AllProjects, and select Add project, andyou get another window (Figure 8) toenter the required information. ClickOK to create the project. Project Viewerthen takes you to that project and putsup a prompt about importing files intothe project. By default, it importseverything below the root directory.
INDEPTH
Figure 7. SqlVFS, Database as a Filesystem
120 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
INDEPTH
Figure 8.Setting Up a ProjectUsing ProjectViewer
Figure 9.Files inProjectViewer
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 121
At this point, the group of fi les isbound together as a project. The benefit is that Project Viewer keepstrack of their state and allows you toreturn to that state at a later time.Note that it’s possible to add otherdirectories/fi les to the project later.Simply right-click on the projectname, select Add fi les and navigate to the desired location(s). You can addnew files from the existing directory intwo ways: one is manual and theother automatic. The manual option isto right-click the project name andselect Re-import fi les. The automaticoption is to open the project name
context menu, select Properties→AutoReimport and enable it with a timeparameter. Figure 9 shows the visualindicators as to the status of fi les,where underlining represents openfiles and color indicates type of fi les.One really handy feature is the
Compact View of a project. This isenabled in the General Options of theProject Viewer plugin options. It pre-sents a flattened view of a directorystructure. This article does not reallyhave the directory depth to illustratethe benefit, so take a look at a screen-shot from another project (Figure 10).Each line takes you directly to a directory.
INDEPTH
Figure 10. Project Viewer Compact View
The benefit is that Project Viewer keepstrack of their state and allows you toreturn to that state at a later time.
Also of note is the Working Files tabin the project window. This groups allyour current open files together, whichis handy in a large project.Some other features include archiv-
ing the project fi les in a JAR fi le andsearching in the project or projectsubdirectory fi les for a string. Theability to consolidate all of the fi lesrelated to a project in a single inter-face is something I’ve come to appre-ciate even more as time passes.Walking through a Project Viewerdirectory tree renaming/moving/deleting
files while looking at the actual fi les is priceless.What I have presented above barely
scratches the surface of what is possiblewith jEdit. It has a macro facility that Ihave not even started to explore. Themost important part of jEdit, to me, isthat it lets me get work done withoutgetting in my way. Furthermore, itmakes that work easier, and I hopeyou find it useful also.�
Adrian Klaver works with computers, and when that proves frustrating, he pushes wheelbarrows of heavy stuff around toremind himself that maybe computers are not so bad after all.
INDEPTH
25TH LARGE INSTALLATION SYSTEM ADMINISTRATIONCONFERENCESPONSORED BY
IN COOPERATIONWITH LOPSA and SNIA
Save the Date!
December 4—9, 2011, Boston, MA
6 days of training on topics including:
• Virtualization
• Security
• Configuration Management
• And more!
Come to LISA ’11 for training and face time with experts in the sysadmin community.
The theme for LISA ’11 is ’’DevOps: New Challenges, Proven Values.”
• Invited Talks
• Paper Presentations
• Guru Is In Sessions• Practice and Experience Reports
• Vendor Exhibition• Workshops• Posters and WiPs
Plus a 3-day Technical Program:
LISA ’11 will feature:
Find out more at www.usenix.org/lisa11/lj
Perl has been around for more than20 years. During that time, it hasreceived its share of both praise andcriticism, and lots of misconceptionssurround it. Much of this stems fromlong-outdated notions of what Perlused to be, but have nothing to dowith what Perl actually is today.Perl hasn’t been standing still. It’s
been growing continuously and evolv-ing, and that growth has accelerateddramatical ly in the past few years.Moose is one of the technologies atthe heart of this “Perl Renaissance”,which also includes other excitingprojects that have emerged, such asCatalyst and DBIx::Class.Moose is essentially a language exten-
sion for Perl 5 that provides a modern,elegant, fully featured object system. Isay “language extension”, but Moose iswritten in pure Perl, and as you’ll see,its syntax is still normal Perl. You don’tneed to patch Perl itself to use Moose;under the hood, it’s just Perl 5.Because Moose is sti l l just Perl 5,
it’s fully compatible with all of thosewonderful modules on CPAN, regard-
less of whether they are written inMoose (and most aren’t, as CPAN hasbeen around for so long, and Mooseis relatively new).For me, this is still the single biggest
reason to choose Perl. Whatever youare trying to accomplish, chances are,there already is a refined module for iton CPAN. This usually means dramaticcuts in total development time, becausesomeone else already has written a lotof your program for you.And now, with all the modern
object-oriented features Moose bringsto Perl, you get to have your cake andeat it too.In this article, I provide an introduction
to object-oriented programming inMoose and cover some of Moose’score features with useful examples. Toget the most out of this article, youalready should be familiar with object-oriented programming concepts, suchas classes, objects, methods, attributes,construction and inheritance.You also need to know Perl—at least
the fundamentals. If you don’t knowPerl, learning it is not very hard to do.
124 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
INDEPTH
MooseWrite powerful object-oriented code in a modern andconsistent style—in Perl. HENRY VAN STYN
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 125
At the end of the day, it’s just syntax.The good news is you don’t need tomaster Perl by any stretch to startusing Moose.Perl does have its quirks, and Moose
doesn’t make them all totally go away(and you wouldn’t want them all togo away, because a lot of them arereally useful). The most importantconcepts to understand are how Perlreferences work (the “perlreftut”tutorial is a great place to start—seeResources), and also the basics ofworking with Scalars, Arrays andHashes. Also, learn what the fatcomma is (=>) if you aren’t alreadyfamiliar with it. Moose makes heavyuse of it as an idiom. It’s actually notthat scary; it’s interchangeable withthe normal comma (,).Most of the rest of it you can learn
as you go. Normal language stuff likeloops, conditionals and operators aren’tall that different in Perl than any otherlanguage. So give it a shot. I think you’llfind it’s well worth the investment.
Getting MooseChances are you already have a distri-bution of Perl installed on your system.You at least should have Perl 5.8, butpreferably 5.10 or 5.12. Installing Moosefrom CPAN is an easy task; simply runthe following command:
cpan Moose
This should download and installMoose for you, as well as all of Moose’sdependencies.
Object-Oriented Perl (the Old Way)Even though Perl has had object-orientedfeatures for a long time, it was notoriginally designed—syntactically—as anobject-oriented language. This is moreabout the API provided to the program-mer than it is about the underlyingtechnical design of Perl itself.Perl 5 provides a lean environment
with the fundamental features and hooksneeded for object-oriented programming,but then leaves most of the details (such
INDEPTH
What about Perl 6?A lot of the features in Moose were inspired by Perl 6. Perl 6 stil l is beingdeveloped actively, and I believe that when it’s finally released for productionuse, it won’t disappoint. The fact is Perl 5 is solid, proven and fast, so thereis no reason to rush Perl 6. It is better that the developers take the time todo it really right, which is exactly what they’re doing.
126 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
as setting up object constructors, imple-menting attributes and handling validation)to you. As a result, the “right way” to goabout implementing these concepts isopen to interpretation.The fundamental feature utilized by
Perl to support objects is the “blessed”reference. This is like the flux capacitorof objects in Perl. Blessing simply associ-ates a normal reference (usually a Hashreference) with a class. The blessedreference then becomes the “objectinstance”, and its referent is used as thecontainer to store the object’s data.The class name is the same thing as
the package name, which is nothingmore than the namespace in whichsubroutines and variables are defined.The subroutines defined in the givenpackage namespace become the methodsof the class and can be called on theobject reference.All object-oriented languages have
to do something along these lines toimplement objects under the hood.Other languages just don’t impose somany of the low-level details on theprogrammer as in pure Perl.Here is an example of a simple class
in old-school Perl 5 OO:
package MyApp::Rifle;
use strict;
sub new {
my ($class, %opts) = @_;
$opts{rounds} = 0 unless ($opts{rounds});
my $self = bless( {}, $class );
$self->rounds($opts{rounds});
return $self;
}
sub rounds {
my ($self, $rounds) = @_;
$self->{_rounds} = $rounds if (defined $rounds);
return $self->{_rounds};
}
sub fire {
my $self = shift;
die "out of ammo!" unless ($self->rounds > 0);
print "bang!\n";
$self->rounds( $self->rounds - 1 );
}
1;
Save the above class definition into afile named MyApp/Rifle.pm within oneof your Perl’s include directories, andthen you can use it in a Perl programlike this:
use MyApp::Rifle;
use strict;
my $rifle = MyApp::Rifle->new( rounds => 5 );
print "There are " . $rifle->rounds . " rounds in the rifle\n";
$rifle->fire;
print "Now there are " . $rifle->rounds . " rounds in the rifle\n";
INDEPTH
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 127
Moose SugarMoose is really nothing more than syn-tactic “sugar” that automatically takescare of the boiler-plate tedium andlow-level details of implementing objectsautomatically. This is possible because ofPerl’s powerful introspection capabili-ties—Moose dynamically manipulatesthe class definition at compile time justas if it had been written that way.The previous class could be implemented
like this with Moose:
package MyApp::Rifle;
use Moose;
has 'rounds' => ( is => 'rw', isa => 'Int', default => 0 );
sub fire {
my $self = shift;
die "out of ammo!" unless ($self->rounds > 0);
print "bang!\n";
$self->rounds( $self->rounds - 1 );
}
1;
Not only is this code much shorter,cleaner and easier to read, but it isdoing all the things the non-Mooseclass was doing and more. First, Mooseis automatically creating the “new”constructor method behind the scenes.It is also automatically setting up“rounds” as an attribute (aka object
variable), which Moose understands asa distinct concept.Pure Perl has no such understanding;
if you want “attributes”, you have toimplement them yourself by writingaccessor methods by hand and decidinghow they should work (the non-Mooseversion above illustrates only one ofmany possible approaches).Moose, on the other hand, provides a
refined, fully integrated object attributeparadigm. It sets up the accessormethods, handles the data storage andretrieval, and automatically configuresthe constructor to initialize attributesoptionally with supplied parameters—and that is just scratching the surface!One of the problems with the non-
Moose version of our class is that thereis no validation for “rounds”. For example,nothing stops us from doing this:
my $rifle = MyApp::Rifle->new( rounds => 'foo' );
This is one of the areas where Moosereally shines; it provides a complete Typesystem that is very straightforward touse. In the Moose version, the optionisa => 'Int' sets up the roundsattribute with a type constraint thatautomatically will throw an exception(with a meaningful message) if you tryto set the value to anything that is nota valid integer. This would stop youfrom setting rounds to 'foo' because it’s
INDEPTH
128 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
not an integer, it’s a string.This illustrates an important point
about Moose’s design and approach. Itssyntax is declarative rather than impera-tive. This means you just need to specifywhat you want it to do instead of howit needs to do it. This is in sharp contrastto the traditional Perl 5 OO style, wherethat is exactly what you would have todo—add additional lines of code in theaccessor method to test the value forvalidity and handle the result.The syntax isa => 'Int' doesn’t
provide any insight on how Moose willgo about checking and enforcing thetype constraint. And that’s the wholepoint—you don’t care. But, you canrest assured it will do it in a far morethorough and robust manner than anything you’d want to waste time on doing yourself.
More about AttributesYou declare attributes in Moose with the“has” function. This consists of a uniqueattribute name, followed by a l ist ofnamed options (key/values). Althoughit looks and behaves l ike a built-inlanguage keyword, it is really just afunction call. Its documented syntax isjust idiomatic for the purpose of codereadability (it’s a fancy way to pass anargument l ist).Moose provides all sorts of options
that define the behavior of a given
attribute, including setup of accessors,data types, initialization and eventhooks. The simplest attribute is just anobject variable that is set up as eitherread-write (rw) or read-only (ro) withthe “is” option:
has 'first_name' => ( is => 'rw' );
The is option tells Moose to set upthe accessor method, which is what youuse to get and set the attribute’s value.You set the value of an attribute bypassing a single argument to the acces-sor (such as $obj->first_name('Fred')),and you get the current value by call-ing the accessor with no arguments($obj->first_name). Setting the valueis only allowed if the attribute “is” =>“rw”. If it’s “ro”, and you try to set itsvalue through the accessor an exceptionwill be thrown.This is the core of the attribute
paradigm, but lots of other optionsprovide useful features. Here are afew of the noteworthy ones:
� is: ro or rw, creates either a read-onlyor read-write accessor method.
� isa: string representing an optionaltype constraint.
� default: the default value of theattribute.
INDEPTH
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 129
� builder: alternative to default—name of a method that will generatethe default.
� lazy: if true, the attribute is not initialized until it’s used.
� required: if true, attribute value mustbe provided to constructor or havedefault/builder.
The builder option lets you specify a method to use to populate theattribute’s default value. A builder is anormal method defined within the class,and its return value is used to set theattribute’s initial value. If the builderneeds to access other attributes withinthe object, the attribute must be lazy(to prevent it from potentially beingpopulated before the other attributesit depends on).Because this is such a common sce-
nario, for convenience, Moose providesthe “lazy_build” attribute option thatautomatically sets the lazy option andsets the builder to _build_name (such as_build_first_name for an attributenamed first_name). For example:
has 'first_name' => ( is => 'ro', lazy_build => 1 );
sub _build_first_name {
my $self = shift;
return $self->some_lookup('some data');
}
Attributes Containing ObjectsSo far, I’ve talked only about attributescontaining simple scalars. Attributescan contain other types of values aswell, including references and otherobjects. For example, you could add aDateTime attribute to your MyApp::Rifleclass to keep track of the last time“fire” was called:
package MyApp::Rifle;
use Moose;
use DateTime;
has 'rounds' => ( is => 'rw', isa => 'Int', default => 0 );
has 'fired_dt' => ( is => 'rw', isa => 'DateTime' );
sub fire {
my $self = shift;
die "out of ammo!" unless ($self->rounds > 0);
my $dt = DateTime->now( time_zone => 'local' );
$self->fired_dt($dt);
print "bang!\n";
print "fired at " . $self->fired_dt->datetime . "\n";
$self->rounds( $self->rounds - 1 );
}
1;
This is fairly straightforward. I’m cre-ating a new DateTime object and storingit in my fired_dt attribute. Then, I can
INDEPTH
130 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
call methods on this object, such asthe datetime method, which returns afriendly string representing the dateand time.
DelegationsAlternatively, you could utilize Moose’sdelegation feature when you set upthe fired_dt attribute, l ike this:
has 'fired_dt' => (
is => 'rw',
isa => 'DateTime',
handles => {
last_fired => 'datetime'
}
);
This will set up another accessormethod named last_fired and map it to the datetime method of theattribute. This makes the invocationsof $self->last_fired and $self->fired_dt->datetime equiva-lent. This is worthwhile because itallows you to keep your API simpler.
TypesMoose provides its own type system forenforcing constraints on the value towhich an attribute can be set. As Imentioned earlier, type constraints areset with the isa attribute option.Moose provides a built-in hierarchy of
named types for general-purpose use.
For example, Int is a subtype of Num,and Num is a subtype of Str. The value'foo' would pass Str but not Num orInt; 3.4 would pass Str and Num butnot Int, and 7 would pass all of Str,Num and Int.There also are certain built-in types
that can be “parameterized”, such asArrayRef (a reference to an array). Thislets you not only require an attribute tocontain an ArrayRef, but also set typeconstraints on the values that ArrayRefcan contain. For example, setting isa => 'ArrayRef[Int]' requiresan ArrayRef of Ints. These can benested multiple levels deep, such as'ArrayRef[HashRef[Str]]' and so on.Another special parameterized type
is Maybe, which allows a value to beundef. For example, 'Maybe[Num]'means the value is either undef or a Num.You also can use type “unions”. For
example, 'Bool | Ref' means eitherBool or Ref.If the built-in types aren’t sufficient
for your needs, you can define yourown subtypes to do any kind of custom validation you want. TheMoose::Util::TypeConstraints docu-mentation provides details on creatingsubtypes, as well as a complete l istingof the built-in types that are available(see Resources).Finally, instead of specifying the name
INDEPTH
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 131
of a defined type, you can specify a classname, which will require an object ofthat class type (such as in our DateTimeattribute example). All of these conceptscan be intermixed for maximum flexibility.So, for example, if you set isa =>'ArrayRef[MyApp::Rifle]', itwould require an ArrayRef ofMyApp::Rifle objects.
InheritanceSubclassing is relatively painless inMoose. Use the extends function tomake a class a subclass of another. Thesubclass inherits all the parent’s methodsand attributes, and then you can definenew ones or override existing onessimply by defining them again.Moose also provides helpful attribute
inheritance sugar that allows you toinherit an attribute from the parent, butoverride specific options in the subclass.To tell Moose to do this, prepend theattribute name with a plus sign (+) in a“has” declaration in the subclass. (Note:attribute options related to accessormethod names cannot be changed usingthis technique.)For example, you could create a new
class named MyApp::AutomaticRifle thatinherits from the MyApp::Rifle classfrom the previous example:
package MyApp::AutomaticRifle;
use Moose;
extends 'MyApp::Rifle';
has '+rounds' => ( default => 50 );
has 'last_burst_num' => ( is => 'rw', isa => 'Int' );
sub burst_fire {
my ($self, $num) = @_;
$self->last_burst_num($num);
for (my $i=0; $i<$num; $i++) {
$self->fire;
}
}
1;
Here, MyApp::AutomaticRifle can doeverything MyApp::Rifle can do, but italso can “burst_fire”. Also, the defaultof the rounds attribute has beenchanged to 50 in AutomaticRifle, butthe rest of the options for the roundsattribute still are inherited from theparent Rifle class.You might use MyApp::AutomaticRifle
like this:
use strict;
use MyApp::AutomaticRifle;
my $rifle = MyApp::AutomaticRifle->new;
print "There are " . $rifle->rounds . " rounds in the rifle\n";
$rifle->burst_fire(35);
print "Now there are " . $rifle->rounds . " rounds in the rifle\n";
INDEPTH
132 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
The BUILD MethodAlthough Moose automatically sets upthe “new” constructor for you, therestill are times when you need to exe-cute custom code at construction. Ifyou need to do that, define a methodnamed BUILD, and it will be calledimmediately after the object has beenconstructed. Don’t create a “new”method; that will interfere withMoose’s operation.BUILD is also special as it relates to
inheritance. Unlike normal methods thatoverride the parents’ methods whenredefined in subclasses, BUILD can bedefined in every class in the inheritancetree and every one will be called, inorder from parent to child.
RolesRoles define some set of behaviors(attributes and methods) without beingfull-blown classes themselves (capable ofinstantiation as objects directly). Instead,Roles are “composed” into other classes,applying the defined behaviors to thoseclasses. Roles are conceptually similar to“mixins” in Ruby.Roles also can require that consuming
classes have certain methods by callingthe “requires” sugar function in theRole definition (or throw an exception).You call the “with” sugar function
to consume a Role by name, just l ikeyou call “extends” to inherit from a
regular class.Here is an example of a simple Role
that could be composed into eitherMyApp::Rifle or MyApp::AutomaticRifle:
package MyApp::FireAll;
use strict;
use Moose::Role;
requires 'fire', 'rounds';
sub fire_all {
my $self = shift;
$self->fire while($self->rounds > 0);
}
1;
You would then add this single line toMyApp::Rifle or MyApp::AutomaticRifleto give either class the fire_all method:
with 'MyApp::FireAll';
In the case of MyApp::AutomaticRifle,the with statement must be called afterthe extends statement, because the“fire” and “rounds” methods don’t existwithin MyApp::AutomaticRifle beforethat, and the Role’s requires statementswould fail.If you add the Role to MyApp::Rifle, it
will be inherited by MyApp::AutomaticRifleautomatically, so there would be noneed to add it there also (although it
INDEPTH
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 133
won’t break anything if you do).
Method ModifiersMethod modifiers are one of the morepowerful and flexible features ofMoose. The most common types ofmodifiers are before, after andaround. Before and after are reallyjust “hooks” to execute some codewhenever a given method is called,either before or after, as the namesimply. For example, this would print astring every time fire_all is called:
before 'fire_all' => sub {
my $self = shift;
print "Say hello to my little friend!\n";
};
The “around” modifier is quite a bitmore powerful than before and afterbecause it actually can change thearguments passed to, and the datareturned from, the original method. It also can programmatically decidewhether even to call the originalmethod at all.Around modifiers actually replace the
original method, but get passed theoriginal method and arguments to beable to call it within the new modifierfunction, but unlike before and after,this has to be done manually in around.The basic blueprint of this is below, whichis an example of an around modifier
that exactly reproduces the originalmethod (having no observable effect):
around 'fire_all' => sub {
my ($orig, $self, @args) = @_;
return $self->$orig(@args);
};
In an around modifier, the firstargument is the method ($orig)instead of the object reference($self) l ike in normal methods. Then,it’s up to you to call the originalmethod ($self->$orig) and capture itsreturn value (or not) and then return.
Method modifiers make a great fitwith Roles to define behaviors at a
INDEPTH
NOTE: The semicolons at the end of
the method modifier definitions in the
examples are required. Like all the key-
words provided by Moose, the modifier
sugar keywords actually are function
calls and are not subroutine definitions.The modifier definitions are all just
function calls with exactly two argu-
ments: a string representing the name
of the method to modify and a codereference to the actual modifier.
CodeRefs are just treated syntactically
as values like any other. It’s not impor-
tant to understand this fully to use
method modifiers, but it is important to
remember to use the semicolons.
134 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
fine-grained level. Let’s take a look atanother example of a Role for ourMyApp::Rifle class that makes use ofmethod modifiers:
package MyApp::MightJam;
use Moose::Role;
use Moose::Util::TypeConstraints;
requires 'fire';
subtype 'Probability' => (
as 'Num',
where { $_ >= 0 && $_ <= 1 },
message { "$_ is not a number between 0 and 1" }
);
has 'jam_probability' => (
is => 'ro',
isa => 'Probability',
default => .01
);
sub roll_dice {
my $self = shift;
return 1 if ( rand(1) < $self->jam_probability );
return 0;
}
before 'fire' => sub {
my $self = shift;
die "Jammed!!!\n" if ($self->roll_dice);
};
1;
This Role adds the random chanceof “Jamming” on any given call to“fire” depending on the probabilityspecified in the jam_probabilityattribute (with the default probabilityset to 1%). I also i l lustrate here howto create a custom subtype, by defininga new type “Probability”, which mustbe a number between 0 and 1.You then could compose simple sub-
classes l ike the following:
package MyApp::CrappyRifle;
use strict;
use Moose;
extends 'MyApp::AutomaticRifle';
with 'MyApp::MightJam';
has '+jam_probability' => ( default => .5 );
1;
And:
package MyApp::NiceRifle;
use strict;
use Moose;
extends 'MyApp::AutomaticRifle';
with 'MyApp::MightJam';
has '+jam_probability' => ( default => .001 );
1;
The difference between these two is
INDEPTH
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 135
that CrappyRifle will jam on average 5out 10 times, and NiceRifle will onlyjam 1 per 1,000 times.
Learning MoreThis article is just meant as an introduc-tion to Moose, and because of spaceconstraints, I have been able to coveronly a few of its core features.One of the other great things about
Moose, and Perl in general, is the com-munity and availability of documenta-tion and resources. The Moose Manual,available on CPAN (see Resources), iswell-written and comprehensive. Thereare also plenty of other docs and infor-mation available, and the number ofthem is growing every day as Moosecontinues to gain popularity.If you get stuck on something and
can’t find the answer, try the #mooseIRC channel on irc.perl.org. Many of thetop experts are in this channel and aremore than willing to help and answerquestions. Although they will expect youto RTFM and have done your homeworkfirst, they will get you unstuck andpointed in the right direction.If nothing else, I hope that this arti-
cle has at least piqued your interestin modern development with Perl and
Moose, and that you can see that Perlcode can, in fact, be clean, easy toread and modern, while still being“Perlish” and powerful.As you learn Moose, and modern
Perl in general, be sure to check outsome of the other projects and modulesthat are available, including Catalyst,Template::Toolkit, DBIx::Class, Try::Tiny,Test::More and Devel::NYTProf, just toname a few. You might be surprisedwhat’s out there, and what is reallypossible with Perl today.�
Henry Van Styn is the founder of IntelliTree Solutions, anIT consulting and software development firm located inCincinnati, Ohio. Henry has been developing softwareand solutions for more than ten years, ranging fromsophisticated Web applications to low-level network andsystem utilities. He is the author of Strong Branch Linux,an in-house server distribution based on Gentoo. Henrycan be contacted at www.intellitree.com.
INDEPTH
Resources
Moose CPAN Page: search.cpan.org/perldoc?Moose
Moose Manual: search.cpan.org/perldoc?Moose::Manual
Moose::Util::TypeConstraints Documentation:search.cpan.org/perldoc?Moose::Util::TypeConstraints
Moose IRC Channel: #moose on irc.perl.org
perlreftut—Perl Reference Tutorial: perldoc.perl.org/perlreftut.html
If you get stuck on something and can’t find theanswer, try the #moose IRC channel on irc.perl.org
KYLE RANKIN
BILL CHILDERS
136 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
As much as I love working with Linuxand configuring software, one majorpart of being a sysadmin that always hasappealed to me is working with actualhardware. There’s something aboutworking with tangible, physical serversthat gives my job an extra dimensionand grounds it from what might other-wise be a completely abstract job evenfurther disconnected from reality. Ontop of all that, when you get a largeshipment of servers, and you view theservers at your company as your servers,there is a similar anticipation andexcitement when you open a serverbox as when you open Christmas presents at home.This story so happens to start during
the Christmas season. We had justreceived our first shipment of a com-pletely new blade infrastructure that wewere really excited to try out. As theresident server monkey and general min-ion working under Bill’s iron fist, I wasto meet up with an engineer from our
vendor at the data center and assistwith the installation in any way I could.It was a big job—two completely popu-lated blade chassis comprising 32 bladeservers, integrated SAN switches and allthe assorted power supplies and networkpass-throughs that went along with it.We budgeted a full day of the engineer’stime to rack the new chassis, slot theblades and make sure all hardware wasfunctional and up to date.
[Bil l: Iron fist? I l ike the sound ofthat. Reminds me of a mid-1970s MarvelSuperhero...but I digress. I rememberthis occasion. We’d just finished pilotinga VMware/Blade infrastructure at ourcorporate office, and we were about toroll it out to our production data center,on next-generation hardware. It was anexciting time!]I arrived at the data center a few
hours before the engineer so I couldget all the boxes from shipping andreceiving and move them into our cage.If you ever have ordered a blade chassis,
Unboxing DayIt’s Christmas time, and Kyle is about toopen his new present: a large shipmentof blade servers. Find out about theextra present his vendor left him.
TALES FROMTHE SERVER ROOM
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 137
you know that everything arrives inthese gigantic cardboard boxes thatincidentally were about the size of ourshared cubicle space back at the office.These boxes open up to smaller boxesfor the blade servers, chassis, powersupplies and the rest. At first thingsmoved smoothly. I broke down thefirst set of boxes, and after a numberof trips, the empty blade chassis andthe blades themselves were stackedneatly near our cage.
The Jack in the BoxIt wasn’t until I opened the last box thatI realized I was in trouble. Instead ofcontaining a dozen large boxes andmostly empty space, this box looked likea failed game of Tetris. It was filled tothe brim with hundreds of tiny boxes ofall shapes and sizes. The engineer wasgoing to be there soon, so I tried toorganize the boxes into different pilesand then fil led my pushcart with swaying stacks of tiny boxes and madetrip after trip to the colocation cageuntil all of its walls looked like theinside of a brown-brick house.
[Bill: I seem to remember you sent acouple e-mails to me along the lines of“Wow, this new stuff sure has a lot of
boxes compared to the old stuff.”]This is probably a good point in the
story to tell you that up to this time,we normally had taken advantage ofour vendor’s integration service. Westandardized on servers with a certainamount of RAM, CPU revision, storageand network configuration that deviat-ed from the base model, so our vendorwould take the base order model anddo the work to add CPUs, RAM and theextra parts we wanted so that when wegot a server, we could just rack it andturn it on.In this case, for some reason, we
failed to request this integration service,so not only was I looking at the boxesfor blades, chassis and power supplies,I had hard drives, CPUs, RAM, fiber-channel HBAs, extra NICs and evenbattery-backed write caches all individ-ually wrapped in their own boxes.Instead of unboxing a blade and slidingit into its slot to install it, every singleblade would need to be opened, andthen each and every component wouldhave to be opened, removed from itsstatic wrap, and installed into theblade one by one.
[Bil l: I have to say here that up until this point, the vendor always had
TALES FROM THE SERVER ROOM
Instead of containing a dozen large boxes and mostlyempty space, this box looked like a failed game of Tetris.
138 | SEPTEMBER 2011 WWW.LINUXJOURNAL.COM
“thrown in” the integration service forus, and it’s something we (and by we, Imean me) had taken for granted...untilthe day my boss called the vendor anddeleted this “superfluous service” fromthe quote, without telling me.]
Drowning in CardboardWhen the engineer arrived, I explainedthe situation, and we both realized wehad a long day ahead of us. At thebeginning, we made great progress.He opened up and racked the chassisand power supplies until the point thatwe were ready to install the first bladeserver. At that point, we agreed on anassembly-line system where he wouldopen up a blade, and I, like a surgicalassistant, would unwrap and hand himeach component in a certain order so hecould install it. Then, while he finishedup the blade, I would fil l up the cartwith empty wrappers and boxes androll it to the trash area, so we didn’tdrown in anti-static wrap and card-board. After a full-day’s work, wewere able to integrate 20 out of our32 blades successfully.Unfortunately, we had booked the
engineer only for one day, but he wasable to shuffle appointments around andreturn on Friday morning of that weekto finish up. Halfway through Fridaymorning, we were able to finish withthe blade servers so that they all were
racked. We were ready to be done atthat point, but we were only halfwaythere. We still had to install all thehard drives, integrated network pass-throughs, fiber-channel switches andfinally, upgrade the firmware.
[Bill: I had forgotten how long thatjob took. Now that you mention itthough, there was an amazing amountof cardboard generated from that. Itdidn’t help that the data center didn’tallow cardboard on the data centerfloor, and you had to shuttle all thatstuff back and forth.]
Late for the PartyOnce we powered on the blades, itlooked like we were close to the finishline. I started packing up all my thingsso I could head home early and getdressed up for our company’s bigChristmas party later that evening.Naturally, it was at this point that a fewof the blades wouldn’t power on. Afterminimal troubleshooting, we were leftwith just one misbehaving blade. Theengineer started the hardware trou-bleshooting process as I watched theminutes tick by. I realized I had tosomehow power through Bay Area traf-fic, get home, put on my suit and driveback through the traffic to the party,and rush hour was rapidly approaching.Ultimately, we had to open up the serv-er, remove all of the hardware we had
TALES FROM THE SERVER ROOM
added only hours before, and insert thehardware one piece at a time until weidentified a faulty DIMM slot. Finally, wewere done and I was able to get to theparty fashionably late.I think the moral to this story is pretty
clear. If we had only gotten all of ourservers integrated ahead of time, theentire install would have taken a frac-tion of the time, and any hardwareproblems in the system would havebeen identified before anything wasshipped to me. When you have theoption, especially when it comes tolarge orders of servers, get all yourcomponents integrated ahead of time.
[Bill: The moral for me as a manager,is always to double-check the quote forservices, and make sure that all of thoseare understood so they don’t get labeledas non-essential and cut by people higherup the food chain. I’m usually not a fanof too many vendor services, but gettingthe entire system integrated by a vendorwill accelerate deployment time by atleast a couple days.]�
Kyle Rankin is a Sr. Systems Administrator in the San FranciscoBay Area and the author of a number of books, including TheOfficial Ubuntu Server Book, Knoppix Hacks and Ubuntu Hacks.He is currently the president of the North Bay Linux Users’ Group.
Bill Childers is an IT Manager in Silicon Valley, where he liveswith his wife and two children. He enjoys Linux far too much,and he probably should get more sun from time to time. In hisspare time, he does work with the Gilroy Garlic Festival, but hedoes not smell like garlic.
Advertiser Index
ATTENTION ADVERTISERS
The Linux Journal brand's following hasgrown to a monthly readership nearly onemillion strong. Encompassing the magazine,
Web site, newsletters and much more,Linux Journal offers the ideal content
environment to help you reach your marketingobjectives. For more information, pleasevisit www.linuxjournal.com/advertising.
Advertiser URL Page #
CHECK OUT OUR BUYER'S GUIDE ON-LINE. Go to www.linuxjournal.com/buyersguide where you can learnmore about our advertisers or link directly to their Web sites.
Thank you as always for supporting our advertisers by buyingtheir products!
1&1 INTERNET, INC. www.oneandone.com 3
ABERDEEN, LLC www.aberdeeninc.com 142
ARCHIE MCPHEE www.mcphee.com 141
BZ MEDIA www.hackerhalted.com/2011 93
CPANEL www.bootcamp.cpanel.net 101
DEBIAN CONFERENCE debconf11.debconf.org 111
DIGI-KEY CORPORATION www.digi-key.com 141
EMAC, INC. www.emacinc.com 25
EMPERORLINUX www.emperorlinux.com 59
FUDUNTU www.fuduntu.org 141
GENSTOR SYSTEMS, INC. www.genstor.com 31
HIGH PERFORMANCE COMPUTING www.flaggmgmt.com/hpc 49
IXSYSTEMS, INC. www.ixsystems.com 2, 7
LINODE, LLC www.linode.com 41
LOGIC SUPPLY, INC. www.logicsupply.com 15, 75
LULLABOT doitwithdrupal.com 12, 13
MICROWAY, INC. www.microway.com 27, 143
MIKRO TIK www.routerboard.com 39
RACKMOUNTPRO www.rackmountpro.com 37
RENDEK ONLINE MEDIA linuxcareer.com 47
SAINT ARNOLD BREWING COMPANY www.saintarnold.com 141
SILICON MECHANICS www.siliconmechanics.com 28, 29, 43
SOFTWARE FREEDOM INTERNATIONAL www.softwarefreedomday.org 140
TECHNOLOGIC SYSTEMS www.embeddedx86.com 21
USENIX ASSOCIATION www.usenix.org/lisa11/lj 123
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 139
LINU
X JOU
RN
AL M
AR
KETP
LACE
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 141
ANYONE INTERESTEDIN SAVING MONEY?
888-297-7409www.aberdeeninc.com/lj038
Looks like these guys are comfortable overpaying for enterprise storage. Are You?
“Hewlett-Packard Co. agreed to buy 3Par Inc. for $2.35 billion” — Bloomberg.com
Above specific configurations obtained from the respective websites on Feb. 1, 2011. Intel, Intel Logo, Intel Inside, Intel Inside Logo, Pentium, Xeon, and Xeon Inside aretrademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. All trademarks are the property of their respective
owners. All rights reserved. For terms and conditions, please see www.aberdeeninc.com/abpoly/abterms.htm. lj038
“EMC to Buy Isilon Systems Inc. for $2.25 Billion” — Wall Street Journal
“Dell to Buy Compellent for $960 Million” — CNBC
So what “benefit” will you see by this spending spree, other than higher costs?The AberSAN Z-Series scalable unified storage platform, featuring the Intel® Xeon® processor 5600series, brings the simplicity of network attached storage (NAS) to the SAN environment by utilizingthe innovative ZFS file system. The AberSAN Z20 is easily found starting under $20,000.
Who gives you the best bang for the buck?3Par Compellent Isilon Aberdeen
InServ F200 Storage Center Series 30 NL-Series AberSAN Z20
Storage Scale-Out ✓ ✓ ✓ ✓
Thin Provisioning ✓ ✓ ✓ ✓
HA Clustering ✓ ✓ ✓ ✓
VMware® Ready Certified ✓ ✓ ✓ ✓
Async / Synchronous Replication ✓ ✓ ✓ ✓
iSCSI / Fibre Channel Target ✓ ✓ iSCSI Only ✓
Unlimited Snapshots x ✓ ✓ ✓
Native Unified Storage: NFS, CiFS x x ✓ ✓
Virtualized SAN x x x ✓
Deduplication x x x ✓
Native File System none none OneFS ZFS 128-bitRAID Level Support 5 and 6 5 and 6 Up to N+4 5, 6 and Z
Raw Array Capacity (max) 128TB 1280TB 2304TB UnlimitedWarranty 3 Years 5 Years 3 Years 5 Years
Online Configurator with Pricing Not Available Not Available Not Available Available
Cut Execution Time by >50% with WhisperStation-GPUDelivered ready to run new GPU-enabled applications:
WhisperStation with 4 Tesla Fermi GPUs
2U Twin2 Node with 4 Hot-Swap MotherboardsEach with 2 CPUs and 256 GB
1U Node with 2 Tesla Fermi GPUs
OctoPuter™ 4U Server with up to 8 GPUs and 144 GB memory
Microway’s Latest Servers for Dense Clustering
4P, 1U nodes with 48 CPU cores, 512 GB and QDR InfiniBand 2P, 1U nodes with 24 CPU cores, 2 Tesla GPUs and QDR InfiniBand 2U Twin2 with 4 Hot-Swap MBs, each with 2 Processors + 256 GB 1U S2050 servers with 4 Tesla Fermi GPUs
Microway Puts YOU on the Cutting Edge
Design your next custom configuration with techs who speak HPC. Rely on our integration expertise for complete and thorough testing of your workstations, turnkey clusters and servers. Whether you need Linux or Windows, CUDA or OpenCL, we’ve been resolving the complicated issues – so you don’t have to – since 1982.
Integrating the latest CPUs with NVIDIA Tesla Fermi GPUs, Microway’s WhisperStation-GPU delivers 2x-100x the performance of standard workstations. Providing explosive performance, yet quiet, it’s custom designed for the power hungry applications you use. Take advantage of existing GPU applications or enable high performance with CUDA C/C++, PGI CUDA FORTRAN, or OpenCL compute kernels.
Nvidia Quadro for state of the art professional graphics and visualization
Ultra-quiet fans, strategically placed baffles, and internal sound-proofing
Up to 24 cores with the newest Intel and AMD Processors, 128 GB memory, 80 PLUS® certified power supply, and eight hard drives
Up to Four Tesla Fermi GPUs, each with: 448 cores, 6 GB GDDR5, 1 TFLOP single and 515 GFLOP double precision performance
New: Microway CL-IDE™ for OpenCL programming on CPUs and GPUs
GSA ScheduleContract Number:GS-35F-0431N
ANSYS MechanicalAutodesk Moldflow Mathematica
Simulation
MATLABACUSIM AcuSolveTech-X GPULib
3ds MaxBunkspeed ShotAdobe CS5
Design
AMBERGROMACSNAMD, VMDTeraChem
BioTech
Configure your next WhisperStation or Cluster today!microway.com/quickquote or call 508-746-7341Sign up for technical newsletters and special GPU promotions at microway.com/newsletter
pC4_Microway.indd 1pC4_Microway.indd 1 10/17/10 5:07:51 PM10/17/10 5:07:51 PM