the minimum you need to know about service oriented architecture

7/27/2019 The Minimum You Need to Know About Service Oriented Architecture

1/76

About Service Oriented Architecture

By Roland Hughes

Logikal Solutions

The Minimum You Need to Know


2/76

Copyright2007 by Roland Hughes

All rights reserved

ISBN 0-9770866-6-6

ISBN-13 978-0-9770866-6-5

This book was published by Logikal Solutions for the author. Neither Logikal Solutions nor the

author shall be held responsible for any damage, claim or expense incurred by a user of this book

as a result of its use or reliance upon.

These trademarks belong to the following companies:

ACMS Hewlett-Packard Corporation

Ask.com IAC (InterActiveCorp)

Attunity Attunity Ltd.Borland Borland Software Corporation

DEC Digital Equipment Corporation / Hewlett-Packard Corporation

DECdtm Digital Equipment Corporation / Hewlett-Packard Corporation

DECForms Digital Equipment Corporation / Hewlett-Packard Corporation

DR DOS DRDOS, Inc.

MS DOS Microsoft Corporation

JBuilder Borland

JSP Sun Microsystems

Linux Linus Torvalds

MQSeries IBM (International Business Machines Corporation)

MySQL MySQL AB

OnPlatform Hewlett-Packard Corporation / Attunity Ltd.

Oracle Oracle CorporationPostgreSQL PostgreSQL Global Development Group

RDB Oracle Corporation

Reactivity IBM

RMS Hewlett-Packard Corporation

Tru64 Hewlett-Packard Corporation

Ubuntu Canonical Ltd.

Unix The Open Group

Windows Microsoft Corporation

All other trademarks inadvertently missing from this list are trademarks of their respective owners. A best effort wasmade to appropriately capitalize all trademarks that were known at the time of this writing. Neither the publisher nor the

author can attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the

validity of any trademark or service mark.


3/76

Acknowledgments

Books like this, the planned closing of a series segment, come about because dedicated

readers like yourself purchased, read and wrote about the prior books in the series. When thatdoesn't happen, a series segment simply gets left undone. To paraphrase the Bartles & James

commercials: I thank you for your support.


4/76

Source Code License

Any person owning a copy of this book may use the source code from this book and

accompanying CD-ROM freely when developing software for their personal use, their

company's use or their client's use. Such persons may include the source code either modified orunmodified provided that the source delivered makes reference to the original author and is part

of a fully functional application. It is expressly forbidden for anyone to post this source code on

any bulletin board system, Internet Web site, or other electronic distribution medium without the

express written permission of the author. It is also expressly forbidden to sell this source as part

of a library or shareware distribution of source.

Users of the source code contained within this book and on the accompanying CD-ROM

agree to hold harmless both he author and the publisher for any errors, omissions, losses, or other

financial consequences resulting from the use of said source. The source code is supplied asis with no warranty of any kind expressed or implied.


5/76

Table of Contents

1. Introduction...............................................................................................................................1

1.1 SOA Where We Came From............................................................................................1

1.2 The Disaster We Are Headed Toward................................................................................4

1.3 Why We Are Headed There................................................................................................6

1.4 The Roots of SOA...............................................................................................................8

1.5 Additional Reading..............................................................................................................8

2. Clustering..................................................................................................................................9

2.1 What is Clustering...............................................................................................................9

2.2 Distributed Transaction Management...............................................................................14

2.3 Two-phase Commit...........................................................................................................15

2.4 Shared Resources..............................................................................................................16

3. What is Wrong Today.............................................................................................................19

3.1 We Sold Modules..............................................................................................................19

3.2 We Are Still Selling Modules...........................................................................................20

3.3 The End of Modules..........................................................................................................21

3.4 How Do We Fix This?.......................................................................................................22

3.5 Today's SOA Camps.........................................................................................................24

3.6 Start Small.........................................................................................................................28

4. Green Screens on the Web......................................................................................................31

4.1 Obtaining a Web Server....................................................................................................31

4.2 Obtaining a Web Terminal................................................................................................32

4.3 Is This SOA?.....................................................................................................................35

5. ACMS Fundamentals..............................................................................................................37

5.1 Why Are You Only Hearing About ACMS Now?............................................................37

5.2 Flow Description...............................................................................................................39

5.3 General Principals.............................................................................................................40

5.4 IO Routines........................................................................................................................43

5.5 The Major Pieces...............................................................................................................44

5.6 Which Type of ACMS Application?.................................................................................49

5.7 Security..............................................................................................................................50

5.8 The March Forward...........................................................................................................51


6/76

6. Our First Steps.........................................................................................................................55

6.1 Deciding on the Method....................................................................................................55

6.2 Direct Database Access via Remote Java..........................................................................56

6.3 Some Thoughts on Editors................................................................................................62

Illustration 6.1. .................................................................................................................64

Illustration 6.2. .................................................................................................................65

Illustration 6.3. .................................................................................................................66

Illustration 6.4. .................................................................................................................68

Illustration 6.5. .................................................................................................................70

6.4 Database Access via Java Applet......................................................................................72

Illustration 6.6. .................................................................................................................73

Illustration 6.7. .................................................................................................................81

6.5 A Bit More on Shell Scripts..............................................................................................83

6.6 What About MySQL?.......................................................................................................84

Illustration 6.8. .................................................................................................................90

6.7 What Have We Learned?..................................................................................................94

7. A Direct Port............................................................................................................................95

7.1 Why This Method?............................................................................................................95

7.2 How Will We Proceed?.....................................................................................................96

7.3 The Login Dialog..............................................................................................................98

Illustration 7.1. .................................................................................................................98

Illustration 7.2. ...............................................................................................................103

Illustration 7.3. ...............................................................................................................104

Illustration 7.4. ...............................................................................................................105

Illustration 7.5. ...............................................................................................................113

Illustration 7.6. ...............................................................................................................114

Illustration 7.7. ...............................................................................................................130

Illustration 7.8. ...............................................................................................................131

Illustration 7.9. ...............................................................................................................132

7.4 Assignments....................................................................................................................143

7.5 Experts.............................................................................................................................143

7.6 Too Much Time On Your Hands....................................................................................144

7.7 Is this SOA?.....................................................................................................................144


7/76

8. Service Types........................................................................................................................149

8.1 Different Types of Service..............................................................................................149

Illustration 8.1. ...............................................................................................................152

8.2 Exposing RMS Files via Services...................................................................................154

8.3 Assignments Part 1..........................................................................................................174

8.4 Using Those Services......................................................................................................174

Illustration 8.2. ...............................................................................................................175

Illustration 8.3. ...............................................................................................................186

8.5 Assignments Part Two.....................................................................................................189

8.6 Is This SOA?...................................................................................................................192

8.7 What About Security?.....................................................................................................193

8.8 Summary.........................................................................................................................195

9. Load Balancing and Fault Tolerance.....................................................................................197

9.1 Brute Force......................................................................................................................197

9.2 Back-End Balancing........................................................................................................198

9.3 Warehousing Policy Effect on Load Balancing..............................................................205

9.4 How Do You Balance the Load?.....................................................................................208

9.5 How Do You Protect the Transaction?............................................................................208

9.6 Can I Use DECdtm Without ACMS?..............................................................................213

9.7 How Much Fault Tolerance Does DECdtm Buy You?...................................................214

Illustration 9.1. ...............................................................................................................216

Illustration 9.2. ...............................................................................................................217

9.8 Can I Use MQ Series Without ACMS?...........................................................................221

9.9 Queue Trigger.................................................................................................................221

9.10 Summary.......................................................................................................................222

10. An ACMS Example..............................................................................................................223

10.1 Scope of Our Example..................................................................................................223

10.2 Our DTD........................................................................................................................226

10.3 Our Files .......................................................................................................................231

10.4 ACMS Queues...............................................................................................................240

10.5 The Interim Record Formats.........................................................................................249

10.6 Parsing XML on OpenVMS With Xerces.....................................................................251

10.7 Parsing XML on OpenVMS With libXML2.................................................................256

10.8 Our Standalone XML Parser.........................................................................................262

10.9 One Last Bit About ACMS Queues..............................................................................278

10.10 Standalone to Server....................................................................................................283

10.11 Authorizing, Installing, Starting, Stopping and Testing..............................................316

10.12 Monitoring Your ACMS Applicaiton.........................................................................336


8/76

10.13 Controlling Your ACMS Application.........................................................................338

10.14 Our Port Service..........................................................................................................339

10.15 Our Order Generator...................................................................................................342

10.16 Assignments................................................................................................................346

11. Whine and Snivel..................................................................................................................347

11.1 But, but, but...................................................................................................................347

11.2 The Item Service...........................................................................................................347

11.3 Our New Applet............................................................................................................350

Illustration 11.1. .............................................................................................................350

11.4 Summary.......................................................................................................................360

12. Observations and Incantations..............................................................................................363

12.1 Overview.......................................................................................................................363

12.2 The Future of SOA........................................................................................................363

12.3 Java, Write Once, Run a Few Places.............................................................................367

12.4 Grade 8 Bolt Syndrome.................................................................................................368

12.5 In Closing......................................................................................................................370


9/76

Chapter 1

Introduction

1.1 SOA Where We Came From

For quite some time now, marketing shills have been preaching and selling SOA (Service

Oriented Architecture). Vast quantities of money have been poured into The Next Big Thing.

Well, it is time to ask a few questions and rain on their parade.

What is the Service Oriented Architecture? This is a re-hash of an old business model that

played itself out from the 1970s into the 1980s. Back then computers were multi-million-dollarinvestments. Small to mid-sized companies simply couldn't afford them. Programmers were

scarce and qualified programmers even scarcer. Small to mid-sized companies could afford a

few thousand dollars for terminals and special modems. Then they spent several more thousand

dollars per month leasing time on computer systems. There were actually businesses that did

little more than lease time. They would also sell consulting services and lease access to some of

their applications.

Those of you who grew up with $20/month Internet access probably have a hard time

imagining someone paying thousands of dollars per month for access to a computer system, but

it happened. Commercial-class operating systems provided system and application run-time

accounting. Among other things they would measure the amount of computer time, connection

time, and IO that each user ID performed on the system. Every month the subscribing

companies would be billed by the CPU second, connection minute and block of IO. It shouldn't

take a rocket scientist to do the math here. One thousand customers paying $10K+/month to use

your computer system paid off a $2 million price tag pretty quickly. Even 100 customers could

make it viable if you were using the computer system to run your own business. You bought the

machine on time and they made the payments for you.

Of course, the companies looking to lease time didn't have programmers (initially) and didn't

have applications, so those time-share businesses got into the custom software business. They

had proprietary data layouts that locked their clients into ever escalating support costs. It was

like a drug addiction you couldn't kick. (Sound familiar to those of you using stored procedures

in a commercial database?)


10/76

2 Chapter 1 - Introduction

Mid-range computers came onto the scene. Many of the customers of mainframe time-share

systems opted to buy a mid-range computer for about 6-12 months worth of their time share cost.

Eventually, they too got into the custom application and time-share business, but not on as widea scale.

Then came the PC. Promising to put computing power in the hands of the people without

any regard for where that would lead. Vendors of hardware and software for these little boxes

(for the time anyway) made money hand over fist. Competition became cut-throat, and soon

carcasses of once proud vendor corporations littered the landscape. Companies had a rash of

desk drawer applications that were now required to keep their businesses functioning. Very

little could communicate effectively or exchange data reliably.

To stem the bloodshed, companies began focusing on networking technology. Multiple

competing proprietary networking protocols emerged. File servers and network application

servers sprang up all over the place. If you could afford the proprietary network interface

equipment and the licensing, you could all use the same applications and share data (in theory).

Greed set in even worse than before. It was nothing to end up paying well over $1,000 for a

network card to get a computer online. Sadly, you usually only paid about $3,000 for the PC to

begin with. Somewhere, in my souvenirs from the past, I have an ISA-based BiSync modem that

operated at a whopping 1200 baud. I paid just over $1,300 for it and it was a buy! The namebrand version was around $2,500 as I recall. The great part about these proprietary networking

systems was that they didn't talk to each other. Communicating across them required even more

expensive bridging software and dedicated hardware. Even when you had the networking people

configuring your machine in the office, you didn't always get network access. In their own weird

way, the same barriers people complained about provided a nearly unparalleled level of security.

Networks were so finicky to get working from the inside, hacking in from the outside was almost

a myth.


11/76

Chapter 1 - Introduction 3

Next came the TCP/IP standard, and the Internet. People wanted everything to connect to

the Web. Hackers now had a standardized medium to exploit and they took to it with zeal. PC

prices dropped dramatically and PCs started popping up all over the place. PC operating systemswere (and still are) so unstable and riddled with security holes that every application and project

had to have its own server. Servers started appearing around corporations like rabbits taking

both Viagra and fertility drugs. Suddenly companies found they were spending as much on these

cheap systems as they had been on the mainframes that took up less total space and consumed

less power.

Enter the great server consolidation move. Completely disregard the fact your PC server OS

is unstable, requiring frequent rebooting and patching, taking the applications off-line. Load

virtualization software on one, and consolidate 8 or more into one machine. Uptime be damned!Now when you have to reboot you can take out 8 or more servers.

One problem still remained though. Each application running on a server was pretty much

its own data island. Even when it stored its data in a database, that database was either on the

same server itself, or the database wasn't the same one used by other applications. Management

had cost cut its way into a very bitter pickle indeed.

Fear not though, the marketing shills had even more to sell them under the heading of

cutting costs. Management always listens intently to the buy more to spend less sales pitch.Repackage these applications with this magic middleware (for an additional fee) and you can use

pieces of them, tying them together as you wish. They will all have a protocol to exchange data.

You can use order entry from this company, inventory management from that company, and if

you buy today, company three will sell you a piece of their picking ticket & truck loading

module.

In fact, if you want to use it all from one vendor, it will work the best. No need to install

their applications, simply lease time on their machines. Get rid of all these pesky servers. Let

someone handle the updates and capacity issues for you. Vendor A has built a massive datacenter in a single location with redundancy upon redundancy all in the same 40 acres. After all,

the world is one network now isn't it? Why should you keep buying all of this equipment,

simply pay for the time you use...


12/76


Sounds kind of familiar doesn't it? Pay by the application module, computer use and

storage. If it only ended there, it would be laughable. But it gets worse.

1.2 The Disaster We Are Headed Toward

What are they using to host all of this wonderful stuff? Racks and racks of PC-on-a-card

servers. Heat is a bigger problem for them than it was for the old mainframes. Some companies

are now looking into liquid cooling systems for those personal computer chips. Liquid

cooling, just like some of the big mainframes used.

What operating systems are they running? The same crummy operating systems that were

running at the client company. Security holes and patches are a seemingly daily if not hourly

occurrence, but the client companies are used to outages. Outages have become a business

standard.

What happens when one of those PC-on-a-card devices fries? Why, they load you up

another one at some point. Can't really say how long the outage will be, but you're used to it

already. Sorry about the transactions that were being processed, but you are used to losing those

as well.

These faults can all be classified under the heading Ignorance of a Cheap Mind.

What cannot be classified under that heading, or even forgiven, is the packaging. These

supposed forward thinkers have all bought into the idea that one massive data center located on

X acres some place cheap is a good idea. What is even more unthinkable is that Congress is

letting them do it.

I'm not talking about Congress being in control of business. I'm talking about Congress

letting the National Security Agency take these industry leaders into a dark alley with ball bats

and cattle prods to educate them on the finer points of economic and industry stability.

Let us assume for a moment that Congress is too busy getting re-elected to do anything

useful. (Admittedly, not a big assumption.) These data centers on steroids are allowed to

prosper. They have huge dollars behind them and drive out competitors so we have only a

handful. Nah, let's shrink the number. We end up with only three, just like we did with auto


13/76


manufacturers years ago.

With greed coursing through their veins stronger than any narcotic on the market, they entera cut-throat competition just like the PC manufacturers did. Pretty soon, they've driven the price

down to just a couple thousand per month no matter how much you use. Talking heads at

companies around the country start chanting cut costs and looking in the direction of these

service providers. Yes, it's a financial Utopia all right.

In a need to differentiate themselves in this market, the Big 3 buy up applications focusing

on specific market segments. One provides the best of the best in banking software. Soup to

nuts online checking, accounting, wire transfer, you-name-it-they-can-do-it banking product.

Another ends up focusing on the complete solution for manufacturing. We are talking orderentry, inventory control, sales force management, assembly line systems, shop floor control, the

works. The last of the three ends up providing best in class shipping logistics. From truck

loading to routing, and everything in between, they have modules to do it.

Ah, Utopia. They even develop a working relationship, so the manufacturing systems can

use the shipping logistics modules and they both use parts of the financial modules. It truly is

bliss. This is what marketing has promised and they finally delivered. Sort of...

No matter how you slice it, these providers spent heaps of money building up one location.Usually where there was really cheap power. One of them ends up getting built well within the

blast radius of Mt. Rainier when it finally pops its top. (According to scientists within the next

20 years, but they guessed a little long on Mt. St. Helens, so it is probably at the shallow end of

the 20.) Another gets built on a major fault line because power was half the price of anywhere

else. The last gets built in the low cost real estate district found near the end of major airport

runways because they got a sweet deal from the state and city governments.

It shouldn't be any surprise where this is heading. It doesn't matter how much power

generation and redundancy each location has built into it, they will be taken out, either formonths or permanently. A jet has engine trouble on take off and crashes, Mt. Rainier decides to

prove the nay sayers wrong, and that 8.2 quake finally does originate from the dormant fault line

just like geologists said it would eventually. In this day and age, we also have to deal with the

possibility of a terrorist group either crashing a jet or driving a truck load of homemade

explosives into the site. Let us not forget things like Katrina and the tsunami for the centers

within a few miles of a coast line. Once we get to a Big 3 or even Big 4 situation with these


14/76


single location service centers, the destruction of one, by any means, cripples a nation and quite

possibly the world economy. PC and UNIX people think only about loss of a single piece of

equipment or power. Midrange developers think about the loss of an entire facility.

1.3 Why We Are Headed There

Where we are headed is not good because of the premise which started it all. Actually, there

were multiple premises that lead us down this gilded path:

Running core business systems on feeble operating systems was OK because the

hardware was cheap.

Using operating systems which cannot cluster completely is OK when they are cheap.

Replacing existing systems with a flat monthly fee service was a great way of cutting

costs.

Building a single massive location to do everything and get economies of scale was a

good idea.

Cheap hardware lead to the PCs on Viagra and fertility drugs scenario. Well, that, and

unstable operating systems. Even IBM is running ads about servers being so numerous you

cannot walk in a building. While they are an exaggeration, they aren't off by much. Oddly

enough, IBM marketed the original IBM PC direct to businesses in the first place, so this is kind

of like profiting from the sins of your past.

Cheap operating systems. Gotta love that. I remember companies and people complaining

about spending $80 for DOS. At least with DOS, once you bought it, it was yours. Now

Windows is moving toward the As soon as you stop paying for the service agreement your use

licenseexpires model with an initial rental price of around $300. Commercial flavors of UNIX

had their own trappings when it came to license agreements. Now people hold up Linux as the

be all and end all. Only one problem, none of these operating systems cluster correctly or well.

They network, but do not cluster. Even the Tru64 version of UNIX, released by the company

that gave the world its first (and still only) commercially viable clustered operating system didn't

manage to get clustering working correctly in UNIX. The problem is, you can't be loyal to the

UNIX installed base and cluster. The platform is missing critical pieces, and when those pieces

are added, it won't be UNIX anymore, it will be a clone of OpenVMS with some command

syntax changes.


15/76


Flat monthly cost. Management loves the flat monthly cost model. Indeed, Internet access

providers and now cable phone providers are herding consumers towards that mentality.

Management loves it because they now only have one thing to look at when they sit around in atalking heads meeting chanting cut costs. Leaves more time for the golf course.

A single massive data center. Now here is an idea straight from the 1970s. You have to be

old to remember it. The classic IBM mainframe data center. Raised floor to hide cabling, route

air conditioning, and provide a water trap. Bullet resistant glass wall so people could walk by

and be amazed by the equipment. Two to three layers of fire protection. Multiple lock entry

security. In some cases, even data center guards by the doors. Let's not forget the massive diesel

engine generators parked right outside the building and in later years the room full of UPS

(Uninterruptable Power Supply) equipment.

IBM was the originator of the big iron trend. It didn't cluster, but it did have it's own

proprietary networking for communications. As the platform progressed it also had machine

partitioning. This would let you divide up your big iron box so it looked like multiple

mainframes to the outside world. Sounds a lot like virtualization doesn't it? Big iron was big

bucks. They didn't bother with clustering technology because very few companies could afford

to scatter a dozen or so of these around the country. Instead, they put forth what became known

as the moat and fortress data center design. This lead to the bank teller bullet resistant glass

plates and the other things we talked about. I never really did figure out what the moat was,unless it was the raised floor or the fact many companies were told to locate their data center on

the second floor, not the first or the basement. They used to get put in the basement because the

equipment was so heavy, then someone thought about the flooding issue.

DEC (Digital Equipment Corporation) focused primarily on the midrange market for mid-

sized companies. These were companies that could afford to spend one half to two million for

their entire data center and computer system. The companies knew they needed a computer

system to compete and keep growing, but they couldn't bite the bullet for a multi-million-dollar

mainframe setup. As the companies ran more efficiently with the computer system they keptfinding more and more uses for it. For a time they bought bigger and bigger boxes, but the

engineers at DEC came up with an even better solution. Buy more of the same and hook them

all together so files, batch jobs, users and processing power could be utilized throughout a cluster

of computers. As communications improved worldwide, these clusters, literally spanned the

globe.


16/76


1.4 The Roots of SOA

Both DEC and IBM hit on the need for transaction management. IBM created CICS

(Customer Information Control System) and DEC created ACMS (Application Control

Management System). Both of these were and still are amazing products. ACMS had the most

far reaching implications, yet few realized it. Sadly, much of the market missed it.

While the concept will be completely foreign to those coming from the PC and UNIX sides

of life, these systems allowed you to both package functions for a Service Oriented Architecture

and provided guaranteed execution. A side effect was that each SOA task you created with them

had a messaging format that did not allow for data overrun or SQL injection like the current Web

fiascoes. Yes, there are many third-party messaging systems these days making claims about

guaranteed message delivery, but none will go so far as to guarantee its execution.

Because ACMS was the most far reaching and the one I have the most experience with, we

will be discussing ACMS combined with clustering in this book.

1.5 Additional Reading

If you like the style and content of this book, or would simply like to learn more about

application development on OpenVMS, please pick up copies of The Minimum You Need toKnow to Be an OpenVMS Application Developer ISBN 0-97708660-7. You can follow it up

with The Minimum You Need to Know about Java on OpenVMS Volume 1 ISBN

0-9770866-1-5 ISBN-13 978-0-9770866-1-0.

At some point The Minimum You Need to Know About Logic to Work in IT will be

available as well. This is not an OpenVMS specific book, but meant to be used as a textbook to

teach those new to IT about logic. It appears today's graduates haven't the foggiest clue about it.

ISBN 0-9770866-2-3 ISBN-13 978-0-9770866-2-7.


17/76


Chapter 2

Clustering

2.1 What is Clustering

Clustering, even today, is one of the most forward thinking concepts in the IT industry. It

was so far ahead of its time, the market couldn't really comprehend it. Clustering requires that

when a node joins the cluster, it honors and respects the consensus of the cluster. To start with,

it must use the same UAF (User Authorization File) as the rest of the cluster. Access to cluster

resources is controlled both by the standard (System, Owner, Group, World) protection settings

provided by many platforms and ACLs (Access Control Lists).

ACLs require quite a bit of knowledge to implement. You cannot simply bolt them onto an

OS which doesn't have native support for them. Their use and concept must be firmly

entrenched within the OS kernel. Let me show you an example of a set of ACLs on a directory.

$ dir/sec dka1200:[000000]acl_test.dir

Directory DKA1200:[000000]

ACL_TEST.DIR;1 1/18 29-NOV-2004 17:37:24.04 [HUGHES] (RWE,RWE,RE,E) (IDENTIFIER=GST_S,OPTIONS=DEFAULT,ACCESS=READ+WRITE+EXECUTE+DELETE)

(IDENTIFIER=GST_RE,OPTIONS=DEFAULT,ACCESS=READ+EXECUTE) (IDENTIFIER=GST_RO,OPTIONS=DEFAULT,ACCESS=READ) (IDENTIFIER=[*,*],OPTIONS=DEFAULT,ACCESS=NONE)

The series of RWE letters enclosed in parenthesis to the right is the standard (System,

Owner, Group, World) protection I mentioned before. Read, Write, Execute and Delete are the

access privileges you can use for each class of user. If a user is missing one of those letters, it

cannot perform that function on the file or directory. UNIX and its derivative works do not

implement four levels, so they fall a little short from the start.

Take a look at the ACL list though. Each identifier gets its own unique set of access

privileges. The possible entries are READ, WRITE, EXECUTE, DELETE and CONTROL.

Because this is a directory, the OPTIONS=DEFAULT clause is used to cause the ACL to be

inherited by all files and directories created underneath this directory. The very last entry in this

list tells the operating system to completely ignore the standard (System, Owner, Group, World)

settings for anyone other than the file owner. The wild card entry [*,*] matches all identifiers.


18/76

10 Chapter 2 - Clustering

Order is very important within an Access Control List. The entries must be in the order you

wish them to be found. For this reason, the highest privilege access levels are always listed first.

The reason: the ACL is searched sequentially for the first matching rights identifier. Once a

match is found, that level of privilege or access is given to the user or process. If [*,*] were first,

it wouldn't matter what came after.

But what is a rights identifier, and where does it come from? Rights identifiers are created

by one of the system administrators from within the UAF utility. They are then granted to user

accounts as the company or system administrators see fit. Normally, companies set up

procedures and forms for creating rights identifiers and granting them to accounts.

$ set def sys$system

$ run authorizeUAF> add/ident gst_ro%UAF-I-RDBADDMSG, identifier GST_RO value %X80010001 added to rights databaseUAF> add/ident gst_re%UAF-I-RDBADDMSG, identifier GST_RE value %X80010002 added to rights databaseUAF> add/ident gst_s%UAF-I-RDBADDMSG, identifier GST_S value %X80010003 added to rights database

UAF> grant/ident gst_ro guest%UAF-I-GRANTMSG, identifier GST_RO granted to GUESTUAF> grant/ident gst_re guest%UAF-I-GRANTMSG, identifier GST_RE granted to GUESTUAF> grant/ident gst_s guest%UAF-I-GRANTMSG, identifier GST_S granted to GUEST

As you can see, the operating system chooses a numeric value for them at the time of

creation. This is another reason each member in the cluster must respect the UAF chosen by thecluster. While you might be able to manually create matching authorization files as far as users

go on all machines, getting the rights identifiers to have the same value on all machines would be

a very tedious task.

Once a user has some rights identifiers, it doesn't matter if they don't have any operating

system level privileges. As long as they have the authority to log in, they will be able to use any

resource with a matching ACL entry for one of their rights identifiers. In this way, you can

spoon out access on a file by file, directory by directory, disk by disk, batch queue by batch

queue, executable by executable, etc. basis. Most companies don't take it to quite that level, butyou can if you wish.


19/76

Chapter 2 - Clustering 11

The UAF and rights identifiers provide the primary access keys to opening up the power of a

cluster. The hidden beauty of a cluster is the support for it built into the kernel. One of the main

foundations for that hidden beauty is logical name tables. Most PC and UNIX type developers

are only familiar with the concept of environment variables. Under OpenVMS those are known

as symbols. Symbols aren't really useful in a secured and robust operating system. They have

some purposes, mostly for programs running under a single user ID, but they don't allow for

controlled access or for being cluster aware. The solution here comes in the form of logicals.

OpenVMS and RMS (Records Management System) utilize logicals extensively. A logical

is much like an alias only much more powerful. Some of them are provided by the system and

others you can create yourself as a programmer. Logicals occur at what are called levels. A

level is nothing more than a logical name table in your logical search path. Your search path isdefined by a logical known as LNM$FILE_DEV. Every logged in process starts out with a

default version of LNM$FILE_DEV, which is found in the logical name table LNM

$SYSTEM_DIRECTORY. Various levels of privileges are required to define logicals, but

every process can define its own process level logicals.

$ show log/table=lnm$system_directory lnm$file_dev LNM$FILE_DEV = LNM$PROCESS (LNM$SYSTEM_DIRECTORY) = LNM$JOB = LNM$GROUP = LNM$SYSTEM = DECW$LOGICAL_NAMES

1 LNM$SYSTEM = LNM$SYSTEM_TABLE (LNM$SYSTEM_DIRECTORY) = LNM$SYSCLUSTER2 LNM$SYSCLUSTER = LNM$SYSCLUSTER_TABLE (LNM$SYSTEM_DIRECTORY)1 DECW$LOGICAL_NAMES [table] = (LNM$SYSTEM_DIRECTORY)

The default definition of LNM$FILE_DEV shows you the primary levels of logical name

tables. LNM$JOB is the job level name table. Each time a user logs in or a detached/batch

process is created, it gets assigned a unique job ID and a job level logical name table is created.

The process level logical name table is created for each process as that process is created.

Processes which are spawned, or otherwise created as children of a parent process inherit the

parent job level table. The group level logical name table exists for everyone in a group. One

table is created for each user group found in the UAF at system startup. Each user process mapsonto the group table for its group.


20/76


The system level table gives us some interesting talking points. As each node boots it

creates system level logicals which are available to all processes and users. As each node joins a

cluster, the cluster imposes its will by defining cluster-wide logicals. These logicals exist in the

local copy of the system table on each node in the cluster, but any update or modification of a

cluster-wide logical gets replicated across the entire cluster.

Each application you create, buy or otherwise install, can, and usually does, create its own

application level logical name table. Sometimes access to these tables is thrown open to the

world, and other times the table is created with an ACL for security reasons. (You don't want a

guest user to be able to look at logicals used by your accounting system.) When a user needs

access to these application level logicals, the name table is usually inserted into their local copy

of LNM$FILE_DEV.

CREATE_NAME_TABLE.COM

$ create/name_table/parent_table=lnm$system_directory my_app_tbl$!$ set security/class=logical/acl=(identifier=gst_re,access=read+execute) my_app_tbl$!$ set security/class=logical/prot=(w) my_app_tbl$!$ define/table=my_app_tbl first_logical sys$login$!$ exit

The above is a command file or script for creating a logical name table. Below is itsexecution and output.

$ @create_name_table$ show log/full/table=my_app_tbl

(MY_APP_TBL) [super] [shareable] [Protection=(RW,RW,R,)] [Owner=[HUGHES]] (IDENTIFIER=GST_RE,ACCESS=READ+CREATE)

FIRST_LOGICAL [super] = SYS$LOGIN

You will notice that the protection on this table wasn't opened up to the world. The world

portion of (S,O,G,W) has no entries in it. Somewhere in the user's login, or the startup for the

application, there would be a command like the following to insert this application name table

into the users definition of LNM$FILE_DEV.

$ DEFINE/TABLE=LNM$PROCESS_DIRECTORY LNM$FILE_DEV - LNM$PROCESS, - LNM$JOB, - LNM$GROUP, - LNM$SYSTEM


21/76


Logicals are required to make clustering work. You cannot try to simply expand symbols or

environment variables past their current local and global types. It simply isn't their lot in life.

Global symbols, by their nature, are unprotected. There primary use is to simplify commands or

exchange non-sensitive data. Trying to add security to them would break their functionality

badly. Logicals were designed with security in mind.

One thing you may or may not have caught onto is that an application level logical name

table can also be declared as a cluster-wide name table. Instead of having the parent be lnm

$system_directory, the parent would be lnm$cluster_table. As each node joined the cluster it

would inherit the table and enforce all access rules to it. Any change made to the table on one

node would be reflected in the table on all nodes.

Earlier I told you that each node joining a cluster makes its resources available to the cluster.

In the early days of clustering, making the disk drives mount across the cluster and appear as

local drives to a user was a great achievement. Now that most disk farms consist of multiple

SAN (Storage Area Network) or disk array subsystems, this may not seem like such an

achievement. To the contrary, it still is.

Disk storage subsystems have the built in capability to map an amazing number of spindles

into what looks like a single physical disk to the connecting computer. Their built in RAID and

hot swapping capabilities off load a lot of work from the computers utilizing them. The built incaching supplements the operating system level drive caching, increasing overall performance.

These are great achievements indeed! They have some limits though.

Because disk arrays and storage networks focus on the IO subsystem directly, in order to

support a large number of operating systems, they don't provide anything in the way of

transactional integrity. Yes, the subsystem can do local drive shadowing, and will not return

success until all writes have completed, but this still doesn't provide you with transactional

integrity. To achieve transactional integrity, you need a distributed transaction manager built

into the OS kernel.


22/76


2.2 Distributed Transaction Management

A distributed transaction manager (DECdtm) makes the transactional integrity provided byrelational databases seem almost weak by comparison. A distributed transaction manager

provides a transaction level safety net across a wide range of items. These include, but are not

limited to, indexed files, message queues and relational databases. If you write a program

utilizing the distributed transaction manager, you can read a message from a queue, update some

indexed files and write information to a relational database with the full confidence no changes

will take place until you actually commit the distributed transaction.

Most of you are familiar with the COMMIT and ROLLBACK commands found in relational

databases. These commands are part of the integrity provided by the relational database engine.

They do not provide any functionality outside of the relational database. If 100% of our data

resided inside of a relational database, this would be a good thing. The fact of life is our data

doesn't all reside there. In the world of SOA, we have hundreds, if not thousands of message

queues to deal with. There always seems to be some legacy data that must remain in indexed

files as well. Quite simply, you can never get there. DECdtm understands this and makes your

life better.

What happens when you are half way through processing a message from a queue and your

program crashes for some reason? Without a distributed transaction manager, your relational

database might be able to realize your process has died and ROLLBACK your transaction

there, but what about the indexed files you wrote to? What about the message you pulled off the

queue? You have a partially processed transaction with no method of starting over. You have

lost data integrity.

The vast majority of commercial messaging systems try to provide their own form of

COMMIT and ROLLBACK. A few of these might also have the ability to determine the process

that read a message from them no longer exists. Most do not have that capability. Because the

messaging systems try to support a large number of operating systems to increase their market

share, they tend to implement things like triggers, which kick off a job to read and process the

message. They only promise the message will not be lost between queues, and that they will fire

the trigger the first time a message arrives. If you read the message, start processing and fail,

they will never fire the trigger for that message again.


23/76


Many shops try to implement a work around by having the trigger fire whenever the

message depth hits N messages in the queue. The trigger process then is supposed to drain at

least that many messages from the queue. All well and good I suppose, but what about that

partially processed message? Nothing has done any cleanup, and your next trigger fire will try to

process it again if it has been restored to the message queue. If the message has not been

restored, it is lost forever. What happens when your trigger depth is set to 10, and the very first

message being processed was the one that made the trigger process abend? Depending on how

your queuing system counts, the depth will either be nine or still be sitting at 10. If it is at 10,

your trigger will never fire again. If it is nine, your trigger will fire with the very next message,

but the message being processed during the abend is lost and gone forever.

A cluster-aware distributed transaction manager is able to determine your process has diedno matter what node it was running on. It then performs the ROLLBACK operation for you.

Robust queue managers like IBM's MQ Series are fully integrated with DECdtm. When you

start a distributed transaction and read a message from an MQ queue, the message stays there,

but is flagged as unavailable until the transaction completes. If the transaction is committed, the

message goes away. In the event of a ROLLBACK, the message is reactivated, maintaining its

place at the head of the queue. This ensures the message will not be dispatched to another server

prior to the cleanup having completed, and it ensures the message will not be lost. Guaranteed

delivery without guaranteed execution is a meaningless feature.

2.3 Two-phase Commit

A feature common to the OpenVMS market and quite rare on other platforms is something

called a two-phase commit. While there are several very detailed (and some might say mind

numbing) explanations around, let me sum it up for you. The DEC transaction manager

communicates with all other resource managers. When you issue a DECdtm commit, it queries

all of the involved transaction managers. If they say they are ready and able to perform the

commit, it issues the commit to them. Once they have all responded the commit completed

successfully, the commit then returns a completion success to your application. If even a single

resource manager responds that it cannot participate at this time, no commit happens.

There are quite a few conceptual states to the two-phase commit. If you really wish to know

how it works, you can delve deeper into the documentation provided by HP. What is important

to know is that you will never get a partially committed transaction.


24/76


ACMS is capable of handling this for you. Each time a transaction starts it can

automatically start a distributed transaction. You can, if your logic requires it, start your own

DECdtm transactions from within your ACMS server.

Without a two-phase commit, it is physically impossible to coordinate a transaction across

multiple resources. When looking at potential message queue software, you need to see what

external transaction managers they support. Roll-your-own logic is what you will find most

shops using free or cheap message queuing systems doing. This is inherently flawed and leads to

significant problems. When a message queue cannot be synchronized with database and legacy

file updates, you end up with either lost messages, or partially committed transactions. This is a

problem when you have to face an audit. It's a major PR problem when you have to explain to

your biggest customer why you processed their order multiple times.

2.4 Shared Resources

We have mentioned a little bit about shared resources in a cluster, but haven't really

discussed it. We have already spoken about disk drives (either physical or the virtual kind

provided by a disk array) being mounted and shared across a cluster. We have also discussed

that user accounts and rights identifiers are common across the cluster. Now we need to talk

about the other resources.

A batch queue created on any given node in the cluster will make itself available to all nodes

in the cluster. A user anywhere in the cluster can submit a job to it, provided they have the

correct rights identifiers or privileges to do that. You can also create generic queues that simply

hold and spool jobs for other batch queues. This generic queue can be made to know about all of

the batch queues on all of the nodes. Each batch queue typically gets created with a limit of how

many batch jobs it can run simultaneously. The generic queue will dispatch to whatever actual

batch queue is available at the time it receives a job.

Although it may seem somewhat primitive, it is a form of dynamic load balancing. Just

because you are on one node and all of the available batch queue slots are consumed by long

running jobs, doesn't mean your job has to wait for their completion. With all of the resources

available to all of the nodes, the job simply gets routed to a node with the capacity to run it.


25/76


Generic batch queues tend to be served by select groups of actual batch queues. This isn't

due to any restriction of the cluster, it has to do with software licensing. Some commercial

packages get licensed in funky ways making it prohibitively expensive to license a product for

all nodes in the cluster. If you bought some form of commercial accounting system, but only

licensed it to run on three nodes of a 20-node cluster, you may create a generic batch queue

called ACCOUNTING$BATCH that only distributes jobs to batch queues on those three nodes.

The setup for the package, or the user login, would define this generic queue as the default batch

queue for that application.

Print queues have similar features. You can create both generic spooling and device specific

print queues. When you are on a node running in California and create a report for a department

at your Germany location, you can simply print it to the queue being serviced by a printer in theirarea. The queue looks like it is a local queue to you. Some shops will go so far as to set up

symbols in each user's account for printing to the different print queues and assign the logical for

their default print queue (SYS$PRINT) to be the printer closest to where they sit.

While physically separate nodes do not directly share memory, it is possible to create

regions of memory with common contents kept in sync via QIO calls and/or daemons. It is also

possible for users on one node to create processes that run on other nodes without them ever

actually logging into those nodes. Once again, the user must have the correct level of privilege

and rights identifiers.

The largest sharing of resources occurs when you implement your SOA architecture using

ACMS. Each node may run an instance of your ACMS application. As tasks are queued for

servers in that application, ACMS searches and finds the instance of the application with a

currently available server. If no servers are currently available, but the application running on a

given node hasn't reached its limit for that server (set by you or an operator) ACMS will start

another instance of that server underneath the application and dispatch the task to it. Likewise,

after servers have been idle for a certain period of time (set by you or an operator) it will begin

shutting down those servers, thus decreasing overall cluster usage and helping to maintain loadbalancing.


26/76


There is also a minimum keep alive server count which you control. It is important to

note that these servers will not stay alive forever. After a server has been idle for the period of

time which you set, ACMS will kill it off. If the resulting server count is below the minimum

keep alive value, it will start a new one. While that may seem odd to many readers, it is an

incredible feature. It promises your application will not come to a screeching halt due to hung

servers.

If your server initialization code has an issue with tramp data or some other programming

bug that causes it to go into sleep mode, it will only sleep as long as the idle server timeout

period. While it is true that this could mask a bug for you, ACMS errs on the side of application

stability. The last executable statement in each of your procedures (initialization, process and

termination) should be to write a single text line to the server log file. When you see the messagefrom the termination procedure in a log which doesn't have the message from the initialization

procedure, you know you have a problem and need to take corrective action.


27/76

Chapter 3

What is Wrong Today

3.1 We Sold Modules

My first programming job was at a DEC (Digital Equipment Corporation) VAR (Value

Added Re-seller). There were a lot of them back then. Every one of them seemed to have some

canned package they were selling along with the hardware. Most of them focused on the

basics of manufacturing: Order Entry, Shipping, Inventory Management and Bill of Materials

(BOM). Every one of these packages came in a customizable form. This means the base

package was so stripped you couldn't do anything but operate the most generic of businesses.

Why were they sold this way? Because the real money was in the modifications. A staff of

developers existed at each of the VARs to both work on the next version of the package and to

provide time & materials modification services to the clients. No customer bought the canned

version of the package. They all wanted it tweaked for their own business rules. Few, if any, of

the customers had their own programming staffs. Every customer had their own unique set of

business rules they wanted incorporated into the application.

The VARs sold their applications in modules. You could buy as little or as much as you

could afford. Start with Order Entry. When you decide you liked it, buy one or two moremodules. Eventually you ended up with the entire system. Of course, there were some gotcha's.

Part of the Inventory and Customer Management software had to be included with the Order

Entry modules. They simply couldn't work without it. Eventually, sales would talk most

customers into buying the holy trinity, Order Entry, Inventory and Customer Management. The

rules they needed would be implemented in those modules and only those modules. When they

came back to get additional modules, they had to pay for another round of modifications to

support the modifications already in place in what they bought the first time. Once you got a

customer to buy the holy trinity, it was a license to print money.


28/76

20 Chapter 3- What is Wrong Today

Customized systems have a caveat that vendors don't talk about even today. You are either

locked into that version forever, or you pay for the same modifications over and over again with

each new release of the product. There is no middle ground. Oh, yes, the billing rates being

charged for modifications to the new version of the package are higher than they were for the old

version as well. This problem exists even with the big vendors today. No matter how external

you try to make the rules modules, you still end up having to open up their code to hook in the

rules. As soon as the code has been opened, you are looking at having to make those same

modifications every time a new release comes out.

Some companies tried to ease this pain. I know of some payroll systems that used ANSI

COBOL with line numbers. They would ship modification source merge files to the customer

and make it the customers responsibility to ensure none of their modifications would be walkedon by these new lines. For small releases and fixes it worked. With major releases, you didn't

have a prayer.

3.2 We Are Still Selling Modules

The 1980s established a mind-set that applications should be sold by module. That mind-set

still exists today. When you move to one of the big vendors, you buy a module at a time. They

would like to sell you every product they have, but you simply cannot afford it. Some

companies have sought to eliminate the locked into a version problem by modifying their

business to fit the rules already built into the system they are buying.

I love watching technology commercials on TV, especially software commercials. From the

1970s to the early 1990s, the software you ran your company on was a strategic edge. You may

have sold the same products as your competitors, but you did business differently, that was why

customers came to you instead of them. Then the marketing arms...err...I mean the independent

analysts, got paid to promote the concepts of software as an expense and using the same

software as everybody else. The lemmings snapped into line. They started axing all of the

programmers they had which understood the business rules, buying canned packages from the

major vendors (as dictated by the independent analysts), and modifying their business to use only

the rules already coded into the package so they could upgrade. Now you sell the same products

as your competitors and use the same software to run your business in the exact same way.

Hello price war, goodbye stock options. Recently, I've seen some commercials on TV saying, if

you're just like everyone else, why do business with you? I guess the fallout nobody thought


29/76

Chapter 3- What is Wrong Today 21

about when resurrecting a 1970s business model is that 1970s business mentality would come

back with it.

In the 1970s, companies leased time on computer systems because they couldn't afford their

own. They paid to have customized applications installed on those systems so they could

continue to run their businesses the way they always had, just more efficiently. In 2007,

marketing types are trying to get customers to lease time on computer systems owned by

someone else, but still use the same canned generic business packages everyone else is using.

Does anyone else see the problem here?

3.3 The End of Modules

If the industry really is going to embrace Service Oriented Architecture (SOA) then the

module mentality has to die. You can still use modular development and stepwise refinement,

but the concept of an application module has to die. This is going to be a hard sell, especially to

the current leaders in the SOA field. They are trying to sell modules as a service. This is

destined to fail.

Some of you reading this might not be familiar with what I mean when I say module. I'm

talking about an application module. Let me try to break it down for you with an example.

Consider a Customer Management module for your typical manufacturing company. There

will be many components of this module. The bulk of these components will be accessed via the

Customer Maintenance screen or menu. The main or driving screen will allow you to

lookup/enter/modify all of the information for a main customer account. Additional screens will

allow you to setup secondary locations or alternate shipping addresses for this master account.

The accounting people will have their own screens where they adjust the customers credit and

status. Higher-end systems will also have a history table showing total dollars for the master

account and break it down into total dollars for each alternate address or secondary location.

There will also be a mailing piece using this information to inform the customer of various price

cuts and company news. This same piece will be used by accounting to send ever increasingly

nasty messages to customers who are past due.


30/76


All of the above would be a Customer Management module. There would be additional

pieces for businesses which offering rolling accumulated pricing discounts to customers.

(Rolling accumulated pricing discounts would be an additional percentage off for total sales. i.e.

$8 million in total sales gets you an additional 5%, $10 million an additional 8%, etc.)

The module mentality has worked for 30 some years in this industry, but it won't work in

SOA when your services are hosted by multiple vendors at multiple locations. This is the dirty

little secret behind the sales pitch for SOA. Once your data is on their system, it's not yours

anymore. You have to buy what they sell from that point on, and pay the price they ask. You no

longer have computers capable of hosting all this stuff. Even if you did you couldn't afford the

licensing.

3.4 How Do We Fix This?

When SOA is done in-house, it represents what the programmers were telling management

they needed all along. Management has been unwilling to bear the cost of it in the past, so I'm

doubtful they will bear the cost of it now. If they are willing to bite the bullet, it could be as

industry changing as the invention of the semiconductor chip.

To start with, the pieces in your existing modules aren't small enough. They need to be

much smaller than you would imagine. Every feature in that customer maintenance screen has

to be broken out into its own service that will accept a message and issue a response. You have

to standardize what those messages are and publish their format to all who will use them.

Let's look at the basic functionality. The very first screen for new customer entry usually

accepts the customer name, contact info, address, etc., prior to writing it to the database. It also

does something which goes unnoticed. It assigns a unique customer number to that customer.

You should split this into two services. The first will accept a request for a new customer

number and return it. The second will send a message with all of the new customer data. It

won't send it to an add customer service, it will send it to the update customer service.

Why? Because the services assigning the unique customer number will have to create the record

in your database, unless you create a separate table which contains only unique customer

numbers. The current module is most likely locking the customer number it has during the

entry process, or waiting to request it at the time of record creation. If you take the last

approach, you can have an add customer service, but it must be capable of returning the


31/76


customer number back in its response, or an error message if there was a problem with the add.

The update customer service gets a little trickier. In a traditional environment, the

modifying application reads the record and locks it while the user is making modifications. The

application generally has some form of built-in idle time out to kill off the lock if the user has not

entered anything for a while. This exists to prevent deadlocks in your database. The SOA

version cannot operate in this manner. When a user wants to update a customer record from

some Web screen, they have to send a request to the customer inquiry service to obtain the

record, then send a message to the update customer service for the updates to be written. The

message to update customer has to contain before and after images of the customer record.

Why does the update customer service have to get both copies of the record you ask?Because it has to have brains. The record was not locked while it was being modified. Another

user could have used the service to modify it. The update customer module has to perform a

field by field comparison of the before image with what is currently on the database to determine

if there were any changes. Most shops will attempt to be lazy at this point. They will simply

return an error and tell the user to retrieve again. The smart shops will apply the changed fields

if their before image matches the database current value.

There are some schools of thought which say you should only send the before and after

images of the changed fields to reduce network traffic. These schools of thought also tell you touse XML for the transfer. If you really are worried about network traffic, you don't use XML,

you transfer binary data as it is the most compact and secure. Binary data is certainly not in

vogue, until you talk to people who have to do encrypted transmissions.

You use XML when you need the widest possibility of acceptance. Know this: Every

service will have to parse and generate XML if you plan to go this route. You cannot have some

of the external services use it and some not when they are all part of the same business concept.

If the update customer service is going to send and receive XML, then the customer inquiry

and customer add service also must use it. They are all services related to the same businessconcept. Generating and parsing XML is a processing overhead. While there are free libraries

such as Xerces, their CPU and memory usage is not free.


32/76


We are now seeing what some refer to as a middle ground between secure encryption and

the XML users. The data contained between the XML tags is encrypted in a textual format. The

vendors of XML tools are trying to keep their products relevant in the world of identity theft.

When I see a file transmitted in this manner I am always reminded about the definition of an

elephant. An elephant is a mouse designed to government specifications.

When moving to an in-house SOA, you must make a decision about how many interfaces

you wish to allow. Is the bulk of your SOA going to be for in-house use only over a secured

network? If the answer is yes, you don't need XML. The token few services which the outside

world could access can use XML, but send it through some form of general server that is both on

the secured network and capable of bidirectional XML. The part of it that communicates with

the actual services you have hosted will be communicating using binary records or messages.This is by far the best implementation because you can ensure no Web page has direct access to

your customer database. SQL injection techniques simply don't work when the message being

sent is going to be parsed into a binary or fixed-width record that gets passed on. That server in

the middle will need some form of maintenance every time you add information to the outside

world, but if written correctly, it can ignore any new fields you don't allow the outside world to

see.

3.5 Today's SOA Camps

Currently there are two competing camps championing their vision of the Service Oriented

Architecture. The first camp is run primarily by the large package vendors. Their vision of SOA

is a return to the 1970s computing model. You pay only for the pieces you use, but you use it on

their machines. This means your only choice is what they sell, and they kind of like that model.

Our second camp is run primarily by the software vendors. Yes, they are promoting open-

source tools like Java, Ruby and Web 2.0. They are also pushing very expensive commercial

middleware with ever escalating support contracts. If they can no longer sell you the compiler,

they are going to sell you something you can't live without. Their view of the Service Oriented

Architecture is that you write thousands of little Web apps with the tools they vend to directly

access your databases from the Web. Yes, they think it is a great thing. Nothing better than

being able to access your customer's private data from anywhere on the Web. All of these little

servers will be scattered around the world and you will hook them together on a pay-per-use

basis. They will each do one little thing and return you the result. Forget about that Internet


33/76


being free stuff. Web 2.0 will be a completely commercial pay-per-use environment in their

mind.

Both of these approaches will have billions flushed down the toilet after them. Both of these

approaches will ruin lives. Neither of these approaches will create a successful Service Oriented

Architecture for your company. We've already discussed the module by module approach, so

let's take a look at the second approach.

PC and UNIX vendors can't get enough of doing what has been done before, putting a new

name on it or providing some other form of marketing spin. They have consistently repeated

development blunders that occurred during the evolution of the mainframe and midrange

computing market.

Mainframes were data islands, they didn't network, then midrange computers came out and

were able to cluster, so mainframes got better at networking. Initially PC's were data islands.

They didn't network. Then people started developing file servers so people wouldn't have to

hand walk floppies around, and they started networking.

Initially, mainframes and midranges didn't have databases. When data requirements

exceeded the capabilities of what they had, they got hierarchical databases. When the limits of

those were encountered, they got relational databases. Some companies now even have object-oriented databases. Initially, PC's did not have any form of records management system. It was

a roll your own situation. Then they got some stand-alone products like the various XBASE

products. Next they got a good server-based product, Btrieve. For the most part they seem to

have skipped hierarchical databases and went straight to either relational or object-oriented

databases now.

Initially midrange and mainframe computers didn't have any quick method of tossing up a

data entry screen or writing a report. You had to code it all by hand in either a 3GL or

Assembler. Next they got some report writers and screen generators. Then they got CommonData Dictionaries and 4GL tools. PC's initially didn't have anything useful along these lines

either. Then they came out with a bunch of development tool sets (not really 3GL and not really

4GL) and code generators that would access the database to generate data entry screens and

reports. They kind of skipped the Common Data Dictionary thing because there was little in the

way of organized code control or multi-tool development like midrange and mainframe

computers.


34/76


When DOS finally gave way to a GUI version of itself, then a re-engineered GUI, this

development tool set concept got recycled all over again. Report generators, screen generators,

you name it, all directly connected to your databases across the company network. Your

programs could now integrate with spreadsheets and a host of other things they couldn't do

before.

Enter the World Wide Web. After a period of time giving it away for free, it is being

redeveloped as Web 2.0, the commercial version. Make no mistake, while some parts of the new

Web will remain free, this is quickly becoming a pay-per-use business model. Guess what

concept is being recycled? You betcha! A whole bunch of tool sets and languages that will let

you directly (via commercial middleware) connect to your database from any Web page

anywhere in the world. All of your information and your customer's information exposed towhomever tries an SQL injection or other hacking technique first.

What is different between then and now is the network and the risk. Back around the time

frame of DR DOS 6.0 and Windows 3.1, networks didn't have a lot of external exposure. True,

some companies put password protected modems on the network to allow full access, but most

companies communicated over leased lines from location to location. The development tool sets

were used in-house by people who already had access to the data.

Don't forget, hackers are also pushing for the Service-Oriented Architecture. They want thevery same data you are obligated to protect, passed around the Web to utilize service components

which will be out there. If you take this to the logical end, it means that your customer credit

application screen would have an entry for Social Security Number and a button for Check

Credit Score. The SSN would be put into an XML message and passed to a server returning the

customer's credit score. One of three situations would have to occur;

1) your account and password would have to be stored on the machine to be included in

the message.

2) The user at the terminal would have to have this information and enter it every time,even though you make use of dozens of these different services on that same screen.

3) The raw SSN with all identifying information would have to be passed to the service

across an insecure Internet connection.

Any way you slice it, things sure don't look good for data security.


35/76


Opponents of what I say here will try to tell you I'm painting a worst-case picture for you

and it really won't be that bad. Well, as an IT professional I have to always look at the worst-

case scenario and defend against it.

The argument always offered up is you will link in things like a Google search to your page

or some other free service which has no security risk. While services like that are both useful

and familiar in this day and age, they don't completely fulfill the Service-Oriented Architecture

mantra.

Companies headed down the path of developing a Service Oriented Architecture must

develop with a single mind-set. All of the services they develop must be optically isolated from

the Internet. Any service available to the outside world must go through a middle server whichreformats messages between internal and external use. Never directly connect a Web page to a

database that has any data that isn't for public consumption anywhere inside it. Eventually, the

tool that made it so easy for you to toss up that page will have a design flaw or a programming

bug exploited that will give an outside hacker complete access to the database. Most of you will

have read about SQL injection techniques; they are pretty commonplace. There are also buffer

overruns and a host of other problems that will continue to surface in each new trendy tool and

be exploited by hackers. Because you cannot eliminate this problem, you must design it out of

your Service-Oriented Architecture plans.

XML seems to be the standard (for now) of how the external world wishes to communicate.

That's fine. Use XML for the messages coming into and going out of that server which optically

isolates your SOA components from the Internet. Use a completely different message format for

your SOA components. Put all of your XML parsing out on that optically isolating server.

Make it create internal consumption messages, an

the minimum you need to know about service oriented architecture

Documents