tp monitors and orbs chapter 5. 2 © jim gray, andreas reuter transaction processing - concepts and...

Aug. 2 Aug. 3 Aug. 4 Aug. 5 Aug. 6 9:00 Intro &

terminologyTP mons& ORBs

Logging &res. Mgr.

Files &Buffer Mgr.

Structuredfiles

11:00 Reliability Lockingtheory

Res. Mgr. &Trans. Mgr.

COM+ Access paths

13:30 Faulttolerance

Lockingtechniques

CICS & TP& Internet

CORBA/EJB + TP

Groupware

15:30 Transactionmodels

Queueing AdvancedTrans. Mgr.

Replication Performance& TPC

18:00 Reception Workflow Cyberbricks Party FREE

TP Monitors And ORBs

Chapter 5

2

© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999

Hardware

Basic Operating System (BOS)

Transaction Processing Operating System (TPOS)

Transaction Processing Services (TRAPS)

processors

process / thread address space scheduling IPC local naming protection

repository file system directory extents blocks

file security

IPC simple sessions (open/close, write/read) naming authentication

TRANSIDs server class request scheduling resource mgr. i/f. authentication

transaction manager log, context durable queue transactional file archive

transactional RPC transactional session RPC

operator interface configuration load control

prog.environ.

databases (SQL) disaster recovery resource managers flow control

distributed name binding server invocation protected user interface

Application

memorywires, fibers, switches

3


The Resource Manager Interface

Resource Manager

res. manager's own service interface

rmCall(...)

transaction management

other resource managers

rmCall(...)

administrative functions and callbacks for installing, starting, and scheduling a resource manager

response

functions

callbacks

invocation

callbacks(depends on application)

To and from TP-monitor and other system resource managers (used for transaction control)

4


Some Terminology

TP-Monitor: This term denotes an operating system extension that understands transactions.

Resource Manager: This is a system component that offers useful services to certain user groups and that protects its resources by transactions. An example of a very important resource manager is a database system.

rmCall: This is used as a generic call to any kind of resource manager. The call specifies the name of the resource manager, the operation that should be executed, and the parameters for the operation. If there is a database containing course information, such a call could look like this:

rmCall (“CourseDB”, “select Course-No, Ctitle from Courses, Offerings where Term = “Summer 1999” and ….”, ...)

Callback: When a module calls some other module and that module in turn invokes the module that called it, the technical term for that mechanism is: callback.

5


Some Interface Definitions

typedef struct /* handle for identifying the specific instantiation */{ /* of a resource manager */

NODEID nodeid; /* node where process lives */PID pid; /* process request is bound to */TIMESTAMP birthdate; /* time of server creation */

} rmInstance; /* identifies a specific server */

typedef struct /* resource manager parameters */{ Ulong CB_length; /* number of bytes in CB */

char CB_bytes [CB_length]; /* byte string */} rmParams;

Boolean rmCall(RMNAME, /*inv. of RM named RMNAME */ Bindld* BoundTo, /* expl. with bind mechanism */ rmParams * InParams, /* params to the server */

rmParams * OutResults); /* results from server */

6


RM Callbacks for the TM

Boolean rm_Prepare();Boolean rm_Rollback_Savepoint(Savepoint); Boolean rm_Commit(Boolean); Boolean rm_Savepoint();

(void) rm_UNDO(&buffer); (void) rm_Abort();(void) rm_REDO(&buffer); (void) rm_Checkpoint(); (void) TM_Startup(lsn);

7


Control Flow in Transactional RPCs

resource manager address space

rmCall(.....)

main portion of the resource manager

return(success);

rm_Prepare()

logic for commit phase 1

return(vote);

rm_UNDO(*log_data)

undo logic

return;

other callback entries

thread 1

transaction manager address space

Commit_Work(...)

two-phase commit protocol

return (decision);

Abort_Work()

abort logic

return;

thread 2

thread 2thread 3

thread 3

8


RM Calls vs. RM Sessions

Independent invocations: A server of class S can be called arbitrarily often, and the outcome of each call is independent of whether or not the server has been called before. Moreover, each server instance can forget about the transaction by the time it returns the result. Since the server keeps no state about the call, there is nothing in the future fate of the transaction that could influence the server’s commit decision. Upon return, the server declares its consent to the transaction’s commit, should that point ever be reached.

9



Invocation sequence: The client wants to issue service requests that explicitly refer to earlier service requests; for example, “Give me the next 10 records.” The requirement for such service requests arises with SQL cursors. First, there is the OPEN CURSOR call, which causes the SELECT statement to be executed and all the context information in the database system to be built up. As was shown, this results in an rmCall to the sql server. After that, the FETCH CURSOR statement can be called arbitrarily often, until the result set is exhausted. If it was an update cursor, then the server cannot vote on commit before the last operation in the sequence has been executed; that is, the server must be called at its rm_Prepare() callback entry, and the outcome of this depends on the result of the previous service call.

10



Complex interaction: The server class must remember the results of all invocations by client C until commit, because only then can it be decided whether certain deferred consistency constraints are fulfilled. Think of a mail server. The server creates the mail header during the first interaction, the mail body during the next interaction, and so on. The mail server stores the various parts of a message in a database. All these interactions are independent; the client might as well create the body first, and the header next. However, the mail server maintains a consistency constraint that says that no mail must be accepted without at least a header, a body, and a trailer. Since this constraint cannot be determined until commit, there must be some way of relating all the updates done on behalf of the client when the server is called (back) at rm_Prepare.

11


Begin_Work

client

S(.....);S(.....);S(.....);

Commit_Work

S1

S2

S3

server class S

Begin_Work

client

S(.....);S(.....);S(.....);

Commit_Work

S1

S2

S3

server class Sa) each call to S is independent of previous calls to S

result of invocation of S1result of invocation of S2

b) the result of each call depends on the result of the previous call

Begin_Work

client

S(.....);S(.....);S(.....);

Commit_Work

S1

S2

S3

server class S

S4result of invocation of S3

S4

results of all invocations needed for commit decision

c) each call to S is independent - except for some global integrity constraints that cannot be checked until commit and that may depend on the results of all previous invocations

Context Management - I

12


client

context

server S1context

session

context kept loc.

client

server S1

server S2

context

request

response

request context

contextresponse

client

context

server S1

server S2

context DBread/

write

request resp.

client

context

server S1

server S2

context

requestresp.

request resp.

a) Context is maintained by a communication session that makes sure that each subsequent request goes to the same server instance as the previous one.

b) Context is passed back and forth explicitly between client and server upon each request and reply.

c) The servers write context information to a database. This might be a private database to the server class or a context database provided by the TP-monitor.

d) All servers of the same class share a segment of read/write memory in which invocation context is kept. Synchronization on this memory is done by the servers.

Context Management - II

13


Types of Context Information

Client-oriented context: The solutions presented so far implicitly assume that this is the type of context to be managed. Typical examples are cursor positions, authenticated userids, etc.

Transaction-oriented context: Consider the following example: Client C invokes server S1, which in turn invokes server S3—all within T. After return from the service call, C invokes S2 (a different server class), which also invokes S3, but needs the context established by the earlier call to S3 from S1. The point here is that the context needed by S3 is not bound to any of the previous client-server interactions, but it is bound to the transaction as such. This leads back to the argument about the similarities between sessions and transactions in terms of context management. Examples of transaction-oriented context are deferred consistency constraints and locks.

14


Managing Context

Session management: If context maintenance is handled through communication sessions, then theTP monitor is responsible for binding a server process to one client for the duration of a stateful invocation.

Process management: Even if the TP monitor has no active responsibility for context management, it may use information about the number of existing sessions per server for load balancing. The rationale is that an established session is an indicator for more work in the future.

15


More Interfaces

typedef struct {rmInstance EndA; /* one end of stateful interaction */ rmInstance EndB; /* other end of stateful interaction

*/ Uint SeqNo; /* sequence no. to distinguish

*/ /* between stateful interactions

*/ /* among the same servers. */

} BindId; /* handle for a stateful interaction */

BindId rmBind(rmInstance); /* passes the id of client to which */ /* a session has to be established. */ /* gets a handle for the interaction

*/ /* among this server and this client. */

/* returns null if binding fails. */

Boolean rmUnbind(BindId); /* inverse to rmBind. */

16


More Interfaces

PID MyProcid(); /* returns ID of process the caller is running in*/ /* this is actually a call to the basic os */

TRID MyTrid(); /* returns the TA ID the caller is executing in */RMID MYRMID(); /* returns RMID of the RM that has called */RMID ClientRMID(); /* returns the RMID of the caller’s client */PCB MyProc(); /* returns a copy of the process control block */

/* descr. the process the caller is running in */TransCB MyTrans(); /* returns copy of the transaction ctl. block. */

/* describing the TA the caller is working for */RMCB MyRM(); /* returns copy of the resource mgr. control */

/* block desc. the RM that issued the call */

RMCB ClientRM(); /* like MyRM, but for the caller’s client */

17


Schema of the Transactional Objects

active transactions

resource managers

processes/ threads

sessions

server classes

m n

1

1n 1

1

n

n

2

n

1

resource managers participating in a transaction

transactions a resource manager is involved in

for each resource manager there is one server class

processes in a server classeach session has

two processes as its end-points

which transaction is process working for at the moment

which transaction uses a session at the moment

m n

address spaces a process can switch to

18


Name Binding for rmCalls

RMNAME address

name server

interface ptypemailer farnode.mailserver

order entryorder entryorder entry

snode1.prodsnode2.prodsnode3.prod

sqldb database.server

***

************

RMTable

ResMgrName rmid RMactive RMup ... RMPR_chain rm_entry[] ... rm_Savepoint rm_Prepare rm_Commit ...

******

DebitCreditDebitCredit

NearNode.1NearNode.2

cached part of name server

***DebitCredit NearNode.1***DebitCredit NearNode.2

sqldb database.server ***

ProcessIx busy NextProcess

<RMNAME, EntryName>Step 1: RM locally available ?

Step 2 (yes): Bind to that RMID

Step 2 (no): Find a node in the name server; send request via RPC; do local binding at that node

Step 3: Find an idle process

Step 4: Find entry point address

RPC to other node; there it will be local binding

Step 5: activate process at entry point address

19


The CORBA-Way of Saying It

20


OTS: The BIG Picture

Object Request Broker (ORB)

Object Transaction Service TA context

TAclient

1

2 3

3

4 567

TAobject

rec.object

resourceSub-TA res.

TA server Recoverable server

21


OTS Interfaces

1 A transactional client requests begin or end of transaction. If upon a begin request, a TA already exists, a sub-TA is started.

2 The client invokes operations from a transactional object. Such an object has no persistent state.

3 Requests can also be made at recoverable objects.

4 Registers a resource object with the OTS for commit coordi-nation; may initiate rollback.

22


OTS Interfaces

5 A resource understanding sub-TAs implements what is needed for completing a sub-TA, passing the locks to the parent, etc.

6 A resource participates in the 2PC, makes changes to persistent state durable, performs recovery if needed, etc.

7 A transactional object may initiate rollback.

23


Transaction Context

OTS maintains a transaction context for each active TA. It is associated with all transactional operations.

The TA context is an implicit parameter for all invocations of TA operations.

The context can also be passed as an explicit parameter.

This is required in environments where both the OTS style and the procedural (X/Open DTP) style of transactions are used in parallel.

24


The Transaction´s New Clothes

Boolean Debit (Amount amountRequested) {Account theAccount;Branch theBranch;Teller theTeller;History GlobalHistory;Current theCurrentTransaction;theCurrentTransaction.begin();

Amount balance = theAccount.query();if (balance >= amountRequested)

{ theAccount.withdraw(amountRequested); theBranch.withdraw(amountRequested); theTeller.withdraw(amountRequested); GlobalHistory.remember(amountRequested); theTeller.dispense(amountRequested); theCurrentTransaction.commit(); return True;

else ...

25


OTS Interfaces

Current denotes a pseudo object, the methods of which are executed within OTS and ORB.

Invoking begin on an object of class current creates a transaction context that is managed by the ORB until the TA either commits or aborts.

As a result of begin, the newly created context is associated with the client´s “thread of control”, which typically translates into “process”. Multiple threads (in different execution environments) can be associated with the same context at the same time.

26


Why Use Queues?

Load control: In case of temporary overload put requests into a queue for the server class rather than flood the system with new processes.

End-user control: Queue can maintain the output of asynchronous transactions for delivery until the user explicitly acknowledgs receipt.

Recoverable data entry: Requests put on server’s queue. Server application processes requests as fast as it can.

Multi-transaction requests (Workflow): Requests are processed by a server which passes results on to another server for further processing. This can go on for many steps. If none of the steps interacts with a user, they can all be executed asynchronously.

27


Workflow Management Using Queues

ClientA Server1 Server2

Server3 Server4 ClientB

might be identical

output queue of server 3 / input queue of server 4



output queue of client A / input queue of server 1

output queue of server 4 / response queue

of client B

28


Durable Queues for Asynchronous Transactions

client server

send request

send response

receive request

receive response

1. transaction2. transaction

3. transaction

29


Queued Transaction Processing

Request-reply matching: The system guarantees that for each request there is a reply—even if all the reply says is that the request could not be processed.

ACID request handling: Each request is executed exactly once. Storing the response is part of the ACID transaction.

At-least-once response handling: The client is guaranteed to receive each response at least once. As explained previously, there are situations where it might be necessary to present a response repeatedly to the client. This means the client must prepare itself for properly dealing with duplicate responses. The important thing about the at-least-once guarantee, though, is the fact that no responses will be lost.

30


Queuing Interfaces

Boolean send( REQUEST DoThis, RQID rqid, QUID ToQueue, QUID RespQueue);

receive(QUID RespQueue, RESPONSE KeepThat);

RESPONSE reReceive(QUID RespQueue);

Registering a client with a queue establishes a recoverable session between the client and the queue resource manager. This is another example of the stateful interaction between a client and a server. The queue resource manager remembers theRQIDs it has processed, and the client knows whether it can send a request to the server queue, or whether it has to wait for a response from its input queue.

31


A Sample Queue Manager

Boolean send(QAttrPtr); /* RM interface for enqueueing a request */{ /* for simplicity no checking if the requesting */

/* rm is in session with the designated queue*//* insert the parameters passed into relation */

exec sql insert into sys_queues values (:(QAttrPtr->quid), :(QAttrPtr->q_type),

:(QAttrPtr->from_rmid),:(QAttrPtr->to_rmid), CURRENT,:(QAttrPtr->rqid),:(QAttrPtr->keepthat),:(QAttrPtr->request_response), 0, NULL); /

return(sqlcode == 0);};

32


A Sample Queue Manager

Boolean receive(QUID from_there, QAttrPtr); { exec sql declare cursor dequ on /* def scan over Q */ select * FROM sys_queues where quid = :from_there

and delete_flag = NULL order by rqid ascending for update <some isolation clauses>;

while (TRUE) /* try until entry is found */ { exec sql open dequ; exec sql fetch dequ into :(QAttrPtr->QAttr);

if ( sqlcode == 0 ) /* found an entry */ { exec sql update where current of cursor dequ set delete_flag = ‘D’; exec sql close dequ; return(TRUE); };exec sql CLOSE dequ;wait(1);

}; };

33


Load Balancing

11.21.41.61.8

22.22.42.62.8

3

0 5 10 15 20 25 30 35 40 45 50

Response Time

Number of Server Processes (Concurrent Transactions)

T = 0.8 R = 0.9 c = 0.02

1

response time threshold

34


Thrashing

thrashing thresholdThroughput

Response Time

normal mode thrashing mode

35


Data Affinity

data partition A

data partition B

server class A

server class A

server class B

disk server

disk server

server class B

database server

database server

Node 1

Node 2

scheduling by TP monitor

scheduling by

TP monitor

RPC back to node 1 because data reside in partition A

request

36


The Authentication Problem

SQL- database

mail server

presentation service

mail server database system

process A process B process C

authid: localadm

authid: mailguyauthid: DBadmin

userid: myself

click browse function on behalf of : myself

request readmail on behalf of localadm on behalf of myself

select <attribute list> from <database> ... on behalf of mailguy on behalf of localadm on behalf of myself

tp monitors and orbs chapter 5. 2 © jim gray, andreas reuter transaction processing - concepts and...

Documents