intro to sam dbserver rewrite toolkit and components lauri loebel carpenter november 2003
TRANSCRIPT
Last year, if you recall…
DbCorbaClient / SamApi
(or, what I did on my Thanksgiving vacation)
Lauri Loebel Carpenter
December 10, 2002
Where did this lead?
• Complete IDL redesign• Infrastructure to support new concepts:
– Struct Wrappers– Automarshalling DbServerProxy– Automarshalling DbServantImpl– DbAware Objects– ProtectedInterfaces– Exception handlers, db utilities, etc.
IDL: Interface Definition Layer
• Defines anything and everything that needs to be known by two or more code packages.– Constants
– Data Structures
– Available Interfaces
• Allows communication between arbitrary programming languages on arbitrary platforms– Powerful, but error-prone and difficult to debug.
Marshalling Data is complicated; Unmarshalling is
worse!• Marshal: take my object, convert it to a byte
stream and send it to another application– On a different platform
– In a different programming language
• Unmarshal: receive this arbitrary bag of bites, and figure out what it is and what it is supposed to do, and send back some results.
• And handle all of the exceptions.
Bottom Line:
We really do not want arbitrary pieces of code to be marshalling and unmarshalling data structures.
Encapsulate! StructWrappers
• For each IDL data structure, create a representation that knows how to marshal/unmarshal itself.
• Perform all marshalling/unmarshalling in one place (dbProxy, implCreation)
• Treat exceptions as just another type of object to be marshalled/unmarshalled
Struct Wrappers in a nutshell
• initialize_fromPython(self, *args, **kwargs)• initialize_fromCorba(self, corbaObject)• pyRepr()• corbaRepr()• __repr__()• __str__()
Struct Wrappers: the tricky part: Many initialization signatures.
• Valid SamSizes: s1 = SamSize('10g') s2 = SamSize(s1) s3 = SamSize(0) s4 = SamSize('100', 'bytes')
• Valid RunDescriptors: r1 = RunDescriptor(runId=10)
r2 = RunDescriptor(r1)r3 = RunDescriptor(runNumber=12, runType='monte carlo')r4 = RunDescriptor(12, 'physics')r5 = RunDescriptor('physics', 12)
Automarshalling Proxies/Servers:
# get a proxy to the interface of interest:dbProxy = proxyMgr.getProxy('DataFile')
# create the wrapped object representing the# high level data structuretheData = SamStructWrapper('input', 'args')
# pass the high-level data structure to the method; # result will be a high-level data structure alsoresult = dbProxy().interfaceMethod(theData)
Db (and other) Proxies:
• Standard SamServerProxyManager utility for managing proxies
• Marshals all args (via corbaRepr())• Unmarshalls all return values (via
pyRepr())
• Client code needs ONLY high-level (auto-documentable) data structures
ServantFactory and MethodWrappers
• ServantFactory creates an instance of an Implementation
• Then takes that Implementation and wraps the LOCAL, PUBLIC methods in a ServantMethodWrapper
• ServantMethodWrapper unmarshals all incoming data (via pyRepr()), marshals all return values (via corbaRepr())
• ServantMethodWrapper also handles marshalling/unmarshalling of exceptions raised during execution of the method
Implementation Layer:
• Receives the high-level data representation• No CORBA marshalling/unmarshalling code to
distract the developer from SQL nightmares.• Returns another high-level object• ServantMethodWrapper marshals it• dbProxy unmarshals it…
• Server code needs ONLY high-level (auto-documentable) data structures
Conceptual Example:What the User Sees
from SamFile.SamDataFile import SamDataFilerawFile = SamDataFile(fileName='demo.raw', fileType='colliderImport', startTime='10-oct-2003', endTime='12-oct-2003', runType='physics data taking', runNumber=2001, firstEvent=1, lastEvent=10, eventCount=10, datastream='physical generic', dataTier='raw', filePartition=200, appName='datalogger', appFamily='datalogger', appVersion='0.0')fileId = rawFile.declareMetadata()
What the client code looks like (simplistically):
class SamDataFile(DataFileAttributes):
….
def declareMetadata(self):
proxy = self.proxyMgr.getProxy('DataFile')
fileId = proxy().declareMetadata(self)
self.setFileId(fileId)
What the Server code looks like(very simplistically):
class DbAwareSamDataFile(SamDataFile):
….
def declareMetadata(self):
self.validateMetadata()
fileId = self.dbCore.nextId('data_files_id_seq')
self.dataFiles.insertOne({'fileId':fileId,
'fileName' : self.getFileName(),
'fileSizeInBytes': self.getFileSize().getSizeInBytes(),
…. } )
Implementation: Summary
• Define data structure– Sam_idl/sam_corba_base/– Sam_common_pkg/SamStruct/
• Add methods– Client: calls dbproxy().method– Server: DbAware object implements the guts
of the method
Other DbServer Tools and Utilities:
• Protected Interface/Implementation– Samadmin, shifter commands
• nameAndIdUtility– Standard utilities for NameOrId objects
• relationshipUtil– Standard utilities for validating relationships
• DbServer ExceptionHandler– Raise ONLY what is declared in the IDL; notify
developer of any undeclared exceptions and convert them to InternalError
Other General Tools and Utilities:
• SamServerProxyManager– Maintains all proxies
• SamExceptionHandler– Exception auditor– formattedTracebackString– Additional information in argDict
Where to find more info:IDL
Sam_idl/
/sam_corba_base/Typedef.idl
/sam_corba_base/<dataStruct>.idl
/sam_db_srv/<interface>.idl
Where to find more info:Struct Wrappers
Sam_common_pkg/
SamCorba/SamIdlStructWrapperBase.py
SamCorba/SAMToSamStructCodeGenerator.py
Sam_common_pkg/
SamStruct/<dataStruct>.py
Where to find more info:Automarshalling
Sam_common_pkg/
SamCorba/DbServerProxy.py
Sam_db_srv/
src/ServantMethodWrapper.py
Sam_common_pkg/
SamUtility/MethodWrapper.py