data junction
TRANSCRIPT
I N T E G R A T I O N
Product Evaluator’s Guide
Pervasive Data Integration PlatformRelease 9.0
December 2009
2Product Evaluator’s Guide
Table of ConTenTs
PurPose of This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Pervasive: a ComPrehensive daTa inTeGraTion PlaTform . . . . . . . . . . . . . . . . 4adaPTers – daTa and aPPliCaTions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Pervasive daTa inTeGraTion and soluTion delivery modes . . . . . . . . . . . . . . 7 IT and EnTErprIsE applIcaTIon dEvElopmEnT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
EmbEddEd daTa InTEgraTIon for on-prEmIsE applIcaTIons or saas soluTIons . . . . . . . . . . . . 7
sErvIcEs dElIvErEd by sysTEms InTEgraTors and soluTIon provIdErs . . . . . . . . . . . . . . . . . . 8
Pervasive daTa inTeGraTion PlaTform – ProduCTs . . . . . . . . . . . . . . . . . . . . . 9Pervasive daTa inTeGraTor™ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 scalablE ETl and rEal-TImE daTa InTEgraTIon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
pErvasIvE daTa InTEgraTor — applIcaTIon InTEgraTIon . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
pErvasIvE daTa InTEgraTor — InTEgraTIng b2b gaTEways . . . . . . . . . . . . . . . . . . . . . . . . . . 11
pErvasIvE daTa InTEgraTor — ExTEndIng ThE EnTErprIsE sErvIcEs bus (Esb) . . . . . . . . . . . . 11
Pervasive daTa Profiler™ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Pervasive daTa inTeGraTion PlaTform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 dEsIgn Tools ovErvIEw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
ProcessDesigner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
ProcessDesigner–MessagecoMPonentFraMework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
PervasiveMaPDesigner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
oThEr pErvasIvE dEsIgn Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
exPressionBuilDers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
MetaDataDesigners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
PervasivestructureDscheMaDesigner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
PervasiveDocuMentscheMaDesigner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
PervasiveextractscheMaDesigner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
JoInIng mulTIplE fIlEs as a sInglE sourcE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
PervasiveJoinDesigner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
managEmEnT of dEsIgn obJEcTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
oPenMetaDatarePository . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
usEr InTErfacEs — accEss To ThE dEsIgn rEposITory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
rePositoryexPlorer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
rePositoryManager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3Product Evaluator’s Guide
Pervasive inTeGraTion enGines – ProduCTion and TesT environmenTs . . . . . 23 EngInE managEmEnT – pErvasIvE InTEgraTIon managEr™ . . . . . . . . . . . . . . . . . . . . . . . . . . 24
InTEgraTIon sErvIcEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
adapTErs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
GeTTinG sTarTed TuTorials — Pervasive daTa inTeGraTion desiGn Tools . . . 27 basIc pErvasIvE map dEsIgnEr TuTorIal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
overviewoFMaPDesigneruserinterFace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
map dEsIgnEr TuTorIal — crEaTIon of a basIc TransformaTIon map . . . . . . . . . . . . . . . . . 28
basIc pErvasIvE procEss dEsIgnEr TuTorIal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
overviewoFProcessDesigneruserinterFace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
basIcs: map dEsIgnEr, procEss dEsIgnEr and ExTracT schEma dEsIgnEr . . . . . . . . . . . . . . 36
addiTional samPles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37TraininG and serviCes oPTions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38suPPorT for QuesTions While revieWinG The ProduCTs . . . . . . . . . . . . . . . 38
conTacT InformaTIon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4Product Evaluator’s Guide
PurPose of This Guide
For first-time evaluators of the Pervasive Data Integration Platform
•Providebaselineintroductionofdataintegrationconcepts
•Brieforientationfordifferenttypesofsolutiondevelopmentanddeliverymodes
• Introductiontothedesignandproductionplatforms
• GettingStartedwiththeEvaluation–OverviewandSyllabusfortutorialsforthetwo most-useddesigntools:MapDesignerandProcessDesigner
Pervasive: a ComPrehensive daTa inTeGraTion PlaTform
Everyenterprise,nomatterthesize,hasinformationmanagementneeds:compliancerequirements(SOX,HIPAA),achievinghighlevelviewsofthebusiness(BI,MDM),consolidatingdatasegregatedinapplicationsilos(mergersandacquisitions).Andoftenthoseneedscallforadataintegrationsolutionorcomponent.Dataintegrationprojectsofferthepotentialtounifybusinessdataandenhancethevalueofinformationstoredinavarietyofsources:relationaldatabases,EDI,flatfiles,andXMLdocuments.Toremaincompetitive,companiesmustseamlesslyconnectwithcustomers,suppliers,andinternalbusinessunits–frequentlythesegroupsstoreandusedataindifferentformats.AcomprehensiveDataIntegrationPlatformdeliversawiderangeoftoolsanddataservicestoaddresstheever-changingvarietyofapplicationdataandintegrationprojects.Tobeeffectiveforbasicaswellascomplexintegrationprojects,acomprehensivedataintegrationplatformmustalsoprovideflexibilityandagilityformultiplesolutiondeploymentmodes.
Becauseofvendorinnovationand“bigpicture”understandingbycustomers,dataintegrationisbecominganecessaryaspectofITsolutions.Internally,companiesmustsolvemanybusinessproblemswithITsystemsthatwillrequireadataintegrationsolutionorcomponent.Likewise,externally-focusedapplicationandservicesvendorsalsoneedagileembeddeddataintegrationsolutionstoensureadoptionoftheirproductsandenabletheircustomerstomakethebestuseoftheapplications
Anotheraspectofadataintegrationplatformischoosingthemethodusedtoprovidethedataintegrationsolution.Thechoiceoftencomesdowntohand-codingorusingdataintegrationsoftware(theclassic“buildversusbuy”proposition).Productivitygainsareabigreasonthatmanycompaniesusedataintegrationsoftware.Inadditiontosavingtime,thedataintegrationsoftwareplatformprovidescommon,repeatablefunctionsandfeaturesthatsupportdataintegrationprojects,sothatcompaniesdon’thaveto“reinventthewheel”everytimetheydoanewproject.
Usingadataintegrationplatformmeansthatyouhavealloftheintegrationcomponentsinoneplace,documentedandmanagedforcontrol.Withhandcoding,youhavecodealloverthe place,withlittlemanagementandlittledocumentation.Productivityimproveswitheachprojectsincethesameplatformisusedoverandover,withreusabilityofpreviouslycreateddataintegrationcomponents.
Certainsortsofdataintegrationchallengesdemandasoftwareapproachmorethanothers.Whileaone-timedataaggregationprojectcouldbehand-coded,anysolutionsthatrequirefrequentupdatesandsynchronizationsofdataarerightfordataintegrationplatforms.
5Product Evaluator’s Guide
Forinternalsystemsaccessedbymanybusinessusers,orapplicationsthatwillbeexpandedovertimetoencompassmoredatasources,itmakessensetouseadataintegrationplatformtocreatesolutions.ForISVsthatneeddataintegrationsolutionsfortheirapplicationsorhosted/SaaSsolutions,embeddingcomponentscreatedbyacomprehensivedataintegrationplatformisremarkablycosteffective.
WhilethereisquantitativeROIformanysolutionscreatedwithdataintegrationsoftware,therearesignificantqualitativecostbenefitsaswell.The“soft”benefits,particularlyregardingreusabilityandtransparencyofdesign,areoftencompellingenoughtoencourageinclusionofdataintegrationplatformsforbothinternalITsystemsandforapplicationsofallsorts.
Data integration solutions include:
• Integrationandsynchronizationofbackendsystems
•ETLfordatawarehouses
•B2Bexchangesparticularlyforsupplychainandforprocessingtransactionsforbackendsystems
•Migrationsfromlegacyapplications
•Extractionofdatafromunstructuredcontentsourcesandintegrationwithotherdatasources
•DataintegrationservicesforService-OrientedArchitectureinfrastructures
•Profilingdatasourcestoidentify“dirtydata”andreduceriskofinclusioninproductionsystems
• Industryspecificsolutionsforhealthcareandfinancialservices
• Embeddedcomponentsforalloftheaboveforon-premiseapplicationsandhosted services/SaaS
Migrate Extract Interface Exchange
Data Integration Business Integration
SaaSOn Premise
ETLData Files
PortalReports
DataWarehouse
OLD
Flat FilesExcel
OutlookLegacyOther
NEW
Hosted appsPackaged appsCustom apps
New databaseOther
EAIIntra-Company
ESB CRM
SAP SaaS SAP SaaS
PartnersB2BiHub
Inter-company
Data Quality Assessment
Profiling
RESULTS:ReportsTrend AnalysisClean and Dirty Output Files
JOINS
MULTIPLE DATA SOURCES
Access
ASCII
Oracle
COBOL
Data Integration scenarios that are addressed with Pervasive’s data integration platform
6Product Evaluator’s Guide
adaPTers – daTa and aPPliCaTions
ThePervasiveDataIntegrationPlatformsupportsmorethan150factory-suppliedadapters.MostofthePervasivedataadaptershavebeenwrittentotakeadvantageofnativeAPIsandindustrystandards.Manyoftheadaptersareavailableformultipleplatforms.
Databaseadapterssupportarangeofmodesfromclient/serveraccess,localdatabaseaccess,multimodewhereasingledatastreamcanberapidlyspreadacrossmultipletablesinatargetdatabase,andmass-insertforhigh-speedbulkloadingofdata.
AccessAccess 97Access 2000Access XPAlpha FourASCII (Delimited)ASCII (Delimited) JoinASCII (Fixed)
ClarionClipperCLF WebserverDataEaseDataFlexDJ Document SchemasDJ Log File DatadBASE II, III (III+)
dBASE IVdBASE V (IDAPI)DialogDIFEnableExcel 95, 97Excel 2000Excel XP
Folio Flat FileFoxbaseFoxProLDIFLotus 1-2-3Lotus NotesLotus Notes 5Lotus Notes Structured Text
Lotus WorksMac ASCII DelimitedMagic PCMAILERS+4 (dBASE)Microsoft Word (Mail Merge)Paradox v5 (IDAPI)Quattro Pro Windows v5Rich Text Format (RTF)
Unicode (Delimited)Unicode (Fixed)WordPerfect 6.0 (Mail Merge)Wordstar (Mail Merge)WordPerfect 4.2/5.0Visual FoxProVisual dBASE 5.5
Oracle 7.x Oracle 8.x Oracle 8.x Multimode Oracle 9i Oracle 9i Multimode Oracle 10gOracle Direct Path Oracle SQL Loader Pervasive.SQL Pervasive.SQL v8 Pervasive.SQL v8 MultimodePervasive PSQL v9Pervasive PSQL v10PostgreSQL PostgreSQL (ODBC 3.x)
Btrieve v5 Btrieve v6 Btrieve v7 Btrieve v8CA dB-FastC-ISAM CloudscapeCWM DB2 7.2 Univ. DB DB2 7.2 Univ. DB Multimode DB2 Loader DB2 UDB Mass Insert DB2 (ODBC) DB2 Univ. DB DB2 Univ. DB Multimode
eXcelon eXcelon XIS 3.0Hitachi Hi-RDBIDAPIInformix DB LoaderInformix (ODBC) Informix Online DS Informix/SE Ingres (ODBC 3.x)Interbase(IDAPI)JDBCMUMPS (ODBC) MySQLNonStop SQL/MX ODBC 3.x
ODBC 3.x Mass Insert ODBC 3.x Multimode ODBC 3.5 ODBC 3.5 MultimodeRBase 4.5 (ODBC)Red Brick Scalable SQLSQLBaseSQL Script SQL Server 6.x SQL Server 7 Mass Insert SQL Server 7 (OLEDB) Multimode SQL Server 7 (OLEDB) SQL Server 2000
SQL Server 2000 Mass Insert SQL Server 2000 Multimode SQL Server 2005 SQL Server 2005 Mass Insert SQL Server 2005 MultimodeSQL Server BCP SQL Server Mass Insert SQL Server (ODBC)Sybase 11.x Sybase 11.x Mass InsertSybase 11.x Multimode Sybase 12.x Sybase 12.x Mass Insert Sybase 12.x Multimode Sybase BCP
Sybase (ODBC) Sybase SQL Anywhere Sybase SQL Anywhere 6 Sybase SQL Mass Insert Sybase SQL 11Sybase SQL 11 MultimodeTeradata (Fastload)Velocis (ODBC 3.x) Watcom SQL v5 Watcom SQL (ODBC)
AccountMateACT! for WindowsADABAS (NatQuery)Apache CLF Champion Acct. CYMA Acct.GoldMine
Great Plains DOSGreat Plains UNIX/MacNavision Financials Macola Acct.Market Force Microsoft Dynamics CRMMicrosoft Dynamics GP
Microsoft IIS ELFNetSuiteOracle ApplicationsOracle Siebel CRM On DemandPayChex DJFPeachtree Acct.
Personal Librarian Pivotal CRMPlatinum Acct.RealWorld Acct.Remedy ARS Sage CRM SalesLogixSage MAS 90/200
Sales AllySalesforceSAP IDocSAP BAPISAS SBT Acct.Solomon Acct.
SPLUS SPSS StatisticaSYSTATUSMARC
BizTalk XMLEmailEmail AttachmentsErwinFF QueueFile Transfer Builder
File FolderFTPHTTPHTTPSHTMLJMS
LDAPMSMQ (Extractor)MSMQ (Fixed ASCII)MSMQ (XML)MQSeries (Extractor)MQSeries (Fixed ASCII)
MQSeries (XML)Oracle AQOracle BPELProgress Sonic ESBSGMLSOAP
TCP/IPURLURIWebSphereXMLXML Schema
XML DTDXML XDR
ACORDEDI (EDIFACT)EDI (X12)
HCFA 1500-NSFHIPAAHL7
HR-XMLMIBNCPDP
NSFSWIFTText (Delimited – EDI)
Text (Delimited – EDIFACT)Text (Delimited – HL7)TRADACOM
UB92-NSF
ACUCOBOL (ODBC 3.x)ACUCOBOL-GT v7BAFBinary
Binary Line SequentialCOBOL 01COBOL CopybookCOBOL Flat File
C-tree PlusC-tree 4.3Fujitsu COBOLISAM
Micro Focus COBOLQSAMSequential BinaryTape Drive Sequential
VSAMVariable Sequential (MVS)Variable Sequential (Record-V UniKix)
Variable Sequential (SyncSort)Variable Sequential (UserDefined)
7Product Evaluator’s Guide
Pervasive daTa inTeGraTion and soluTion delivery modes
iT and enTerPrise aPPliCaTion develoPmenT
PervasiveDataIntegrationPlatformtoolshavelongbeenutilizedbytraditionalITdepartmentsforinternalsoftwaredevelopmentrequirements.ThePervasiveDataIntegrationPlatformisavaluablepartofanysoftwareecosystemformanyenterprises,frommid-sizedcompaniesontolargercorporations.
Therehasbeenincreasedactivityintheavailabilityofdevelopertoolstoworkwithagiledevelopmentenvironmentsandtotakeadvantageofopensourceproductivitytools.Manyapplicationsforinternalusearenowbrowser-based,astraditionalsoftwaredevelopmentassimilatesnewservicesfromtheWeb.Themainpointistospeedupthecreationofmuch-neededbusinesssystems,whileeasingtheloadonIT,whetheritisthroughtheenablementofbusinessuserstocreatetheirownapplicationsascompositesofexistingsmallappsandwebservices,ortheproliferationofthesamemethodologyadoptedbyITandenterprisedevelopers.OpenSourcedeveloperplatformsincludeEclipse,AJAX,Java,Linux.ThePervasiveDataIntegrationPlatformhaskeptpacewithnewdevelopertrendswithoptionsforjustaboutanyprojectarchitecture.
Vendorsarealsoopeningupapplicationplatformsforextensibility.SaaSvendorSalesforce.comhasestablishedAppExchangeastheself-billedfirston-demandapplication-sharingservice.Mid-MarketERPandCRMvendorNetSuitehasreleasedSuiteFlex,adevelopmentplatformthatenablesthecreationofthird-partyverticalapplicationswithinNetSuite,aswellasend-to-endbusinessprocesscustomizationforanyend-usercompany.Hereagain,ITandEnterprisedeveloperswillencounterextensivedataintegrationneedsworkingthroughapplicationplatformextensibilityandshouldlooktothePervasiveDataIntegrationPlatformforexactlywhattheyneedtocompleteextensibilityprojects.
Foranynewdeveloperplatformtrends,PervasiveDataIntegrationwillcontinuetobeanimportantandagilepartofanynewsoftwareecosystem.Akeypartofcompositesolutionsistheexistenceofaservice-orientedarchitectureasabackbonefortheenterprise.PervasiveDataIntegrationhasthetrackrecordthatprovesitscleanintegrationforsucharchitectures.Reusability,flexibilityandeaseofintegrationwithoverallsolutionrequirementsmakethePervasiveDataIntegrationPlatformapowerfulpartnerforanyITsolutiondevelopmentproject.
EmbeddedintegrationoptionsforITandEnterpriseapplicationdevelopmentareessentialforenablingtheuseofsomanynewdevelopmentoptions.Dataintegrationcomponentscanhelpsimplifybuildinginternalsolutionsbydevelopersforanycompany.ThePervasiveDataIntegrationPlatformhasoneoftheleadingembeddabledataintegrationscapabilitiesintheindustry.
embedded daTa inTeGraTion for isv’s WiTh on-Premise aPPliCaTions or saas soluTions
Vendorsofferingsoftwaresolutionshavemanydeliverymodeoptionsavailabletothem,fromtraditionallicensedon-premiseapplicationstohostedandSaaS(softwareasaservice)subscriptionofferings.Frequentlyvendorsoftwareapplicationsandservicesalsohaveseriousdataintegrationneedstodeliveracompletesolutiontotheircustomers,whethertomigratefromlegacyapplicationsortosynchronizedatawithotherbackendsystems.
Thecustomersofapplicationvendorsalwaysconsiderhoweasilynewapplicationswillintegrateintotheircurrentinfrastructure,whendecidingwhichtechnologysolutionstobuy.Thesolutionproviderwhooffersthebestmethodologywilllikelywinthebusiness.AsRobertAndersonofGartnerGroupnoted,“Anewmarketisemergingthatcandelivercost-effectiveandeasy-to-useintegrationsolutionsratherthantheoldercomplexandhigh-pricedonesthathavetraditionallyreigned.ISVs,ASPsandintegratorscandelivercompetitiveadvantagebytakingadvantageofembeddingthesesolutionsintheirofferings.”
8Product Evaluator’s Guide
PervasiveDataIntegrationprovidesacomprehensivesolutiondevelopmentplatformtomeetthemanyneedsfordataintegrationwithinapplicationsandservices:easingmigrationfromlegacyapplications,helpingcustomersintegrateapplicationswithintheenterprise,tradingdatainB2Bexchanges,providingintegrationservicesinSOAimplementations.
Anembeddeddataintegrationcapabilitymustbelightweightenoughtofitneatlywithinanapplication,bothtechnicallyandeconomically.Agilecustomersolutionsdemandasingleembeddableandcustomizableintegrationcomponentthataddressessize,scopeandthestrategicnatureoftheintegrationtaskathand.Itmustbeflexible,abletorespondquicklytothechangesthatinevitablyhappen.PervasiveDataIntegrationfulfillsthoserequirementsforapplicationvendorsof all sorts.
serviCes delivered by sysTems inTeGraTors and soluTion Providers
AswithITdevelopers,systemsintegratorsandsolutionprovidershavediscoveredtheinvaluablepowerofthePervasiveDataIntegrationPlatformwhencreatingapplicationsfortheirclients.ThesebusinessesalsohavebenefitedfromactivepartnershipswithPervasivetodeliverexactlywhattheirclientsneedfordataintegration,nomatterthesolutiondeploymentmodeorthecomplexityofintegrationrequirements.
Businessesprovidingservicestootherbusinessesrequiretoolsthatpromoteeaseofuseandfastbutreliableimplementations,andthatareadaptabletoawiderangeofprojectneeds.Teamsresponsiblefordataintegrationprojectswanttocontrolcosts,shortentimelines,andminimizerisks.Changingbusinessrequirements,theneedtoconsumedatafromlargenumbersofinternalandexternalsources,andovercommitteddeveloperresourcesposesignificantchallengeswhencrafting solutions for clients.
Theabilitytohandlelargeandgrowingvolumesofdiversedatathroughparallelprocessingonlow-costcommodityhardwareisessentialforanincreasingnumberoforganizations.PervasiveDataIntegrationletsorganizationsstartwithaffordable,definedintegrationscenarios,thenrapidly,quicklyandeconomicallyscaleprocessesupintolarge,diverseenvironments.PervasiveDataIntegrationhelpsorganizationsadeptlymanagereal-timeorbatchintegrationinanysortofproductionenvironment.
Followingthenewtrendsofon-premiseapplicationandSaaSvendors,systemsintegratorsandsolutionprovidershavethefreedomtoimplementPervasiveDataIntegrationprocessesinmanydifferentsortsofmodes:asastand-alonetooltotaketoeachclientengagement,embeddedinothersolutionsorservices,andinlocalordistributedprocessingarchitectures.SystemsintegratorscanusethePervasivedataintegrationplatformtodecreasedevelopmentcostwhilespeedinguptimetosolutionsthatcallfordataintegrationcomponents.
Atclientsites,PervasiveDataIntegrationisavailableforadhocsituationsthatariseduringprojectimplementationsofallsorts.OnceateamofsystemsintegratorsorsolutionprovidersgainexperiencewiththePervasiveDataIntegrationPlatform,thevalueofthePlatformgeometricallyincreasesintermsofreusability,versatility,easeofsolutioncreation,speedtoeachnewsolution.
Certainrepeatableprocessescreatedbysystemsintegratorseventuallybecameproductizedasapplications.Theoptiontoembeddataintegrationcomponentsinsuchproductizedapplicationsgreatlyincreasesthevalueofsuchcompositesolutionstothecustomersofthesystemintegrator.Thebusinessofmanysolutionprovidersandsystemsintegratorsisdesigningandimplementingintegratedsolutionsthatdramaticallyimprovetheircustomers’abilitytoachievebusinessgoals.SystemsintegratorswhoadoptthePervasiveDataIntegrationPlatformdelivertop-notchend-to-end solutions to their customers.
9Product Evaluator’s Guide
Pervasive daTa inTeGraTion PlaTform – ProduCTs•PervasiveDataIntegrator–datamigrationsandextractions,businessdatainterfacesandexchanges
•PervasiveDataProfiler–extendandcontrolthevalueofdata
Suitedtobothlarge-scaleproductionsystemsandsmaller,project-orienteddatamovementtasks,PervasiveDataIntegratorProfessionalEditionenablesaccesstovirtuallyanydata–includingunstructureddata.WithPervasiveDataIntegrator,transformdatasources,thenloadthe outputintooperationaldatastores,datawarehouses,datamartsorothertargetsystems. PervasiveDataIntegratorisdesignedtodeliveraflexiblesolutionthatcanautomateanydatamovementtaskonanevent-drivenorscheduledbasis.
sCalable eTl and real-Time daTa inTeGraTion
Packagedwithfunctionalityforaggregatingmultiplesourcesofdataforquick-hittingdatamartsaswellashigh-performancedatawarehouseloading,thePervasiveDataIntegratordevelopmentenvironmentenablesthedevelopertoquicklydefineandmaintaineventhemostcomplexautomatedExtract,TransformandLoad(ETL)projects.
Keyfeaturesinclude:
• Integrationofdatafrommultipleapplicationsandoperationaldatastores
• Aunifiedtoolsetformanagingawidearrayofdataintegrationprojects
• Graphicalmappinginterfacetoshortenlearningcurves,speedtimetoresultsandreduceprojectcosts
• Abilitytohandlethemostcomplextransformationrequirementswithouttheneedforauserexittocallexternalcode
• Supportfordesignandpackagingofmulti-stepprocessesforautomatedexecutionbythePervasive Integration Engine
• Logging/rollbackintheeventaprocessendsunexpectedly
• “Upsert”tospeedefficiencieswheninserting/updatingdataduringhigh-speedoperations
• Multi-threadedintegrationenginesforscalinglargedataloadprocessingacrossmultipleCPUswithinnarrowtimeframes
• Distributedmodelallowingintegrationprocessestoresideatthepointsofintegration,creatingascalable,flexibleintegrationinfrastructure
• Rulesdefinitionforcomplexdataaggregationandtransformation,allowingone-passwritingtomultipletargets
• Connectivitytoallmajordatabasesandbulkloaders,alongwithmassinsertand multi-tablewriters
• ConnectivitytocommonCRM,ERPandotherapplications
•ConnectivitytolegacybinaryandCOBOLsources
ThefocusofPervasiveDataIntegratorUniversalEditionisonapplicationandmessage-basedintegrationprojects.Withdirectconnectivitytoenterpriseapplicationsandlegacydatastores,alongwithsupportforindustrydataexchangestandards,PervasiveDataIntegratorUniversalEditionhasaninfrastructurebuiltformessage-basedortransaction-basedprocessing.Agraphicaldesign
10Product Evaluator’s Guide
environmentallowsdeveloperstoincreasetheirproductivitywhilesolvingcomplexintegrationchallengeswithinbusinessprocessesanddataflows.
Whetherdesigningintegrationprocessesthattieinternalbusinessapplicationstogetheror automateelectronicdataexchangesbetweentradingpartners,PervasiveDataIntegrator UniversalEditionincludesnumerousfoundationfeaturesthatsimplifycreatingandmanaging flexibleintegrationsolutions.
Keycapabilitiesinclude:
•Real-time,event-drivenandmessage-basedintegration
•Continuousintegration
•Nativeconnectivity
•Point-to-pointapplicationintegration
•Legacyconversion
•Businessprocessdesign
•Emailqueueforauto-notifications
Pervasive daTa inTeGraTor — aPPliCaTion inTeGraTion
CRM and ERP Integration through On-Premise Application and SaaS Solution Delivery ModesPervasiveapplicationintegrationsolutionsoffersupportforEnterpriseServiceBus(ESB),ServiceOrientedArchitecture(SOA),andpoint-to-pointintegrationarchitectures.PervasivesolutionsareidealforintegratingCustomerRelationshipManagement(CRM),accountingdata,andEnterpriseResourcePlanning(ERP)—includingleadingSoftwareasaService(SaaS)offerings.
Pervasiveapplicationintegrationsolutionsfeaturereal-time,event-driven,andmessage-basedinfrastructureforintegrationofmultipleapplications,bothwithinandoutsidetheorganization.Designandimplementamessagingsolutiontomeetsolutionneeds,buildmessagequeueon-rampsandoff-rampsforanyESB,orenablepoint-to-pointexchangeofXML,SOAP,text,orothermessageformatsbetweenapplications.Pervasiveapplicationintegrationsolutionsaretransport-independentandinterfacewithIBMWebSphere,MSMQ,OracleAQ,oranyJMS-compliantqueue.Pervasivesolutionssupport“listening”tofilefoldersandFTPfoldersforthearrivalofdata,andcanrun as a service for event-driven integration.
Keycapabilitiesinclude:
• ProcessDesignerinfrastructurethatsupportsIBMWebSphere,MSMQ,OracleAQ,andanyJMS-compliantqueueandisalignedtocreateandmanageintegrationswithinbusiness-processflow
•Real-time,event-based,andcontinuousprocessing
• Distributedintegrationmodelthatallowsintegrationprocessestoresideatthepointsofintegration,creatingascalableandflexibleintegrationinfrastructureforuseinSOA
•High-speedenterprisedatabaseconnectivity,includingmulti-modeandmass-insertoptions
•Validationoftheaccuracyofmessagescyclingbetweenapplications
• Abilityforalldataobjectsandmessagestobehandledinbufferedmemoryforhigh-performanceprocessing
11Product Evaluator’s Guide
Pervasive daTa inTeGraTor — inTeGraTinG b2b GaTeWays
PervasiveDataIntegratorUniversalEditionfullysupportstherapiddesignandmaintenanceofprocessesthattranslateelectronicdocumentsandmanagetheirflowamonginformationsystems,includingsystemsoftradingpartners.
Keycapabilitiesinclude:
• Sophisticatedmessageprocessingthatallowselectronicdocumentstobequicklyread,validated,andtransformedaccordingtotheprocessdesign
•StandardschemasavailableforEDI,EDIFACT,FIX,ACORDXML,HIPAA,andothers
•Richconnectivitytoapplicationsanddatabases,industrytradestandards,andanyXMLformat
• Messagequeueandfilefolderqueuecomponentsforreal-timeprocessing,aswellasFTPqueue“listener”fordataarrival
• ProcessDesignermessageintegrationforcontinuousandscheduleddocumentretrievalfromfileandFTPfolders
• DocumentSchemaDesignersimplifiedmodificationofstandardschematomeetrequirementsdefinedbytradingpartneragreements
Pervasive daTa inTeGraTor — exTendinG The enTerPrise serviCes bus (esb)Especiallysuitedasanextensibleservicecontainerformessage-drivenESBandSOAenvironments,PervasiveDataIntegratorUniversalEditionintegrationprocessesenabledeveloperstoquicklydefineandorchestratemessageprocessingandroutingbetweenapplications
Keyfeaturesinclude:
• Richdataconnectivityenablinginformationflowbetweenawiderangeofenterpriseapplications,includingconnectorsforSAPandothermajorERPsystems
• Morethan150adapterswithoptionsformulti-tableandmassinserttoawiderangeofenterpriseclassapplicationsanddatabasesincludingOracle,DB2,Sybase,andSQLServer
• AdapterandmetadatasupportforindustrystandarddataexchangeformatssuchasEDI,XML,andHIPAA
• RemoteinvocationofintegrationprocessesusingWebServices,JCA,Message-DrivenBean,andJavaAPIstandards
•Javacomponentframework,providingdeveloperswithhighextensibility
• ProcessDesignerinfrastructuresupportingIBMWebSphere,MSMQ,andany JMS-compliantqueue
•Real-time,event-driven,andcontinuousprocessing
• Distributedintegrationmodelthatallowsintegrationprocessestoresideatthepointsofintegration,therebycreatingascalableandflexibleintegrationinfrastructureforuseinESBorSOAprojects
•Validationofmessagescyclingbetweenapplications
• In-memoryhandlingofdataobjectsandmessagesenablinghigh-performanceprocessing
•Web-basedmanagementconsolesupportusingPervasiveIntegrationManager™
12Product Evaluator’s Guide
PervasiveDataProfilerisavailableasacomplementaryandessentialenhancementtoPervasiveDataIntegratorforconstantmonitoringofdatainanysizeenterprisetoidentifydataqualityrisks.PervasiveDataProfileralsofunctionsasastand-aloneapplicationthatmaybeusedwithanyotherdatamanagementtoolsanddevelopmentinitiatives.
PervasiveDataProfilerworkstounlockthesecretsofcontenttoidentifydatathatisinaccurate,incompleteormissing.Therighttimetofind“dirtydata”foranydata-intensiveprojectsisduringprojectdesignphase.DataProfilercanbeutilizedcontinuouslyduringprojectdesigntime,andthenincorporatedintoproductionprocessestotestdataquality.Dataproblemsidentifiedduringprojectdesignphasescostconsiderablylesstoremediatethaniffoundlaterinprojecttesting.And,ofcourse,when“bad”dataiscarriedthroughproductionprocesses,thecostcanbetremendous:bothindirectremediationcostsandinlossofbusinessduetodecisionsmadeagainstfaultydata.
Versatileandeasytouse,PervasiveDataProfilerenablesyoutoproactivelyauditalltypesofdataandautomatetestingagainstever-changingbusinessneedsandcomplianceregulations.Iteliminatesthecomplexity,risk,andexpenseassociatedwithtraditional,andusuallymanual,meansof assessing and validating data.
•Assessdataacrossmultipledataplatforms(150+adapters)
•Quarantinequestionabledatauntilitcanbecleansed,reducingreworklateron.
• Auditdataformatsandapplicationsourcesinplacewithouttheneedforintermediate data storage.
•High-performanceparallelprocessingengineallowsfullauditsoflargetransactionaldatasets
PervasiveDataProfilerenablesorganizationstoimproveinternalcontrols,increaseITproductivity,avoidcostlyrework,andincreaseROIaroundapplicationsandbusinessprocessesthatareonlyasgoodasthedatauponwhichtheyarebased.
13Product Evaluator’s Guide
Pervasive daTa inTeGraTion PlaTform
ThearchitecturebehindthePervasivedataintegrationplatformconsistsofaunifiedsetofdeveloper-focusedtoolsforpromotingauthoringefficiencyandmaximizingcodereuseacrossprojects.Thedataintegrationplatformmaybeseparatedintofivedistinctparts:
Process Designer Process Flow Logic
Map Designer Mapping Extraction Rules Transformation
Data Profiler Data Auditing Data Analysis
Integration EngineScalableEmbeddableMulti-Threaded Processing
ScriptableBatch and transaction orientedTotal coss-platform compatibility
Small footprintHigh throughputAccepts streaming data
Integration ServicesEngine isolationJ2EE, SOAP integrationSecurity and authentication
Schema DesignerStructuredSchemaDesigner
ExtractSchemaDesigner
DocumentSchemaDesigner
Join Designer Heterogeneous Joins Join Engine
Clients SOAP JCA, RMI, MDB
SOA Enable
RUNTIME
DESIGN ANALYZE
MANAGE
Repository Explorer Manage workspaces and design repositories
Repository Manager Impact Analysis Reporting Packaging
Integration Manager Deploy Scheduling Reporting Monitoring
Command LineAPI’s – COM / JAVA / C
XMLDesign Repository
DJAR Files
Pervasive Data Integration Platform Architecture
Comprehensive Design Tools–ThePervasiveIntegratedDevelopmentEnvironment(IDE)suppliesdeveloperswiththedesigntoolstheyneedtobuildeffectivemulti-stepintegrationprocessesanddatatransformationsaccessinghundredsofdatasourcesandtargets.
Design Repository for Management and Deployment–Therepositoryisanopen,XML-basedfilesystemthatadeveloperusestotrackandstoreworkacrossmultipleprojects.Avarietyofuserinterfacesassistdeveloperswithaccesstothedesignrepository.
Data Analysis–PervasiveDataProfilerisessentialforauditingalldatasourcestoidentify“dirtydata”inordertoeffectrecommendationsforremediationandriskmanagementbeforethosedatasourcesenterproductionsystemsintheenterprise.
Runtime Engines for Production and Testing–RuntimeenginesprovidetheprocessingpowertoexecutemapsandprocessescreatedbydevRuntimeenginesprovidetheprocessingpowertoexecutemapsandprocessescreatedbydevelopers,independentofthedesignenvironmentforautomatedproductionpurposes.Enginesmayberunlocallyforquickaccesstoviewdataandtestdesigns.Inproduction,enginesmaybedeployedtosupportconfigurationsforscaling,distributed architectures and embedding.
Integration Services–forService-OrientedArchitecturesandWebservices
14Product Evaluator’s Guide
desiGn Tools overvieW
Process DesignerPervasiveProcessDesignerhasanintuitiveinterfaceavailablefor“drag-and-drop”integrationprocessdesign.Usingsimpleflowchartsymbols,ProcessDesignerlinksmulti-stepintegrationprocessestogetherinasingle,automatedintegrationtask.Keyusesrangefromsimpleintegrationflowstomulti-stepbusinessprocesscontroltoparallelprocessingoflargedataloadstoevenintegrationorchestrationacrossmessagequeues.Featuresinclude:
•Easy-to-usetreestylenavigationformanagingandopeningallprocessobjects
•MessageConnectorSteps
•Messagequeuesessionmanagement
•ValidationStep
•ExportobjectconnectionWizard
•ExecutionofXSLTscriptsasaprocessstep
•VisualSQLquerybuilder
•Macrodefinitionsupport
•WizardsforInCorememoryandDynamicSQLlookups
Pervasive Process Designer aligns integration projects with business processes
Process Designer – Message Component FrameworkMessageComponentFramework(MCF)providesauniformruntimeenvironmentformessagecomponents.Thisenvironmentprovidessomebasicservicesforcomponentstouseduringexecution.Theenvironmentisalsoresponsibleforthemanagementofthecomponentlifecycleandfortheinvocationofcomponentactions.
15Product Evaluator’s Guide
Ultimately,theframeworksimplifiesthejobofthecomponentwriterbyeliminatingsomeofthecomponentandmessageobjectmanagementissues.Componentsareinvaluabletodevelopers,systemsintegratorsandsolutionprovidersasaquickmethodtocreatereusableservicesfordataintegrationprojects.
ComponentTypes:
Component DesCription
Aggregator Builds a complex message object from a collection of simpler message objects.
Invoker Provides an adaptor for external application interfaces.
Iterator Iterate over a logical collection of messages.
Queue Provides connectivity to a message channel.
Transformer Transform a message object.
Validator Validate a message object.
Pervasive Map DesignerPervasiveMapDesignerisapowerfuldatatransformationdesigner,providingavisuallyappealinguserinterfaceandWizardsforincreasedsimplicity.TheXML-basedunderpinningsthatdefineanintegrationprocessallowsubstantiallymorere-usebyseparatingMapobjects,suchassourcesandtargets,intodiscreteobjectspromotinggreatlyenhancedreusabilityinmediumtolarge-scaleproductionenvironments.
•Visuallycontroldataflowwithinanytransformationregardlessofcomplexity
•DefinerulesformappingcomplexhierarchicalstructuressuchasEDIandXMLdata
•Debugtransformationsonthefly
•UtilizepopulartransportprotocolssuchasHTTP,FTP/SFTPandTCP
• Integratewithmessagequeues(IBMWebSphere,MSMQandanyJMScompliantmessagequeue)
Pervasive Map Designer’s tools put the power of transformation at a developer’s fingertips.
16Product Evaluator’s Guide
oTher Pervasive desiGn Tools
Expression BuildersExpressionBuildersareavailableinthedesignenvironmenttoolstoenabletheuseofthePervasivedataintegrationscriptinglanguagecalledRiFL(RealtimeIntegrationFlowLanguage).RiFLhasbeendesignedtopromotedeveloperefficiencyonmanycommontasksencounteredduringintegrationprojects.TheRiFLlanguagecomeswithnumerouspre-builtfunctionstoautomatedevelopertaskssuchasautomaticconversionofdifferentdateformats,Booleanlogicforcreatingsmartdecisionpointsinatransformationorintegrationprocess,andmathfunctionsthatprovideusefulinformationaboutanintegrationprocesses’executionstatistics.
TheRiFLlanguagehassimilarstructuretoVisualBasic,makingiteasytolearnandaquickpathtoproductivity.Manynon-programmershavebeenabletorapidlylearnhowtocreateexpressionsforsimpleintegrationtasks.Thismodularcodingenvironmentdoesnotrequirecustomer“exits”fromthedevelopmentenvironmenttoreachexternalcustomcode,supportingbettercodereusability,projectmanagementanddebugging.
TheExpressionBuilderloadspreconfiguredfunctionsyntaxintotothemaineditingwindowincludingexpectedparameters.Accesstodataelementsincludingsourceandtargetfieldsandglobalvariablesisavailablethroughthisinterface.Events/Actionssuchas“AfterRecord”reads,“BeforeTransformation”and“OnError”providecriticalprocessingflowcontrolpointswhereadditionallogiccanbeinsertedforperformanceoptimization,errortrapping,user-definedroutines,and more.
The GUI-based Expression Builder is used to quickly build Real-Time Integration Flow Language (RiFL)
17Product Evaluator’s Guide
Metadata Designers: Pervasive’s Structured Schema Designer, Document Schema Designer, and Extract Schema Designer
Metadataplaysakeyroleindataintegrationprojects.Applyingthecorrectmetadataiseverybitasimportantastheconnectingtothedataitself.Howwelladataintegrationplatformhandlesmetadatachallengesshouldbegivenstrongconsiderationwhenselectingasolution.
MultiplemetadatadesignersareprovidedwiththePervasiveDataIntegrationdesignenvironment,includingthePervasiveStructuredSchemaDesignerforparsinglegacydataandflatfiles,thePervasiveDocumentSchemaDesignerforstandardizedbusiness-to-businessdatamessageformatssuchasEDI,andthePervasiveExtractSchemaDesignerforaccessingdatainunstructuredcontent.Allmetadatadesignsmaybesavedandreused.
Pervasive Structured Schema Designer
TheStructuredSchemaDesignerprovidesavisualinterfaceforcreating,savingandmanipulatingmetadataschemadefinitionsforstructureddatasources,suchaslegacydataandflatfiles.TheresultingmetadataschemasarestoredasXML-basedstructuredschemadocumentsthatincludeschema,recordrecognitionrulesandrecordvalidationruleinformation.WiththeStructuredSchemaDesigneryoucanarrangefieldorder,assignfieldnamesandtype,andspecifydatasize.
•Visuallydefineandadjustallrecordandfieldproperties
•Parsebinarydatarecordstructuresanddefinefieldlength,datatype,offsets,andmore
•Createanexternalmetadatadefinitionfromanydataformat
Pervasive Structured Schema Designer provides the flexible structure to visually interpret and manipulate standard and proprietary message formats.
18Product Evaluator’s Guide
Pervasive Document Schema DesignerTheDocumentSchemaDesignergivesyouagraphicalinterfaceforthedefinitionofmetadatafordataexchangeformats,insupportoftradingpartneragreementsandindustrystandards.Standarddataexchangeformatsinvolveinformationtradedbetweentwoormoreenterprises(B2B)forpurposessuchasSupplyChaintransactionsorinsuranceclaims.Industrystandardsincludecommerce-basedEDI,industry-specificXMLformats,HIPAAEDI,HL7,andACORDXML.ThemodificationandcreationofmetadataschemasintheDocumentSchemaDesignergivestightcontroloverdatatransformationsinvolvingB2Bdataexchanges.
• Significantlyreducestimeandefforttodefineandmodifycomplexhierarchicaldocumentstructures
• Allowsthecreationofdocumentstructuresformanystandards,includingEDI,EDIFACT,SWIFT,FIXML,TRADACOMS,HIPAA,HCFA1500,UB92,HL7
• Enablesselectionofspecificsegmentsandelementstoadjustallproperties,referencesandcodetablestodefinetheprecisedocumentstructureneededforatradingpartner
• Generatesavalidationrulesetforensuringthatmessagesreceivedintheformatarestructurallycorrecttopreventdataqualityandnon-complianceproblemsbetweenB2Btradingpartners
Pervasive Document Schema Designer provides the flexible structure to visually interpret and manipulate standard and proprietary message formats.
19Product Evaluator’s Guide
Pervasive Extract Schema DesignerContentmanagementhasbecomeanimportantinitiativeinmanyenterprises,withtheexplosionofdataandinformationacrosstheenterpriseonadailybasis.Itiscriticalthatenterprisesareenabledtoaccess,manage,andcontrolalldata,whetherinstructuredorunstructuredsources.
TheExtractSchemaDesignerisauniquetoolexclusivetothePervasiveDataIntegrationPlatformforgainingaccesstounstructuredcontentaspartofdataintegrationandcontentmanagementsolutions.TheExtractSchemaDesignerworkswithcontent-orientedsourcessuchasemail,reportdata,HTML,printdataoranyotherrawtext.
•Visualinterfaceforrapidcreationofextractionroutinesthatarereusable
• IntegrateswithPervasiveMapDesignerforinclusioninmorecomplexdataintegrationprojects
• GatewaytoincorporationofvaluabledatasuchasemailandInternetcontentintothedataintegrationprojectstoincludeallrelevantdatasources
• Accessibletobusinessuserstocreateextractionroutines,eliminatingexpenseandrelianceonskilledITstaffandoutsideconsultantstomanuallywritecodedroutines
• Self-documentingvisualdesignerensuresthatextractionroutinesaremaintainableandadjustablethroughfuturemaintenancecycles
20Product Evaluator’s Guide
JoininG mulTiPle files as a sinGle sourCe
Pervasive Join Designer PervasiveJoinDesignerisatoolthatallowsyoutojointwoormoresingle-recorddatasourcespriortorunningaMapDesignerTransformation.Thesesourcesdonothavetobeofthesametype.Forexample,youcouldjoinanSQLdatabasetablewithasimpleASCIItextfile.Thefilesarenotphysicallyjoined.TheSourceViewDesignerisusedtocreateSourceViewFilesthatholdmetadataabouteachSource.FromtheseaJoinViewFileiscreated,whichcontainsthemetadataneededbyMapDesignertotreattheSourcefilesasiftheywereasingleSource.InMapDesignerselect“JoinEngine”asthesourceconnectiontousetheJoinViewFile.TheJoinViewFileisdependentontheoriginalSourcefilesremaininginthesamelocationasspecifiedindesign.
Pervasive Join Designer allows users to join two or more single-record data sources prior to running a Map Designer transformation on them.
21Product Evaluator’s Guide
manaGemenT of desiGn obJeCTs
Open Metadata RepositoryThePervasiveDataIntegrationPlatformprovidesanopenXMLrepositoryforhousingallspecificationsandrulesthatdefineanintegrationdeployment’sexecutionandflow.Storingintegrationrulesasnon-proprietaryXMLprovidesenhancedflexibilityandallowsintegrationprojectdesignstobeeasilymanipulatedasanorganization’sintegrationneedschangeanddevelop.
Therepositoryitselfisanopenfilesystemforstoringthevariouscomponentsthatcompriseanintegrationdesign.Foreasyhousekeepingoflargeornumerousprojects,designscanbearrangedintoworkspacesthatcanbefurthergroupedbyuserorbyprojectnames.
Designsaredividedintodistinct,reusablecomponentsthatexistindependentofoneanotherandaresavedasindividualXMLfiles.Byincreasingtheavailabilityofreusableintegrationrules,costsassociatedwithdevelopmenttimeareoftengreatlyreducedsinceindividualcomponentsaresharablebetweenusersandintegrationtasks.Thisreusethemegreatlyincreasestheefficiencywithwhichnewprojectsareimplementedandlowersdevelopmentcostsassociatedwithaprojectas a result.
TheRepositoryandtheXML-basedspecificationsitholdsoffer:
• Easyadministrationofprojectdesignsandworkspacesforasingledeveloperoracrossmultipleprojectgroups
• Improvedcode“reuse”featuresenhancedeveloperefficiencyandprojectscalability
• AlldesignsmaybeadditionallyalteredorevengeneratedthroughanyXML-compliantdevelopmentenvironment
•DesignsareportabletoberunonsixdifferentIntegrationEngineplatforms
• XML-basedspecificationsarebothhumanandmachine-readable,andcompatiblewith sourcecodecontrolapplications
• AllowISVstocreate,manipulateandevenexecutedataintegrationdesignsfromwithin theirapplications
MetadataInterfaces
Design Tools OPEN METADATA REPOSITORY
OPEN
MET
ADAT
A AC
CESS
& M
ANAG
EMEN
T
Repository Explorer
Sche
ma
Desi
gner
s
ip.xml
.rifl
Process spec
Data transformation andflow control spec
Field mapping specTransformation spec
Source specTarget spec
Unstructured textextraction schema
Structured schema
Document schemaDocument schema codeset
Document segment libraryDocument field libraryDocument code library
dsl.xmldfl.xmldcl.xml
ds.xmldsc.xml
ss.xml
.cxl
sc.xmltc.xml
map.xmltf.xml
Metadata ExchangeXML, XMI, XIF, CWMI etc.
3rd Party Designers,UIs, Programs, Web Services
JAVA, VB, C++, PERL
Import/Upgradefrom Competitors
Process Designer
Map Designer
StructuredSchema Designer
Visual Parser
ExtractSchema Designer
DocumentSchema Designer
(EDI, HIPAA, SWIFT etc.)
22Product Evaluator’s Guide
user inTerfaCes – aCCess To The desiGn rePosiTory
Repository ExplorerTheRepositoryExplorerisusuallythestartingpointfordataintegrationdesignprojects.Itprovidesaconsolidatedviewofdesignprocessesandmanagestherepositorycontentscreatedbythevariousdesigntools.Theexplorationandcontrolcapabilitiescanbeexecutedacrossmultiplerepositoriesandasimpledoubleclickonanydesigncomponentopenstheappropriatedesigntool.
Repository ManagerTheRepositoryManagerisatoolsetthathelpsthedeveloperandprojectmanagergetmoreoutofthePervasiveDataIntegrationPlatformdevelopmentexperienceandtocoordinateworkacrossmulti-developerintegrationprojects.
• •Dependencyimpactanalysistotrackdependenciesbetweendesigncomponentsandofferinsightintotheimpactofproposedchangeswithinaproject’sworkplan oAvoidcreatingproblemsinonedesigncomponentwhileworkingonanother oObtainvitalinformationtoestimateimpactofprojecttasks
• SearchandreplaceanyRiFLcodeorXMLelementacrossdesignsandprojects oForeasierdeploymentswhereknownparameterschangebetweendevelopment, testandproductionenvironments
• Deploymentpackagingeasestheprocessofcollectingrelateddesignsintoasecurepackagethatisasingledeploymentcomponentforusewiththeappropriateproductionortestengines
• RepositoryManageroffersreportsonkeyattributesofdesigns:creationdate,lastediteddate,lastexecuteddate,authoring,andmore
Pervasive Repository Manager
23Product Evaluator’s Guide
Pervasive inTeGraTion enGines – ProduCTion and TesT environmenTs
PervasiveDataIntegrationEnginesareuniquelyarchitectedtoprovidemulti-dimensionalscalabilityforcomplexintegrationchallenges.Integrationenginesworkformanysolutiondeliverymodes:basicextract-transform-load(ETL)projects;large-scaleB2Biprocesses;traditionaldataintegrationbrokeringneeds;large-scalemulti-terabyte,parallel-processingtransformations;embeddedapplicationsolutions.Enginesareavailablefordesign/testingandproductiondeployments.Theyareconfigurabletomeettherequirementsofanyintegrationproject.
IntegrationEnginesarealsohighlyembeddablethroughrichinterfacesthataresupportedbySDKsandcommandlinefunctionalityforpinpointcontrolbyexternalapplications.
PervasiveIntegrationEnginesarecurrentlyavailableforthefollowingOSplatforms:
•Windows
•Linux
•HP-UX
•SUNSolaris
•AIX
BITools
COBOLData File
OracleDatabase
DataWarehouse
DataMart
SupplierPortal
ConversionMigration
DATA MIGRATION
DATA INTEGRATION
Message Integration
Point-to-Point APIIntegration
Message Bus/SOA
B2BiHub
Data CollectionFramework
EDIXML
WebContent
Web ServicesXML/SOAP
Data Replication
File TransferFTPq21
BUSINESS INTEGRATION
NewAcctg
COBOL
OldAcctg
ETL
WW
WSU
PP
LIER
S ERP(SaaS)
CU
STOM
ERS
REM
OTE O
FFICES
Pricing
HR CRM(SaaS)
= Pervasive Integration Engine
Potential deployment options with Pervasive’s Integration Engine – distributed or centralized, supporting incremental adoption of a complete platform
24Product Evaluator’s Guide
enGine manaGemenT - Pervasive inTeGraTion manaGer
PervasiveIntegrationManagerprovidesthemeansforautomatingandmanagingdistributedintegrationengines.Withsingle-pointadministrationandpowerfultools,PervasiveIntegrationManagerprovidesscheduling,monitoring,andreportingonengineexecutionswhileprovidingascalablearchitectureforfuturegrowth.
Sinceallmanagementisperformedfromasingleadministrationpoint,IntegrationManagerimprovesoperationalefficiencyinthemanagementofgeographicallydistributedintegrationengines.Withtheabilitytoremotelyadministeranynumberofintegrationpointsthroughouttheorganization,customerscanbuildouttheirintegrationinfrastructureasrequired,usingaflexibleandscalablearchitecturedesignedforeasymanageability.
Trouble-shootingcapabilitiesprovideimmediateresponsetopotentialproblemsduringproductionruns.Reportingonexecutionduringproductionrunsenablesfine-tuningofdataintegrationprocessesforefficiencyandoptimaluseofresources.
Pervasive Integration Manager – Schedule Feature
25Product Evaluator’s Guide
inTeGraTion serviCes PervasiveDataIntegrationsolutionsmaybedeployedviaservicesinavarietyofwaysdependingoncustomerrequirements(seediagrambelow)–fromasingle,lightweightembeddedenginetoadistributed,fullymanagedsetofServiceOrientedArchitecture(SOA)adapters.
WhendeployinginSOAenvironmentsthereareseveralcommunicationmechanismsthatworkinconjunctionwithJ2EEapplicationservers.Collectively,thecapabilitiesbelowarecalledtheIntegrationServicesSDK:
• Apurpose-builtMessageDrivenBean(MDB)thatisconfiguredtolistenonaJMSqueuefortheinputmessage,theninvokeaspecificintegrationspecificationviatheIntegrationServer.TheIntegrationServer,inturn,willfindanavailableIntegrationEngineonthelocalserverandinvokethedesiredintegrationspecificationandreturntheresults
• AJCARAforsynchronousinvocationofintegrationspecsviatheIntegrationServer(asabove)
•AJavaAPIforsynchronousinvocationofintegrationspecsviatheIntegrationServer(asabove)
• AnHTTPWS-IcompliantservicesinterfacetotheIntegrationServertoacceptSOAPinvocationsfromanyapplication(notonlyJ2EE)
Input Message
XML
Connector
Data
Connector
Message
Connector
XML In
Get Msg
PutMsg
tracker
XML Out
OutputMessage
http SOAP
Return Message
Native Connector (e.g.) SAP API)
Synchronous
DATALAYER
Synchronous
COARSE-GRAINEDLossely-Coupled
Asynchronous
FINE-GRAINEDTightly Coupled
Synchronous
(API - Methods/Calls)
COARSE-GRAINEDLossely-Coupled
Asynchronous
(Message Documents)
(Message Documents)
Inte
grat
ion
Man
ager
(Man
ages
Man
y En
gine
s) Inte
grat
ion
Serv
er
Inte
grat
ion
Engi
nes
JAVA
JCA
MCB
BU
S
SOAPINVOCATION
J2EEAPPLICATION
SERVERS
APPLICATIONLAYER
INTE
RFA
CE
BU
SIN
ESS
FILE
S
Database Connections (SQL, ODBC 2.x, 3.x, IDAPU, Pervasive SQL, etc.)
File Format Connectors (ASCII, Binary, Unicode, XML, etc.)
Native API Connectors (COBOL, iSAMs, etc.)
DBMB Bulk Lead ConnectorsDBMBLoadFile
T1 T4T2
T2
Pervasive Integration Engine Deployment Options
26Product Evaluator’s Guide
AdaptersForruntime,theIntegrationEnginessupportmorethan150factory-suppliedadapters.MostofthePervasivedataadaptershavebeenwrittentotakeadvantageofnativeAPIsandindustrystandards.Manyoftheadaptersareavailableformultipleplatforms.
Databaseadapterssupportarangeofmodesfromclient/serveraccess,localdatabaseaccess,multimodewhereasingledatastreamcanberapidlyspreadacrossmultipletablesinatargetdatabase,andmass-insertforhigh-speedbulkloadingofdata.
AccessAccess 97Access 2000Access XPAlpha FourASCII (Delimited)ASCII (Delimited) JoinASCII (Fixed)
ClarionClipperCLF WebserverDataEaseDataFlexDJ Document SchemasDJ Log File DatadBASE II, III III+, IV, V, (IDAPI)
DialogDIFEnableExcel 95, 97Excel 2000Excel XPFolio Flat FileFoxbase
FoxProLDIFLotus 1-2-3Lotus NotesLotus Notes 5Lotus Notes Structured TextLotus WorksMac ASCII Delimited
Magic PCMAILERS+4 (dBASE)Microsoft Word (Mail Merge)Paradox v5 (IDAPI)Quattro Pro Windows v5Rich Text Format (RTF)Unicode (Delimited)Unicode (Fixed)
WordPerfect 6.0 (Mail Merge)Wordstar (Mail Merge)WordPerfect 4.2/5.0Visual FoxProVisual dBASE 5.5
Oracle 7, 8, 9i, 10g, 11gOracle 7, 8, 9i, 10g, 11g MultimodeOracle Direct PathOracle SQL LoaderPervasive.SQL 8, 9, 10Pervasive.SQL 8, 9, 10 MultimodePostgreSQLPostgreSQL (ODBC 3.x)Btrieve 5, 6, 7, 8
CA dB-FastC-ISAMCloudscapeCWMDB2 7.2, 9.5 UDBDB2 7.2, 9.5 UDB MultimodeDB2 LoaderDB2 UDB Mass InsertDB2 (ODBC)eXcelon, eXcelon XIS 3Hitachi Hi-RDB
IDAPIInformix DB LoaderInformix (ODBC)Informix Online DSInformix/SEIngres (ODBC 3.x)Interbase (IDAPI)JDBCMUMPS (ODBC)MySQLNonStop SQL/MX
ODBC 3.5ODBC 3.5 Mass InsertODBC 3.5 MultimodeRBase 4.5 (ODBC)Red BrickScalable SQLSQLBaseSQL ScriptSQL Server 6, 7, 2000, 2005SQL Server 6, 7, 2000, 2005 Mass Insert
SQL Server BCPSQL Server Mass InsertSQL Server (ODBC)Sybase 11, 12 Sybase 11, 12 Mass InsertSybase 11, 12 MultimodeSybase BCPSybase (ODBC)Sybase SQL AnywhereSybase SQL Anywhere 6Sybase SQL Mass Insert
Sybase SQL 11Sybase SQL 11 MultimodeTeradata (Fastload)Velocis (ODBC 3.x)Watcom SQL v5Watcom SQL (ODBC)
AccountMateACT! for WindowsADABAS (NatQuery)Apache CLFChampion Acct.CYMA Acct.GoldMine
Great Plains DOSGreat Plains UNIX/MacNavision FinancialsMacola Acct.Market ForceMicrosoft Dynamics CRM 4Microsoft Dynamics GP 9, 10
Microsoft IIS ELFNetSuiteOracle ApplicationsOracle Siebel CRMOn DemandPayChex DJFPeachtree Acct.
Personal LibrarianPivotal CRMPlatinum Acct.RealWorld Acct.RightNowSage CRM SalesLogixSage MAS 90/200
Sales AllySalesforceSAP BAPI, IDoc, EAASASSBT Acct.Solomon Acct.SPLUS
SPSSStatisticaSYSTATUSMARC
BizTalk XMLDataRushEmailEmail Attachments
ErwinFile Folder Queue (FFQ)File Transfer BuilderFTP
HTTP/SHTMLJMSLDAP
Microsoft MQWebsphere MQOracle AQOracle BPEL
Progress Sonic ESB, MLLP, FIXSGMLSOAP
TCP/IPURL/URIXMLXML Schema, DTD, XDR
ACORDEDI (EDIFACT)EDI (X12)
FIXHCFA 1500-NSFHIPAA
HL7HR-XMLMIB
NCPDPNSFSWIFT
Text (Delimited – EDI)Text (Delimited – EDIFACT)Text (Delimited – HL7)
TRADACOMUB92-NSF
ACUCOBOL (ODBC 3.x)ACUCOBOL-GT v7BAFBinary
Binary Line SequentialCOBOL 01COBOL CopybookCOBOL Flat File
C-tree PlusC-tree 4.3Fujitsu COBOLISAM
Micro Focus COBOLQSAMSequential BinaryTape Drive Sequential
VSAMVariable Sequential (MVS)Variable Sequential (Record-V UniKix)
Variable Sequential (SyncSort)Variable Sequential (UserDefined)
10
27Product Evaluator’s Guide
GeTTinG sTarTed TuTorials – Pervasive daTa inTeGraTion desiGn Tools
AselectgroupoftutorialshasbeenreprintedheretogiveproductreviewersaquickpathintoanunderstandingofthesoftwareproductthattheyhavedownloadedfromPervasiveSoftware.Thetutorialsillustrateproductcapabilitiesandwhatbenefittheyprovidefordataintegrationprojects.
basiC Pervasive maP desiGner TuTorial
Overview of Map Designer User InterfaceThepurposeofthistutorialistoguidethenewevaluatororuserthroughthebasicstepsnecessarytocreate,save,andrunabasicMapintheMapDesigner.ItwillhelptheusertobecomefamiliarwithboththebasicprocessofconvertingdatafromoneapplicationtoanotherusingMapDesignerandthetermsusedthroughoutthedocumentation.Thebestwaytousethistutorialistoprintahardcopysothesequentialstepscanbeeasilyfollowed.TheTargetfilecreatedinthistutorialwillbeusedastheTargetfileintheothertutorialsavailableintheMapDesignerHelpFile,sothistutorialmustbecompletedbeforegoingforwardwithotherMapDesignertutorials.
Beforebeginningthetutorialitisimportanttounderstandthebasicsofmapdesign.MapDesignerisatooldesignedforthecreationofdatatransformations.WhilethevarietyofoptionsMapDesigneroffersthedeveloperforincorporationintotheirdesignsisverywide(numerousconnectorstodifferentdatasourcesandtargets,numerouspre-builtandcustomizablescriptingfunctionsbuiltintoarobustscriptinglanguageforuseinmanagingtransformationlogic,flowcontrollogic,stringmanipulation,dataformatmanipulation,lookuptables,filtering,andsoon),thebasicsformapdesignaresimple:Connect to a source, Connect to a target and Map the data asseeninthethreetabsfromtheMapDesignerscreenshotbelow:
(1) Connect to a Source–PervasiveMapDesignercomeswiththreemaintabs,thefirstofwhichisusedtohandleallconnectivityoptionstothedatasourceforthetransformationdesign.DependingontheIntegrationproductyouareevaluating,youhaveachoicebetweennumerousnativeconnectorstolegacyandmoderndatabases,flatfiles,anddirectlytomanyapplications.Onceconnected,youcanfreelybrowsethesourcedataatanytimetoassistwiththetransformationprocess.
(2) Connect to a Target–Thesecondtabisusedtohandleallconnectivitytothetargetdestination.Again,numerousconnectorsareavailableforwritingthetransformeddatatoyourtargetdatabase,file,messageformat,etc.Dependingontheoptionsselectedinthistab,the
28Product Evaluator’s Guide
transformationcanreplacedatainatarget,appenddata,deletealldataandreload.Also,fordatabasetargets,parent-childrelationshipscanbemaintainedbyusingPervasivemulti-modeconnectors that load data in more than one table in a database.
(3) Create the Map–OncetheSourceandTargetaredefinedandconnected,the“Map”tabiswherethedatatransformationisdesigned.Fieldstructuresaredisplayedforsourceandtargetdataandasimpledraganddropofsourcefieldstothetargetisallthatisneededforasimplepassthroughtransformation.Fromthere,theMaptabincludesarichsetofeventactionsandtransformationscriptingoptionstohandleeverythingfromerrortrappingtovalidationtocomplextransformationandflowlogicusingtheReal-TimeIntegrationFlowLanguage(RiFL)builtintotheproductforscripting.Thescreenimagebelowshowsthemaptabwiththesourcedatafieldsatthetopandthetargetfieldsatthebottom
maP desiGner TuTorial — CreaTion of a basiC TransformaTion maP
AfterreviewingtheoverallMapDesignerGUIlayoutandthebasicsofmapdesign,itistimetoworkthroughthecreationofasimplemapusingMapDesigner.
Open the Repository Explorer to Get StartedOpen Repository Explorer (Start>Programs>Pervasive>RepositoryExplorer).
ClickonthedownarrowbesidetheNewiconinthetoolbarandselectMaptoopentheMapDesignerwindow.OnceMapDesigneropens,theSource Tabwillbedisplayed:
1.Clickonthedown arrow beside the Source Connection boxtoopentheSelect ConnectionWindow.
2.YouwillhaveachoiceofthreetabsintheSelectConnectionwindow:“MostRecentlyUsed,”“FactoryConnections”and“UserDefinedConnections”.ClickontheFactory Connections tabandclickonASCII (Delimited) connectorfromthelist.ClickOK to return to the Map Designerwindow.
3.Clickthedown arrownexttotheSourceFile/URIbox.ThiswillopenthestandardWindowsnavigationdialogbox.Tutor1.ascisthesourcefilethatyouwillbeusing–itislocatedinthedefaultinstallationdirectory\Common800.
NOTE:TheusualdefaultfileextensionsforASCIIdelimitedare“.asc”,“.csv”and“.txt”.
4.ClickonTutor1.ascandthenclicktheOpen button.YouwillbetakenbacktotheSource Connection tabandthedrive,directoryandfilenamewillbevisibleintheSource File box. If a dialogboxdisplayswithaFileSampleSizemessage,click“OK”tocloseit.
29Product Evaluator’s Guide
5.ClickontheSource Data Browser icon todisplaythesourcefile.Youwillseethatitcontainsfieldnamesinthefirstrowofdata.Thisisthefileheader.ClosetheSourceDataBrowserbyclickingontheX buttoninthetopright.
6.TheSourcePropertiesoptionsboxisontherightsideoftheSourcetab.ThisiswherespecificpropertiesoftheSourcefilecanbeset.Clickontheword“False”nexttotheword“Header”in the Source Propertyoptionsbox,andthendouble-click to change it to True.
7.ClicktheApply buttonbelowthelistofProperty Options.
8.ClickontheTarget Connection tab.
9.ClickthedownarrownexttotheTargetConnectionboxtoopenthe“SelectConnection”window.
10.YouwillhaveachoiceofthreetabsintheSelectConnectionwindow:“MostRecentlyUsed”,“FactoryConnections”and“UserDefinedConnections”.Select the “Factory Connections” tab.
11.SelectdBASE IVfromthelist.ClicktheOK button to return to the Map Designerwindow.
12.Clickthedown arrow to the right of the Target File/URI box.ThisopensthestandardWindowsnavigationdialogbox.ThedefaultfileextensionwillappearintheFileNamebox(inthiscase,“.dbf”).Replacetheasterisk(*)with“Tutor1”andclickOpen.
13.TheConnect button should be grayed outatthispointindicatingaconnectiontothetargethas been made. If the Connect button is not grayed out,thenclickontheConnect button toconnecttothetarget.TheOutputModewilldefaulttoReplaceFile/TablewhenyouarecreatinganewTargetfile.Thisoutputmodewillcreateanewfileorreplaceanyexistingfileofthisnameinthisdirectory.
14.Clickthe“Map”tab.
15.TheMap tabdisplaysthreemainpanesontheupperhalfofthewindow.TheupperleftpaneshowsthefieldsassociatedwitheachSourceRecordtype.ThetreeopensalreadyexpandedandthefieldsforrecordtypeR1arehighlighted. Theuppermiddlepanecontainsagridshowingthefieldsassociatedwitheach TargetRecordtype. AndtheupperrightpaneshowstheEvents/ActionsformappingfromSourcetoTarget
Drag-and-Drop functionality:16.Intheupperleftgrid,locatetheboxthatcontainsanasterisk (*)nexttothecellthatsays
<allfields>.ThisasteriskrepresentsthelayoutofallfieldsintheSourceFileandactsasashortcuttoduplicatetheentireSourceSchemaintheTargetfile.Click on this cell and hold downonthemousebutton.
17.Drag the cursortothemiddlegridfortheTarget.Thecursorwillchangetoablackrectanglewhenitisoveracellinwhichtheselectedfileinformationcanbeinserted.Releasethemousebuttonwhenthecursorisinthefirstcellofthemiddlegrid.
18.SincedBASEdoesnotallowlowercaselettersorspacesinfieldnames,anInvalidFieldNamedialogboxwillappear.MapDesignerwilloffertofixtheprobleminvariousways.Thedefault
30Product Evaluator’s Guide
istoautomaticallycreatevalidfieldnamesbasedontheSourcefieldnameforallinvalidfields.
19.ClicktheOKbuttonontheInvalidField Names dialog box.
20.AfteryouclicktheOKbutton,allthelowercaseletterswillchangetouppercaseandspaceswillchangetounderscoresfortheTargetfieldnames.TheTarget Field Expression column willbefilledwiththeSourcefieldnamesinappropriatenotationformappingexpressionstoadBasefile.
21.Fortransformationsthathaveonlyonerecordtype,afteryoumapyourfields,aClearMapPutRecordactionisautomaticallyaddedtotheAfterEveryRecordeventintheSourceEvents/Actionsgrid,soyourtransformationisreadytorun.Formostothertransformations,youwillsetupthenecessaryeventactionsasappropriateforeachtransformationdefinition.
22. ClearMapPut Recordisthemostcommonlyusedactioninatransformation.TheClearMapPutRecordcombinestheactionsof:1–clearingthememorybufferofpreviouscontents(usuallypreviouslyreadrecord),2–mappingtheSourcedatatotheTargetdatastructure,and3–writingtherecordtotheTargetfileortablespecified.
You have now completed the three basic transformation steps.
23.Clickthe“Save Map”icon in the toolbar.
24.ThiswillopentheSave AsdialogboxinwhichyouarepromptedforaSaveLocation.ThiswilldefaulttoyourWorkspace,butyoushouldverifythelocationpriortosavingthemapandchangeitifneeded.Mapsdefaulttothedocumenttype“.map.xml”.
25.Typeadocument nametoreplace“map1”inthetextboxandclickSave. Your transformation is saved.
26.ClickontheValidate Map iconinthetoolbar.TheMapshouldvalidatecorrectly.
27.ClicktheRun Map icon in the toolbar.
28.MapDesignerwillloadalltheinformationitneedstorunthistransformationandthenexecuteit.Youwillseeagaugequicklyflashonthescreenasthetransformationprogresses.Acounterwillestimatethenumberofrecordsconverted.Whenthetransformationisfinished,thetotalnumberofrecordsreadandthenumberconvertedwillbedisplayedinthestatusbaratthelowerleftofscreendisplay.
29.ClicktheTarget Data Browser icon inthetoolbartoviewthenewdBASEIVfile.ThiswillopenanewwindowinwhichyoucanviewandbrowsethedBASEIVfilejustcreated.
30.AfterviewingthedBASEfile,closethebrowserbyclickingtheX buttonintheupperrightcorner.
31.CloseMap DesignerbyclickingtheX buttonintheupperrightcorner.
YouwillbebackintheRepositoryExplorer,whereyouwillseetheMapyoujustcreatedintheRepositoryDocumentslistintherightpanel.Ifyoudonotseethenewmap,clickonyourworkspacerepositoryintheleftpaneltreestructure.
Congratulations, you have just created, validated, saved and run your first Map using Pervasive Map Designer!
31Product Evaluator’s Guide
basiC Pervasive ProCess desiGner TuTorial
Overview of Process Designer User InterfaceThepurposeofthistutorialistoinstructtheevaluatorornewuseronthebasicsofbuildingintegrationprocessdesignsusingtheProcessDesigner.Processdesignsrepresentthehigherlevelworkflowofthevariousstepsinasimpleorcomplexintegrationtask.Thebestwaytousethistutorialistoprintahardcopytoeasilyfollowthesequentialsteps.
Processdesigns,likeanyworkflowtools,haveastartingpointandanendpointforagivenprocess.Withineachprocess,anynumberofstepscanbedraggedontotheProcessDesignercanvasfromthepre-builtlistofprocessflowiconsonthelefthandsideofthedisplay.Theseareusedtobuildprocessstepsthatcancalltransformations(mapscreatedinMapDesigner),decisionstepswherelogiccanberecordedtopushaflowdownoneormorepathsdependingonthesituation,loopinglogic,parallelprocessinglogic,embeddedsub-processes,andsoon.
(1) Process Design Canvas–Thisareaisblank(exceptforStartandStopsteps)whendesigninganewprocess,butlatercontainsalltheprocesselementsinasavedprocessdesign.Theaboveexampleshowsacompletedprocessdesignthatincludesqueuessetuptocollectdataexchangemessages(EDIorHIPAA,forexample),aloopingstructuretocontinuemonitoringthequeueandatransformationsetuptoutilizenewlyreceivedmessages.Italsocontainsbranchinglogicnodesthatcanrouteaprocessdownoneormorepathsdependingonlogicparameters.
(2) Process Design Icon Palette–Alltheitemsonthispalettemaybedroppedontothecanvasandthenbuiltouttocreateasimpleorcomplexintegrationprocessorworkflow.Onceonthecanvas,theprocessdesigniconswillbecustomizedbyadjustingtheirpropertiestoenactthespecificsneededforeachstepintheprocessflow.
Process Designer Tutorial – Creation of a Basic ProcessThisstep-by-steptutorialisavailabletoassistlearninghowtousetheProcessDesignerandwillintroducethegraphicalinterfaceandthedrag-and-dropmethodofcreatingprocesses.Thebestwaytousethistutorialistoprintahardcopysothesequentialstepscanbeeasilyfollowed.ItisassumedthattheevaluatororuserhasfamiliaritywiththeMapDesignerandhasworkedthroughthebasicMapDesignerTutorialinthisdocumentbeforetakingonProcessDesigner,andthattheintroductiontotheProcessDesignerhasbeenreviewed.
32Product Evaluator’s Guide
TaskCreateaprocessthatwillrunonceaday,includingatransformation,butifitisaSundayadditionalroutineswillalsoberun.Forthistutorial,thetransformationandtheSundayroutinesarenotactuallyincluded.Instead,commentboxesareusedtoenabletherunningoftheprocessaspartofthistutorial.Asanadditionalexercise,atransformationmaybecreatedusingthesampledataintheMapDesignerTutorialandaddedtotheprocessforthistutorial.
Procedure1.FromtheRepositoryExplorer,clickonthedownarrowbesidetheNew icon in the toolbar and
select ProcesstoopentheProcessDesignerwindow. AProcessCanvasisdisplayedshowingStartandStopsteps.Everyprocessmusthaveonestartandonestopstep.
2. Perpreviouspresentations,moveafter“Scriptingstepicon.”ClickonceontheScripting step iconontheProcessStepBarandmovethecursoroverthecanvas.Position the cursor below the Start step and click the left mouse button.AScriptstepwillbedisplayedinthisposition. Youcandragasteparoundthecanvasbyleft-clickingthecursoronit,holdingthebuttondownandmovingthesteparound.
3.Double-clickonthissteponthecanvastoopentheScripting step Properties window.
4.Renamethestep“Run Daily Transformation”intheName property.
5. Click on the Expression field and type: MsgBox(“Run Daily Transformation”) Note:SeethehelptopicsfortheMsgBox()functionifyouarenotfamiliarwithitsuse.
6.ClickontheOKbuttontoclosetheScriptingPropertieswindow.
7. ClickontheDecision step icon on the Process Step Bar,positionthecursorbeneath the Script step and clicktheleftmousebuttonagain.AConditional Decision stepwill bedisplayed.
8.Double-clickontheicononthecanvastoopentheDecision step Properties window.
9.Renamethestep“Is it Sunday?”intheName property.
YounowneedtocreateaconditionthatwillbeTRUEifthisprocedureisrunonaSundayandFALSEonanyotherday.TheconditionwillbebuiltintheExpressionpropertyboxusingaVB-likescriptinglanguagecalledRiFL(RealtimeIntegrationFlowLanguage)implementedbyPervasivefordataintegrationtasks.
10.ClickontheExpressionpropertyandtype: DatePart(“w”,Now())=1 Note:SeethehelptopicsfortheDatePart()andNow()functionsifyouarenotfamiliarwiththese.
11.ClickOKbuttontoclosetheDecisionstepPropertiesWindow.
12. Create another Scripting step,thistimetotherightoftheDecisionstep.
13.Double-clickonthissteptoopentheScripting step Properties window.
14.Renamethestep“Sunday Only Routines”intheName property.
15.Click on the Expression property and type: MsgBox(“Sunday Only Routines”) Note:SeethehelptopicsfortheMsgBox()functionifyouarenotfamiliarwithitsuse.
33Product Evaluator’s Guide
16.Youshouldnowseethefollowingdiagram:
00.Nowlinkthestepstoshowthesequenceinwhichtheymustberun.
17.ClickontheLinkicon andmovethecursorovertheStartstep.Thecursorwillappearaswhenitisoverthecanvasand whenitisoverastep.
18.ClickontheStart step.Nowmovethecursortothe“Run Daily Transformation”function step.Againthecursorwillappearas untilitisoverastep,whenitwillappearas .
19.Clickonthe“Run Daily Transformation”step.Anarrowwillnowappearlinkingthesestepsintheordertheyaretobeperformed.Notethatthecursorisstillshowingas .ThisissothatyoucanlinkallyourstepswithouthavingtorepeatedlyreturntotheProcessiconbar.
20.Createalinkfrom“RunDailyTransformation”to“IsitSunday?”inthesameway.
21.Createalinkfrom“IsitSunday?”tothe“SundayOnlyRoutines”functionstep.Inthiscasethefollowingdialogboxwillbedisplayed:
22.SelectTrue.Thisshowsthatthisisthepathtofollowiftheconditionspecifiedinthe“IsitSunday?”evaluatesasTRUE.
23.Createalinkfrom“IsitSunday?”tothe“SundayOnlyRoutines”functionsteptothe“Stop”step.AgaintheDecisiondialogboxisdisplayed.SelectFalse.
34Product Evaluator’s Guide
24.Finallycreatealinkfrom“SundayOnlyRoutines”to“Stop”. Yourdiagramshouldnowlooksomethinglikethis:
25. Validatetheprocess–click on inthetoolbar.Theprocessshouldvalidatecorrectly.
26.Toruntheprocess,click on the button.Ifeverythinghasbeensetupcorrectlythefollowingwillappear:
27.ThisshowstheRunProcessdialogthatdisplaysdetailsoftheProcessasitgoesthrougheachstep.InthiscasetheRiFLScriptMsgBoxshowsthatithasreachedthe“RunDailyTransformation”step. (Inareal-worldexampleatransformationwouldbecarriedoutatthispointandtheprocesswouldnotbestopped.)
35Product Evaluator’s Guide
28.ClickOK button to continue. Thefollowingshowsthecompletedprocessifrunduringtheweek:
28.ThefollowingshowsthecompletedprocesswhenrunonaSunday:
Congratulations, you now have mastered the fundamental process and map design skills needed to evaluate the product.
36Product Evaluator’s Guide
basiCs: maP desiGner, ProCess desiGner and exTraCT sChema desiGner
TofindtheGettingStartedGuide:
Windows Desktop, Start, All programs, Pervasive, Documentation, Getting Started Guide
Toopenaparticulardesigner:
Windows Desktop, Start, All Programs, Pervasive, Integration Architect, Designers–thenselectoneofthedesignersThemajorityofthetutorialsandsamplesinvolveeitherMapDesigner.TherearealsotutorialsforProcessDesignerandExtractSchemaDesigner.
Map DesignerTofindadditionalinformation,tutorialsandsamples,gotothein-productHelpMenu.Thenselectoneofthefollowing:Contents,Tutorials,Samples.
MapDesignerTutorialsList
•Tutorial1showsyouthebasicsofhowtotransformpredefinedsourcedatatoanewtargetfile
•Tutorial2showsyouhowtodefinefieldsinafixedASCIIsourcefileandtransformthedatatoanewtargetfile
•Tutorial3showsyouhowtoappendsourcedatatoanexistingtargetfile
•Tutorial5showsyouhowtotransformdatawherethesourceisasingle-recordtypeandthetargetisamulti-recordtypeXMLfile.Thistutorialisparticularlyusefulforlearninghowtousethe event handlers
NoteForalloftheaboveMapDesignertutorials,see“TransformationandSchemaTutorials”onpage1-1intheTutorials Reference.
Process DesignerTofindadditionalinformation,tutorialsandsamples,gotothein-productHelpMenu.Thenselectoneofthefollowing:HelpTopics,Tutorials,Samples.
ProcessDesignerTutorialsList
•Step-by-StepTutorialforLearningProcessDesigner:ProcessDesignerTutorial-TheBasics
Extract Schema DesignerWhenthisdesignerisfirstopened,adialogdisplaystoinviteevaluatorsandnewuserstolearnmoreaboutExtractSchemaDesigner.Clickonthebuttonfor“LearnMoreAboutExtractSchemaDesigner”tobeginthetutorials.Youmayalsoaccessthetutorialsthroughthein-productHelpMenu,Contents,thenGettingStartedwithExtractSchemaDesigner.
ExtractSchemaDesignerTutorialsList
Step-by-Step Tutorials for Learning Extract Schema Designer
ExtractSchemaTutorial1-TheBasics
ExtractSchemaTutorial2-TaggedDataandAutomaticFeatures
ExtractSchemaTutorial3-ColumnarData
ExtractSchemaTutorial4-FloatingTags
ExtractSchemaTutorial5-ColumnarDatawithaFooter
ExtractSchemaTutorial6-VariableLengthMultiLineDataFields
ExtractSchemaTutorial7-MultipleAcceptRecords
37Product Evaluator’s Guide
addiTional samPles
Dataintegrationprojectscenariosamplesareprovidedwiththeproductstoillustratereal-worlduses.PervasiveDataIntegrator™mustbeinstalledtoworkwiththesamples.Tofindthesamples,openMapDesignerorProcessDesigner,gototheHelpMenuintheToolbarandselectSamples–thenfollowinstructions.
Basic Samples1.UsingConditionalPutActionswithEventHandlers
2.FilteringSourceData
3.SortingSourceData
4.StandardizingMultipleDateFormats
5.WritingRejectRecordstoaFile
6.UsingDJXtoPassVariablestoaSQLStoredProcedure
Intermediate Samples7.UsingGlobalVariablesinTransformations
8.UsingtheFileListFunctioninaProcess
9.MappingfromOneRecordTypetoMany
10.MappingDatabaseRecordstoEDI
11.SettingOnDataChangeEvents
12.UsingBufferedPutTreetoCreateHierarchicalRecords
13.HandlingErrorsinaFileFolderQueueProcess
Advanced Samples14AggregatingRecords
15ManipulatingBinaryDatesattheBitLevel
16ComplexDateFiltering
17WorkingwithDJRowSetandArrays
18DynamicSQLLookup
19DynamicSQLLookupwithErrorHandling
20DynamicSQLLookupwithRejectRecordsHandling
21UsingEDIX12BatchTransactionIteratortoReadMessages
22MappingTwoJoinedSourceTypestoaTarget
23ListeningforMessageswithFileFolderQueue(FFQ)
24WebServiceInvoker WebServiceInvoker–MessageComponentFramework-for Process Designer (seeProcessDesignerHelpContents:SessionsTypes,InvokerSessionsandthe Message Component Framework (MCF) User’s Guide.)
38Product Evaluator’s Guide
TraininG and serviCes oPTions
PervasiveSoftwareoffersanextensivearrayoftrainingoptionstoassistyouandyourteamwithamuchdeeperandmoreformaleducationforusingthePervasiveDataIntegrationproducts.Coursesincludedeveloperproficiencytraining,planningandconsultingservices,anddesignoptimizationservices.Pleasecontactyoursalesaccountexecutiveformoreinformation.
suPPorT for QuesTions While revieWinG The ProduCTs
We’redelightedthatyoutookthetimetoreviewoneormoreofthePervasiveDataIntegrationproducts!
Shouldyouneedassistancewithanyaspectofyourreview,pleasecontactPervasivepre-salessupportat:
•800.287.4383
•512.231.6000
• www.pervasive.com/supporttofilloutawebformrequestingassistanceandtoreadaboutothersupportoptions.
39Product Evaluator’s Guide
Contact Information
PervasiveSoftwareInc. 12365-BRiataTraceParkway Austin,Texas78727
UnitedStates 800.287.4383 512.231.6000 Fax:512.231.6010 [email protected]
EMEAA +800.1212.3434 [email protected]
http://www.pervasiveintegration.com
PN0607H10
© 2009 Pervasive Software Inc. All rights reserved. All Pervasive brand and product names are trademarks or registered trademarks of Pervasive Software Inc. in the United States and other countries. Windows Vista is either a registered trademark or trademark of Microsoft Corporation in the United States and/or other countries. All other marks are the property of their respective owners.