talend metadata managerinfo.talend.com/rs/...en_di_talend_metadatamanager.pdf · environments have...
TRANSCRIPT
![Page 1: Talend Metadata Managerinfo.talend.com/rs/...EN_DI_Talend_MetadataManager.pdf · environments have their own repositories and metadata mapping and ... • This new ETL process is](https://reader031.vdocument.in/reader031/viewer/2022020315/5aa987e27f8b9a6c188cfb8b/html5/thumbnails/1.jpg)
TalendMetadataManager
ReduceRiskandFrictioninyourInformationSupplyChain
![Page 2: Talend Metadata Managerinfo.talend.com/rs/...EN_DI_Talend_MetadataManager.pdf · environments have their own repositories and metadata mapping and ... • This new ETL process is](https://reader031.vdocument.in/reader031/viewer/2022020315/5aa987e27f8b9a6c188cfb8b/html5/thumbnails/2.jpg)
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage2Tel:+1(650)5393200
TalendMetadataManagerTalend Metadata Manager provides a comprehensive set of capabilities for all facets ofmetadata management. At the heart of Talend Metadata Manager is a repository whichcontains repository objects, such asmodels andmappings that are organized into folders.Models can be harvested from TalendData Integrationmodels, DataModeling tools, DataWarehouses, external metadata repositories for relational databases (RDBMS), and DataIntegration and Business Intelligence tools. A particular type of repository object calledConfiguration,canconnect“metadatastitching”modelsandmappingstogethertorepresentanEnterpriseArchitecture,includingfullsupportfordataflowlineageandimpactanalysis,aswellassemanticlineagedefinitions.
TalendMetadataManagerconsistsoffourmajorcomponents:
• MetadataBridge(metadataimport)• MetadataManager• DataGovernance• MetadataAuthoringwithForwardEngineering(metadataexport)
![Page 3: Talend Metadata Managerinfo.talend.com/rs/...EN_DI_Talend_MetadataManager.pdf · environments have their own repositories and metadata mapping and ... • This new ETL process is](https://reader031.vdocument.in/reader031/viewer/2022020315/5aa987e27f8b9a6c188cfb8b/html5/thumbnails/3.jpg)
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage3Tel:+1(650)5393200
MetadataBridge
Metadataiseverywhere.Datawarehousing,businessintelligence,CASEandETLtoolsallhavetheirownrepositories.Justabouteveryapplicationhasitsowndatadictionary.XMLcarriesthe metadata with it in the message or document, and enterprise application integrationenvironmentshavetheirownrepositoriesandmetadatamappingandintegrationfacilities.Inordertosucceed,onemusthaveagoodenterpriserepositoryintegrationenvironmentthatcanintegratethedifferentformatofmetadatafromalltools.TheTalendMetadataManagerrepositorybridgesthetechnicalandnon-technicalaspectsofmetadata,whilesimultaneouslyaddressing the chasm between the different metadata source and target systems thatconstituteanymoderninformationmanagementenvironment.The Metadata Bridge imports all metadata via “bridges” (metadata import components),including Extract, Transformation and Load (ETL)/ Data Integration tools, BusinessIntelligencetools,DataModelingtools,databases,mostallmetadataexchangestandards,andnumerousdataformatsincludingXML.
ImportingmetadatafromTalendStudiowithTalendMetadataManager
![Page 4: Talend Metadata Managerinfo.talend.com/rs/...EN_DI_Talend_MetadataManager.pdf · environments have their own repositories and metadata mapping and ... • This new ETL process is](https://reader031.vdocument.in/reader031/viewer/2022020315/5aa987e27f8b9a6c188cfb8b/html5/thumbnails/4.jpg)
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage4Tel:+1(650)5393200
MetadataManager(MM)
VersionandConfigurationManagementNotonlymusttherepositorybeabletoimportondemandinanyformatandtoanytoolorimportmetadatamanytimesasneeded,itmustbeabletomanagetheversionscreatedbythiscontinuous activity. It must also be fundamental to the repository organization foradministrators to then organize, publish and selectively present the information inappropriateconfigurationsofmetadata,asisrequiredforthecorrectandpreciseanswerstoawiderangeof“cuts”acrossthismetadata.TalendMetadataManagerwasdesignedfromthegroundupwithversionandconfigurationmanagementasakeycapability.
MetadataComparisonAllmetadataisrepresentedbyanintegratedmetamodelinTalendMetadataManager.Thisfeatureprovidescomparisonsacrossmetadatafromdatasourceformatssupported,includingdesigntools,databases,etc.,notsimplyamongversionsofagivenmodel.
ComparingmodelsormodelversionswithTalendMetadataManager
![Page 5: Talend Metadata Managerinfo.talend.com/rs/...EN_DI_Talend_MetadataManager.pdf · environments have their own repositories and metadata mapping and ... • This new ETL process is](https://reader031.vdocument.in/reader031/viewer/2022020315/5aa987e27f8b9a6c188cfb8b/html5/thumbnails/5.jpg)
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage5Tel:+1(650)5393200
DataMappingSpecificationsOnceimported,metadatacanbemappedinamyriadofwaystoanyothermetadatawithinTalendMetadataManager.Thisabilityiscriticaltothesuccessofanymetadatamanagementsolution. Inparticular, youcandefinedata flowmappings describingdatamovement typerelationships,e.g.whenadatabaseisreadandtheresultswrittentoanotherdatabase,aswellas semanticmappingswhich identify semantic relationships between elements, oftentimesconceptualorlogicalinnature,suchasforadatadictionaryorconceptualmodelsuchasaUMLmodel.
MetadataStitchingMetadatastitchingisfundamentaltothecorrectandautomatedanalysisofthedataflowandsemanticlineageofmetadataintherepository.Italsosupportsversionmanagementacrosstheconstantrateofupdatesandchangesinarepository.TalendMetadataManagerkeepscompleteversionsofallimportedmetadatainself-contained“models”,whicharethenrelatedviastitching’s(simpleconnectionmappings). Inthisway,versionmanagementandconfigurationmanagement isnotonlyentirelycleanandisolatedfromthedefinitionandmaintenanceofmappings,italsoautomaticallysupportsupdatesandchangesintothefuture.
Gettingahighlevelviewofinformationflowsacrosssystemswithmetadatastitching
![Page 6: Talend Metadata Managerinfo.talend.com/rs/...EN_DI_Talend_MetadataManager.pdf · environments have their own repositories and metadata mapping and ... • This new ETL process is](https://reader031.vdocument.in/reader031/viewer/2022020315/5aa987e27f8b9a6c188cfb8b/html5/thumbnails/6.jpg)
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage6Tel:+1(650)5393200
In this way, the enterprise architecture is correctly modeled, and data flow lineage iscompletelyandaccuratelyderivable.
Thedifferentrolesandtheirneedswithrespecttodataandrelatedmetadata
LineageandImpactAnalysisOncemetadata ismanaged,metadata is then available for detailed technical and businessanalysis. TalendMetadata Manager supports full technical and business level lineage andimpactanalysisprovidingyounewinsightacrossalltheconnectedmetadatasources.
BusinessUser–LineageReportinganalysisisthetypicalusecase,withquestionssuchas:
• Givenanitemonareport,whatdataentrysystemfieldsimpacttheseresults?• Whyarethenumbersonthisreportthewaytheyare?• HowdoIchangethesystemdatatocorrecttheresultsofthisreport?
DatalineagewithTalendMetadataManager
![Page 7: Talend Metadata Managerinfo.talend.com/rs/...EN_DI_Talend_MetadataManager.pdf · environments have their own repositories and metadata mapping and ... • This new ETL process is](https://reader031.vdocument.in/reader031/viewer/2022020315/5aa987e27f8b9a6c188cfb8b/html5/thumbnails/7.jpg)
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage7Tel:+1(650)5393200
TechnicalUser–ImpactAnalysisOfhighinteresttothetechnicaluserarequestionslike:
• IfImustchangetheseelements(datatype,codesets,etc.)inmyoperationaldatastore,whatisthedownstreamimpact?
• ThisnewETLprocessispopulatingmystagingwarehouseinnewways,howdoesthisimpacttheOLAPmodelinmyreportingservices?
TechnicalUser–LineageReverselineagetypequestionsmayalsobeaskedbymoretechnicalusers,suchas:
• HowmanysystemsarerequiredtodeterminethedimensionsforthisportionoftheOLAPmodel?
• Abusinessreportusecase isaskingthe lineageforparticularvaluesonareport,sowheredoesthedatacomefromandhowisitmanipulated?
BusinessUsers–ImpactAnalysisFinally,businessusersmayasktheforwardlineageorimpactanalysisquestions,suchas:
• IfImakeachangetothisfield,whatreportswillbeimpacted?• How is this identity informationmergedwith the personnel system information on
theseotherreports?
ImpactanalysiswithTalendMetadataManager
![Page 8: Talend Metadata Managerinfo.talend.com/rs/...EN_DI_Talend_MetadataManager.pdf · environments have their own repositories and metadata mapping and ... • This new ETL process is](https://reader031.vdocument.in/reader031/viewer/2022020315/5aa987e27f8b9a6c188cfb8b/html5/thumbnails/8.jpg)
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage8Tel:+1(650)5393200
DataGovernance(DG)
Critical to thedevelopmentandmanagementofa completedataarchitecture isaBusinessGlossary. Talend Metadata Manager provides an ISO 11179-based Business Glossary tocapture,define,maintainandimplementanenterpriseBusinessGlossaryofterminology,datadefinitions,codesets,domains,validationrules,etc.Inaddition,semanticmappingsdescribehowelementsinasourceModel(moreconceptualliketheBusinessGlossary)defineelementsinadestinationModel(closertoanimplementationorrepresentation).TheBusinessGlossaryhelpsanenterprisereachagreementbetweenallstakeholdersontheirbusiness assets (e.g. terms) and how they relate to data assets (e.g. database tables) andtechnology assets (e.g. ETL mappings). The Business Glossary can be used to documentlogical/physicaldataentitiesandattributesacrossITcollaboratively.Again,itinvolvestracingdependenciesbetweenbusinessandtechnicalassets.InTalendMetadataManager,aBusinessGlossaryisaself-containedcollectionofcategoriesand the terms sub-categories containedwithin each category. In turn, the termsmay besemantically mapped to objects throughout the rest of the repository, such as tables andcolumns inadatamodel. Oncemapped,onemayperformsemantic lineage tracessuchasdefinitionlookupsandtermsemanticusageacrossanyconfigurationscontainingtheBusinessGlossary,mappingsandmappedobjects.
AuthoringthecommonbusinesstermsusedintheorganizationwiththeBusinessGlossary
![Page 9: Talend Metadata Managerinfo.talend.com/rs/...EN_DI_Talend_MetadataManager.pdf · environments have their own repositories and metadata mapping and ... • This new ETL process is](https://reader031.vdocument.in/reader031/viewer/2022020315/5aa987e27f8b9a6c188cfb8b/html5/thumbnails/9.jpg)
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage9Tel:+1(650)5393200
BootstrappingaBusinessGlossaryBuildingaBusinessGlossarycanbeassimpleasdragginginanexistingwell-documenteddatamodel,viaimportfromothersources(aCSVfileformat),orcanbepopulateddirectlyviatheuserinterfaceduringtheprocessofclassifyingobjectsinotherdatastoremodels.Ingeneral,acombinationofsuchmethodsareemployedinconjunctionwithoneanother.
WorkflowInordertoensurethattheBusinessGlossaryisaccurate,up-to-date,availabletoallwhoneedaccesstoit,andintegratedproperlywiththerestofthemetadataintherepository,TalendMetadata Manager also provides a robust collection of Data Governance tools andmethodologies. The Business Glossary provides a very flexible workflow and publicationprocessthatcanaddressbothbasicandcomplexneeds.Inaddition,onemaymaintainanynumberofbusinessglossaries,eachwithdifferentworkflowandpublicationcharacteristics.TheBusinessGlossarymaybepartofyourlineage.Itwillappearintherepositorypanelandwhen you open a Business Glossary, youwill be presentedwith a different UI than other(imported)Models.
Workflow-drivensearchcriteriaareavailableallowingonetoefficientlyorganizetermsandidentifywhatactionsarerequiredatanygiventime.Whenworkingwith individual terms,whichareatsomepoint intheworkflowprocess,workflowtransitionbuttonspromptyouwithpossibleactions.
SemanticMappingA SemanticMapping describes how elements in a sourcemodel (more conceptual) defineelementsinadestinationmodel(closertoanimplementationorrepresentation).Putanotherway, elements in the destination model are representations or implementations of theassociatedelementinthesourcemodel.Theyarethreeprimaryusesforsemanticmapping:
• DataStandardizationandCompliance• Multi Level Modeling of semantic relationships from conceptual to logical, and to
physicaldatamodelwithafewsubcases• BusinessGlossarytermclassification
![Page 10: Talend Metadata Managerinfo.talend.com/rs/...EN_DI_Talend_MetadataManager.pdf · environments have their own repositories and metadata mapping and ... • This new ETL process is](https://reader031.vdocument.in/reader031/viewer/2022020315/5aa987e27f8b9a6c188cfb8b/html5/thumbnails/10.jpg)
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage10Tel:+1(650)5393200WP208-EN
MetadataAuthoring(MA)withForwardEngineering(MetadataExport)Note:ThefollowingfeaturesonlycomewithTalendMetadataManagerwithAuthoring.
RDBMSandBigDataDocumenterandPhysicalDataModelerThe Talend Metadata Manager Data Documenter allows users to document existing datastores, like databases, big data sources, and imported models, and publish the resultingdocumenteddatastorestotheenterprise.TheDataDocumenteroffersadifferentapproachthantraditionaldatamodelingtools:
• The Business Glossary-driven Data Documentermethodology allows for immediatereuseandcreationoftermsandnamingstandardsonthefly,fasttrackingthedatastoredocumentationprocessensuringcompletesemanticsynchronizationamongyourdatamodelsanddatagovernanceenvironment.
• Web-enabledDataDocumenteroffersbetteraccesstousersthandesktoptools• DataModeling anddiagramming capabilities of theDataDocumenter are similar to
conventionaldatamodelingtools.• Fullintegration(import/export)tomostpopulardatamodelingtoolsisprovided.
VisualizingDataModelswithTalendMetadataManager
![Page 11: Talend Metadata Managerinfo.talend.com/rs/...EN_DI_Talend_MetadataManager.pdf · environments have their own repositories and metadata mapping and ... • This new ETL process is](https://reader031.vdocument.in/reader031/viewer/2022020315/5aa987e27f8b9a6c188cfb8b/html5/thumbnails/11.jpg)
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage11Tel:+1(650)5393200WP208-EN
LogicalDataModelerTalend Metadata Manager provides a completely web-enabled logical data modelingenvironmentforproducinglogicalandconceptualmodels:
• TheBusinessGlossary-drivenmethodology allows for immediate reuse (creating ofentities,attributesanddomains)andcreationoftermsandnamingstandardsonthefly, fast tracking the modeling process and ensuring complete semanticsynchronizationamongyourmodelsanddatagovernanceenvironment.
• TheWeb-enabledmodeleroffersbetteraccesstousersthandesktoptools.• TheDataModelingcapabilitiesarecompetitivewithconventionaldatamodeling
tools.• Fullintegration(import/export)withmostpopulardatamodelingtoolsisprovided.
DataMappingDesignerData Mapping Designs represents data integration process designs containing all thenecessarydatamovementdesigndetails, such as lookups, filters, joins and transformationexpressions. TheseDataMappingDesignsare completeenough that theymaybe forwardengineered into Talend Data Integration using the Metadata Bridge. In this way, TalendMetadataManagerprovidesacompletelyweb-baseddatamappingdesigntoolthatcanreuseandbesynchronizedwithallothermetadataartifactsintherepositoryandyourcompletedatagovernanceenvironment.
DefiningthemappingsdirectlyinTalendMetadataManager
![Page 12: Talend Metadata Managerinfo.talend.com/rs/...EN_DI_Talend_MetadataManager.pdf · environments have their own repositories and metadata mapping and ... • This new ETL process is](https://reader031.vdocument.in/reader031/viewer/2022020315/5aa987e27f8b9a6c188cfb8b/html5/thumbnails/12.jpg)
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage12Tel:+1(650)5393200WP208-EN
VisualizingtheendtoendinformationflowswithTalendMetadataManager