what's new in powercenter and powerexchange 9 · pdf fileinformatica powercenter...
TRANSCRIPT
1
What’s New In PowerCenter and PowerExchange 9.5
Wei Zheng
Senior Director, Product Management
Informatica
John Haddad
Director, Product Marketing
Informatica
2
Agenda
• Data Integration Overview and Update
• What’s New in PowerCenter v9.5
• Data Integration Analyst Option
• PowerCenter Data Virtualization Edition
• Data Validation Option
• Proactive Monitoring for PowerCenter
• Metadata Manager Update (Advanced Edition)
• PowerCenter for Hadoop
• What’s New in PowerExchange v9.5
3
Safe Harbor
The information being provided today is for
informational purposes only. The development, release
and timing of any Informatica product or functionality
described today remain at the sole discretion of
Informatica and should not be relied upon in making a
purchasing decision. Statements made today are based
on currently available information, which is subject to
change. Such statements should not be relied upon as
a representation, warranty or commitment to deliver
specific products or functionality in the future.
4
Gartner Magic Quadrant for
Data Integration Tools, 2011
The Dominant Leader in Data Integration
Forrester Wave: Data
Virtualization, Q1 ‘12
Power of
The Platform
Forrester Wave: Enterprise
ETL, Q1 ‘12
5
Big Trends Defining the DI Market Big Data, Data Virtualization, Self Service, Governance
Web
Services SQL Batch
Analytics Interactions Transactions Archived
PRODUCT CUSTOMER CLAIM ORDER
Home
Search all Go
Term Definition Relationship Diagram History Business Lineage Comments
Term: Past Due
Payments System
Due Date
Grace Period
Mortgage App
In Loan Modification
Past Due
Customer Vision
Past Due Accounts Report
Pending Foreclosure Report
6
Data Integration Analyst Option
7
How Long Does a Change Take?
Change
Request
Deploy to
Production
Business IT
3-6 Months
Reporting Scenario: On-going requests for data that is NOT in the DW
?
Change Request … Approve & Prioritize … Analyze & Design … Build … Test … Deploy
Weeks/Days
What if?
•66% of BI requirements change on between a daily and monthly basis
•71% of the respondents said they have to ask data analysts to create custom reports for them
•36% of custom report requests require a custom cube or data mart to answer the request
•77% of respondents cited that it takes between days and months to get their BI requests fulfilled
Source: Forrester Research, “Agile BI: Best Practices for Breaking Through the BI Backlog,” 2010
8
Data Integration Analyst
Data Integration Analyst Option For PowerCenter & Data Services (Released in 9.1)
• Easily map data sources to both physical and virtual data targets
• Quickly find data via integrated business glossary
• Specify transformation logic using reusable expressions
• Include pre-built rules and mappings (e.g. ETL, data quality)
• Test and validate specification results
• Efficiently collaborate with developers
• Automatically generate PowerCenter mappings and publish SQL views
Informatica
PowerCenter
Informatica
Data Services
New Option
• Immediate access to data
• Savings in project duration and cost
• Improved analyst and developer productivity due to better business-IT collaboration
9
Which source
attributes map to
target attributes?
What transformations
and rules are required
to map the sources to
targets? Easily map data
sources to both
physical and virtual
data targets
Include pre-built ETL
and data quality rules
as transformations in
mapping specification
Include transformation
descriptions to instruct
developer
Specify transformation
logic using reusable
expressions Preview and validate
specification results
Analyst Defines and Validates Mapping Specification
10
Extend the generated
mapping for rapid
development Mapping logic
automatically generated
from analyst defined
specification
Extend mapping to
consume web services Extend mapping to
dynamically mask
sensitive data
Easily Extend Auto-generated Mapping
11
Deploy as a SQL view
What is the fastest
way to deploy this
change to
production
Deploy as a web service
Deploy to PowerCenter
Rapidly Deploy as a SQL View, Web Service or Batch ETL
12
PowerCenter Data Integration Analyst Option
• New in 9.5
• Guided Mappings – Join recommendations from metadata, auto-map source & target, midstream data preview
• Usability improvements for mapping specification edit and join logic definition
• Embedded profiling within mapping specifications
• Better PowerCenter collaboration using descriptions
• SQL query interface against mapping specification
• Support for adapter SDK
Informatica proprietary/confidential. Informational purposes only. No representation, warranty or commitment
regarding future functionality. Not to be relied upon in making purchasing decision.
13
PowerCenter Data Virtualization Edition
14
Enterprise Challenges
• Organizations have an incomplete, old and
inaccurate view of the business because:
• It takes too long to deliver integration projects (Example – Business Intelligence (BI), MDM, SOA)
• Many iterations are needed to get the requirements right
• The underlying architecture is inflexible making it too hard to support change
NO AGILITY & PRODUCTIVITY
15
Portal BI Composite Apps
Enterprise
Data Sources
Data
Abstraction
Logical Data Objects
PRODUCT … CUSTOMER ORDER
Data
Consumers
Logical View of All Underlying Data
Data Virtualization Can Help
Think Virtual Machines for DATA!
FAST, DIRECT
ACCESS
ABSTRACTION &
REUSE
COMMON DATA
ACCESS LAYER
KEY USE CASES
BI / DW MDM SOA
16
PC DVE – Key Use Cases
Weeks/Days
Change
Request
Deploy to
Production
Business IT DW/Business Intelligence (BI) Prototype DW & accelerate new data
& reports from months to days
1
MDM Deliver a complete view of master &
transactional data in real-time
2
Months
INCOMPLETE VIEW
OF CUSTOMER
MDM
HUB
TRANSACTIONAL
SYSTEMS
DATA
WAREHOUSE
Vir
tual
Vie
w
17
Introducing…
PowerCenter
Data Virtualization Edition
Data Federation
Developer Tool
Analyst Tool
Data Profiling
ETL
(PC Standard Edition)
Partitioning
NEW
2 Adapters
(PWX for Relational)
New PowerCenter Edition for
AGILITY & PRODUCTIVITY
Combines:
Data integration (PowerCenter SE)
Data Virtualization (IDS Full Use)
Data Profiling (IDE Full Use)
Business-IT Collaboration (Analyst)
Packaged for simplicity and
attractively priced
Reuses existing skills and
resources
Informatica proprietary/confidential. Informational purposes only. No representation, warranty or commitment
regarding future functionality. Not to be relied upon in making purchasing decision.
18
Data Validation Option
19
Why Data Validation?
Data is moved
or transformed
But is it correct?
Data is moved
or transformed
But is it correct? Enterprise Data Enterprise Data
20
Some DVO use cases
• Data Being Transformed
• ETL Reconciliation
• Data Masking
• ETL Testing
• Application Migration
• Data is Identical
• ETL version upgrade
• ETL Migration
• Database migration
• Application Retirement
21
Current Approach: Like a Photo Hunt
22
Current Approach: Stare and Compare
Data Set #1 Data Set #2
23
What is the Data Validation Option?
DVO is a “black box” testing solution
that provides automation,
repeatability and auditability to
virtually any data testing or
reconciliation process.
24
How DVO works with PowerCenter
Data Validation Option
Repository & Warehouse DVO Clients
Reports Database Views
Id: name
name: string
Price: integer
Date in: date
Date out: date
Salary: float
V_Summary
Id: name
name: string
Price: integer
Date in: date
Date out: date
Salary: float
V_Tests
Id: name
name: string
Price: integer
Date in: date
Date out: date
Salary: float
V_Results
Enterprise
Data
Repository and Integration Services
Repository
PowerCenter
Define Tests
Execute Tests
Data Accessed
Results
25
Data Validation Option: Benefits
• Increased likelihood of project success, lower project risk
• Significant cost savings, faster time to market
• 50% source-to-target testing
• 80% - 90% regression / upgrade testing
• Ability to test all data, not just a small sample
• Ability to test in heterogeneous environments
• No need to write SQL
• Complete Audit Trail of all your testing activities
• No need to acquire additional server technology: leverage
PowerCenter’s scalability, platform support and data access
26
PowerCenter Data Validation Option
• New in 9.5
• Focused functionality on Operational (Production) validation scenarios
• Incremental validation • Enhanced reports and dashboards with drill down (Jasper) • Single Sign-on
• Testing across heterogeneous table joins
• Extending data support
• SalesForce.com, SAS, SAP, Mainframe
Informatica proprietary/confidential. Informational purposes only. No representation, warranty or commitment
regarding future functionality. Not to be relied upon in making purchasing decision.
27
Proactive Monitoring for PowerCenter
28
Why Monitor PowerCenter?
• “I have to manually login everyday
to determine if I have any issues.”
• “It is difficult for me to see when
data is trending off the norm by X%.”
• “We monitor a lot of individual
activities but it’s difficult for us to
correlate them.”
• “We have a lot of custom scripts for monitoring, but they are hard to
update (often the developer has left)
and inflexible.”
29
Proactive Monitoring for PC Operations
Customer Written Rules
Built-in Rules
Customizable Rules
Built-in Analytics
Solution Infrastructure
PowerCenter
Runtime
PowerCenter
Repository
PC Monitor
PC User
PC Admin
Data Architect
App Architect
IT Security
Alert Recipients
• Continuous monitoring 24x7
• Pre-built Application + Self-Service Extendable Tool =
Immediate ROI + Continuously increasing ROI
Data Sources
Case Study: Sample rules
Built-in: PowerCenter Session succeeded, but loaded zero row s(Data Architect)
Built-in: PowerCenter Domain is down (PC Admin)
Customizable: PowerCenter Workflow Running Time exceeded SLA (App Architect)
30
Workflow
Controls
Monitor & Alert
Workflow
Statistics
Proactive Monitoring
for PowerCenter (Source Feeds, Rules/Templates,
Watchlists, Alerts)
Real time Workflow
and Session Data
OS (CPU, Memory, Disk)
DB (Tablespace, Listener)
Alerts
(Dashboards, e-mail, DBs, other systems, etc)
PowerCenter
LM API
PowerCenter
Repository
Environment &
Correlation
Enrichment (other monitors, etc)
Governance & Operations In Action
31
Operations – Improve Uptime
Identify issues before workflows fail or lead to erroneous reports
20+ prebuilt alerting rules and templates
NEW environmental variables (CPU, memory, tablespace) to track and correlate against schedules and jobs
Governance – Improve Quality
Identify development best practices violations before they hit production
100s of pre-configured attributes simplify rule customization
Regulate SQL overrides, enforce commit and cache sizes and dozens of other documented practices
What’s New: Proactive Monitoring for PowerCenter
32
Metadata Manager 9.5 Update
33
Metadata Manager
Repository
Personalized Data Lineage
Consolidated Metadata Catalog
Metadata
Reports
3rd party BI
Metadata
Bookmarks
Mainframe Data Modeling
Tools
Relational &
Flat Files ERP BI Tools Custom
Metadata Connectivity
Business Glossary
Informatica proprietary/confidential. Informational purposes only. No representation, warranty or commitment
regarding future functionality. Not to be relied upon in making purchasing decision.
PowerCenter Metadata Manager
34
PowerCenter Metadata Manager
• New in 9.5
• New XConnect for SAP BW 7.0, 7.3
• New XConnect for custom metadata
• Rule-based linking
• Easier to develop and manage:
Validation, user-friendly logging
• Revamped all XConnects for BI and data modeling tools
• New certifications
• Cognos 10, OBIEE 11g, Microstrategy 9.2,
ER/win 8, PowerDesigner 16, ER/studio 9, …
Informatica proprietary/confidential. Informational purposes only. No representation, warranty or commitment
regarding future functionality. Not to be relied upon in making purchasing decision.
35
PowerExchange for 9.5
36
Big Transaction Data PowerExchange for Relational, Data Warehousing, Applications
New with 9.1:
• PWX for Greenplum
• PWX for MS Dynamics CRM
• PWX for Oracle CDC – Direct to Log
• Support for Exadata: Source/target and repository
New with 9.5:
• PWX for SAP: SAP HANA extract/load via ODBC, SAP
for BW 7.3
• PWX SQLServer: Datetime2 support, bulk load
performance
Informatica proprietary/confidential. Informational purposes only. No representation, warranty or commitment
regarding future functionality. Not to be relied upon in making purchasing decision.
37
Big Interaction Data PowerExchange for Social Media
Available with 9.1+:
Connect to leading Social Platforms with
PowerExchange for Facebook, Twitter, LinkedIn
Informatica proprietary/confidential. Informational purposes only. No representation, warranty or commitment
regarding future functionality. Not to be relied upon in making purchasing decision.
38
Checkout Related Breakout Sessions
• Breakout Sessions
• What’s New in Data Services 9.5
• Informatica for Hadoop
• Proactive Monitoring for PowerCenter
• AutoDVO: Data Validation for Production Use Cases
• HOLs (Hands on Lab)
• Powercenter and Self Service DI
• PowerExchange
• Metadata Manager and Business Glossary
• Data Services & Web Services
• Hadoop Data Processing
• hParser
• Proactive Monitoring for PowerCenter
• Informatica Data Validation
39
Questions?