a collaboration of data disciplines and business...

SECOND QUA R T ER 2010

UNIF IED DATA MANAGEMENT

www.tdwi.org

TDWI BEST PRACTICES REPORT

By Philip Russom

A Collaboration of Data Disciplines and Business Strategies

Research Sponsors

ASG

DataFlux

Informatica

SAP

Talend

Teradata

Trillium Software

www.tdwi.org 1

© 2010 by TDWI (The Data Warehousing InstituteTM), a division of 1105 Media, Inc. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. E-mail requests or feedback to [email protected]. Product and company names mentioned herein may be trademarks and/or registered trademarks of their respective companies.

Table of ContentsResearch Methodology . . . . . . . . . . . . . . . . . . . . . . 3

Introduction to Unified Data Management . . . . . . . . . . . 4

Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Definitions of UDM. . . . . . . . . . . . . . . . . . . . . . . . 4

Related Terms and Concepts. . . . . . . . . . . . . . . . . . . 6

Why Care about UDM Now? . . . . . . . . . . . . . . . . . . . 7

The State of Unified Data Management . . . . . . . . . . . . . 8

UDM Adoption Rate . . . . . . . . . . . . . . . . . . . . . . . 8

Benefits of UDM . . . . . . . . . . . . . . . . . . . . . . . . . 8

Barriers to UDM . . . . . . . . . . . . . . . . . . . . . . . . . 10

Organizational Issues for Unified Data Management . . . . . . 12

Points of Cross-Functional Coordination and Collaboration . . . . 12

Organizational Structures for UDM’s Coordination . . . . . . . . . 12

The Role of Data Governance in UDM . . . . . . . . . . . . . . . 13

Business Strategies Supported by UDM . . . . . . . . . . . . . 14

Some Strategic Business Initiatives Need UDM . . . . . . . . . . 14

UDM Has the Potential to Be Highly Strategic . . . . . . . . . . . 15

Data Management Practices Coordinated by UDM . . . . . . . 16

Disciplines Prioritized by UDM Need. . . . . . . . . . . . . . . . 16

A General Framework for UDM. . . . . . . . . . . . . . . . . . . 21

Vendor Platforms for UDM . . . . . . . . . . . . . . . . . . . .22

Defining UDM Suites and Platforms . . . . . . . . . . . . . . . .22

A Framework for UDM Platforms. . . . . . . . . . . . . . . . . . 23

Examples of UDM Platforms. . . . . . . . . . . . . . . . . . . . 24

The Future of UDM Best Practices and Tool Types . . . . . . . 26

Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . 27

By Philip Russom

UNIF IED DATA MANAGEMENTA Collaboration of Data Disciplines and

Business Strategies

SECOND QUARTER 2010TDWI BEST PRACTICES REPORT

2 TDWI RESE ARCH


About the AuthorPHILIP RUSSOM is the senior manager of TDWI Research at The Data Warehousing Institute (TDWI), where he oversees many of TDWI’s research-oriented publications, services, and events. Before joining TDWI in 2005, Russom was an industry analyst covering business intelligence (BI) at Forrester Research, Giga Information Group, and Hurwitz Group. He also ran his own business as an independent industry analyst and BI consultant and was contributing editor with Intelligent Enterprise and DM Review magazines. Before that, Russom worked in technical and marketing positions for various database vendors. You can reach him at [email protected].

About TDWIThe Data Warehousing Institute, a division of 1105 Media, Inc., is the premier provider of in-depth, high-quality education and training in the BI and data warehousing industry. TDWI is dedicated to educating business and information technology professionals about the strategies, techniques, and tools required to successfully design, build, and maintain data warehouses. It also fosters the advancement of data warehousing research and contributes to knowledge transfer and the professional development of its Members. TDWI sponsors and promotes a worldwide Membership program, quarterly educational conferences, regional educational seminars, onsite courses, solution provider partnerships, awards programs for best practices and leadership, resourceful publications, an in-depth research program, and a comprehensive Web site (www.tdwi.org).

About the TDWI Best Practices Reports SeriesThis series is designed to educate technical and business professionals about new BI technologies, concepts, or approaches that address a significant problem or issue. Research for the reports is conducted via interviews with industry experts and leading-edge user companies and is supplemented by surveys of BI professionals.

To support the program, TDWI seeks vendors that collectively wish to evangelize a new approach to solving BI problems or to an emerging technology discipline. By banding together, sponsors can validate a new market niche and educate organizations about alternative solutions to critical BI issues. Please contact TDWI Research Director Wayne Eckerson ([email protected]) to suggest a topic that meets these requirements.

AcknowledgmentsTDWI would like to thank many people who contributed to this report. First, we appreciate the many users who responded to our survey, especially those who responded to our requests for phone interviews. Second, our report sponsors, who diligently reviewed outlines, survey questions, and report drafts. Finally, we would like to recognize TDWI’s production team: Jennifer Agee, Marie Gipson, Rod Gosser, and Denelle Hanlon.

SponsorsASG, DataFlux, Informatica, SAP, Talend, Teradata, and Trillium Software sponsored the research for this report.

www.tdwi.org 3

Research Methodology

PositionCorporate IT professionals 66%

Consultants 19%

Business sponsors/users 15%

Industry Consulting/professional services 17%

Financial services 13%

Healthcare 11%

Software/Internet 9%

Insurance 8%

Government: federal 5%

Government: state/local 4%

Manufacturing (non-computers) 4%

Retail/wholesale/distribution 4%

Education 3%

Telecommunications 3%

Utilities 3%

Other 16%

(“Other” consists of multiple industries, each represented by 2% or less of respondents.)

Geography United States 60%

Europe 15%

Asia 9%

Canada 8%

Central/South America 3%

Africa 2%

Australia 2%

Other 1%

Company Size by Revenue Less than $100 million 18%

$100 to $500 million 13%

$500 million to $1 billion 14%

$1 to $5 billion 18%

$5 to $10 billion 9%

More than $10 billion 16%

Don’t know 12%

Based on 179 survey respondents.

Research MethodologyReport scope. Data and other information are managed in isolated silos by independent teams using various data management tools for data integration, quality, profiling, federation, master data management, and so on. As an antidote to the silos, this report describes a trend toward unified data management (UDM), a practice that holistically coordinates teams and integrates tools. The report shows data management professionals and their business sponsors how the coordination of diverse data disciplines yields greater efficiency for technical solutions and the teams that build them, plus a tighter alignment of data management tasks to business goals.

Research methodology. Most of the market statistics presented in this report are based on a survey. In November 2009, TDWI sent an invitation via e-mail to the data management professionals in its database, asking them to complete an Internet-based survey. The invitation was also distributed via Web sites, newsletters, and conferences from TDWI and other firms. The survey drew complete responses from 195 survey respondents. From these, we excluded respondents who identified themselves as academics or vendor employees, leaving the responses of 179 respondents as the core data sample for this report.

Survey demographics. The wide majority of survey respondents are corporate IT professionals (66%), whereas the remainder consists of consultants (19%) or business sponsors/users (15%). We asked consultants to fill out the survey with a recent client in mind.

The consulting (17%) and financial services (13%) industries dominate the respondent population, followed by healthcare (11%), software (9%), and insurance (8%). Most survey respondents reside in the U.S. (60%) or Europe (15%). Respondents are fairly evenly distributed across all sizes of companies and other organizations.

Other research methods. In addition to the survey, TDWI Research conducted numerous telephone interviews with technical users, business sponsors, and recognized experts in the field of data management. TDWI also received product briefings from vendors that offer products and services related to the best practices under discussion.

4 TDWI RESE ARCH


Introduction to Unified Data Management

BackgroundIn most organizations today, data and other information are managed in isolated silos by independent teams using various data management tools for data quality, data integration, data governance and stewardship, metadata and master data management, B2B data exchange, content management, database administration and architecture, information lifecycle management, and so on. In response to this situation, some organizations are adopting what TDWI calls unified data management (UDM), a practice that holistically coordinates teams and integrates tools. Other common names for this practice include enterprise data management and enterprise information management. Regardless of what you call it, the “big picture” that results from bringing diverse data disciplines together yields several benefits, such as cross-system data standards, cross-tool architectures, cross-team design and development synergies, leveraging data as an organizational asset, and assuring data’s integrity and lineage as it travels across multiple organizations and technology platforms.

But unified data management isn’t purely an exercise in technology. Once it paves the way to managing data as an organizational asset, the ultimate goal of UDM becomes to achieve strategic, data-driven business objectives, such as fully informed operational excellence and business intelligence, plus related goals in governance, compliance, business transformation, and business integration. In fact, the challenge of UDM is to balance its two important goals—uniting multiple data management practices and aligning these with business goals that depend on data for success.

The purpose of this report is to help organizations plan and execute effective UDM efforts. Many need the help, because UDM is a relatively new shift in best practices for data management. Toward that end, the report drills into the business initiatives that need UDM, the data management practices and tools that support it, and the organizational structures that enable the cross-functional collaboration that’s critical to UDM success.

Definitions of UDM With all the above in mind, here’s a nutshell definition of UDM:

TDWI Research defines unified data management as a best practice for coordinating diverse data management disciplines, so that data is managed according to enterprisewide goals that promote technical efficiencies and support strategic, data-oriented business goals.

The term UDM itself seems focused on data management, which suggests that it’s purely a technical affair. But this is misleading, because UDM—when performed to its full potential—is actually a unification of both technology practices and business management. For UDM to be considered successful, it should satisfy and balance both of the following requirements:

UDM coordinates diverse data

management disciplines and aligns them to data-

driven business goals.

UDM unifies both technology and business

practices.

www.tdwi.org 5

Introduction to Unif ied Data Management

• UDM must coordinate diverse data management disciplines. This is mostly about coordinating the development efforts of data management teams and enabling greater interoperability among their servers. UDM may also involve the sharing or unifying of technical infrastructure and data architecture components that are relevant to data management. There are different ways to describe the resulting practice, and users who have achieved UDM call it a holistic, coordinated, collaborative, integrated, or unified practice. Regardless of the adjective you prefer, the point is that UDM practices must be inherently holistic if you are to improve and leverage data on a broad enterprise scale.

• UDM must support strategic business objectives. For this to happen, business managers must first know their business goals, then communicate data-oriented requirements to data management professionals and their management. Ideally, the corporate business plan should include requirements and milestones for data management. Hence, although UDM is initially about coordinating data management functions, it should eventually lead to better alignment between data management work and information-driven business goals of the enterprise. When UDM supports strategic business goals, UDM itself becomes strategic.

Let’s expand TDWI’s terse definition of UDM by drilling into more specific details and issues.

UDM is largely about best practices from a technical user’s viewpoint. Most UDM work involves collaboration among data management professionals of varying specialties (such as data integration, quality, master data, etc.). The collaboration fosters cross-solution data and development standards, interoperability of multiple data management solutions, and a grander concept of data and data management architectures.

UDM isn’t a single type of vendor-supplied tool. Even so, a few leading software vendors (including all the vendor companies sponsoring this report) are shaping their offerings into UDM platforms. Such a platform consists of a portfolio of multiple tools for multiple data management disciplines, the most common being BI, data quality, data integration, master data management, and data governance. For the platform to be truly unified, all tools in the portfolio should share a common graphical user interface (GUI) for development and administration, servers should interoperate in deployment, and all tools should share key development artifacts (such as metadata, master data, data profiles, data models, etc.). Having all these conditions is ideal, but not an absolute requirement. As one interviewee put it: “The tools’ servers have to interoperate or—at the end of the day—the solution isn’t unified. So that’s a ‘must have,’ as is shared metadata. If there are multiple development GUIs, I can live with that.”

UDM often starts with pairs of practices. UDM is a matter of degree. In other words, it’s unlikely that any organization would want or need to coordinate 100% of its data management work via UDM or anything similar. Instead, organizations opportunistically select combinations of data management practices whose coordination and collaboration will yield appreciable benefits. The most common combinations are pairs, as with data integration and data quality or data governance and master data management. Over time, an organization may extend the reach of UDM by coalescing these pairs and adding in secondary, supporting data management disciplines, such as metadata management, data modeling, and data profiling. Hence, the scope of UDM tends to broaden over time into a more comprehensive enterprise practice. And the scope can get rather broad, as a user interviewed for this report explained: “Enterprise-scale data management is like most things: it’s a mix of people, process, and technology. The range of each is diverse, so there’s potentially a place for just about anything.”

UDM is mostly about user practices, but also vendor tools, business strategy, and organizational units.

6 TDWI RESE ARCH


A variety of organizational structures can support UDM. It can be a standalone program or a subset of larger programs for IT centralization and consolidation, IT-to-business alignment, data as an enterprise asset, and various types of business integrations and business transformations. UDM can be overseen by a competency center, a data governance committee, a data stewardship program, or some other data-oriented organizational structure. UDM is often executed purely within the scope of a program for BI and data warehousing (DW), but it may also reach into some or all operational data management disciplines (such as database administration, operational data integration, and enterprise data architecture).

UDM unifies many things. As its name suggests, it unifies disparate data disciplines and their technical solutions. On an organizational level, it also unifies the teams that design and deploy such solutions. The unification may simply involve greater collaboration among technical teams, or it may involve the consolidation of teams, perhaps into a data management competency center. In terms of deployed solutions, unification means a certain amount of interoperability among servers, and possibly integration of developer tool GUIs. Technology aside, UDM also forces a certain amount of unification among business people, as they come together to better define strategic business goals and their data requirements. When all goes well, a mature UDM effort unifies both technical and business teams through IT-to-business alignment germane to data management.

Related Terms and ConceptsMost likely, you’ve never heard the term “unified data management.” After all, most organizations coordinating diverse data management disciplines do so without giving their actions a name. For example, the survey for this report asked: “What do you call the coordination of data management disciplines in your organization?” Two-thirds (66%) of survey respondents answered: “We don’t have a formal name for it.” (See Figure 1.) Likewise, in the user interviews conducted for this report, users and consultants alike described how they coordinate data management work and align it with stated business goals for data—but few had a name for it. Even stranger, most software vendors that offer a portfolio of multiple data management tools have no term for the coordinated use of the portfolio!

However, a third of survey respondents (34% in Figure 1) have given coordinated data management a name, and they typed that name into the online survey. The names they report using reveal much about how users think about such coordination. (See Figure 2.)

Generic terms for UDM. A lot of users keep it simple by referring to their coordinated efforts as simply data management (16%) or information management (11%). In fact, most users interviewed for this report stated that they just assume that good data management involves technical people of diverse disciplines learning from each other, complying with data and development standards, considering cross-discipline architectures, and the other best practices this report associates with UDM.

Generic terms, but with enterprise aspirations. If you put the word “enterprise” in front of common terms like data management and information management (thus denoting a broad enterprise scope), then you get two of the most popular terms entered into this report’s survey: enterprise data management (EDM, 15%)1 and enterprise information management (EIM, 11%)2. By coincidence, these acronyms are strongly associated with vendors that have promoted them, namely EIM with SAP and Business Objects and EDM with DataFlux and SAS.

Most organizations do some form of UDM,

although they have no name for it.

Most users think of UDM as simply a part

of data management or some other data-driven

practice.

1. See the 2009 TDWI Monograph Enterprise Information Management: In Support of Operational, Analytic, and Governance

Initiatives, online at tdwi.org/research/monographs.2. See the 2009 TDWI Checklist Report Enterprise Data Management, online at tdwi.org/research/checklists.

www.tdwi.org 7

Introduction to Unif ied Data Management

UDM as a subset of other programs. A number of survey respondents called their UDM-like activities by the names of other data management practices, in particular master data management (MDM, 11%), data or information governance (8%), BI or data warehousing (7%), and data or information architecture (5%). Each is a rather broad program, and each typically involves multiple data management practices. So it’s possible that users responding to the survey were thinking of UDM as a subset that helps unify the data management solutions created and maintained by these programs.

Why Care about UDM Now?There are many reasons why organizations need to step up their efforts with UDM:

Technology drivers. From a technology viewpoint, the lack of coordination among data management disciplines leads to redundant staffing and limited developer productivity. Even worse, competing data management solutions can inhibit data’s quality, integrity, consistency, standards, scalability, architecture, and so on. On the upside, UDM fosters greater developer productivity, cross-system data standards, cross-tool architectures, cross-team design and development synergies, and assuring data’s integrity and lineage as it travels across multiple organizations and technology platforms.

Business drivers. From a business viewpoint, data-driven business initiatives (including BI, CRM, regulatory compliance, and business operations) suffer due to low data quality and incomplete information, inconsistent data definitions, non-compliant data, and uncontrolled data usage. UDM helps avoid these problems, plus it enables “big picture” data-driven business methods such as data governance, data security and privacy, operational excellence, better decision making, and leveraging data as an organizational asset.

Common terms for UDM, according to survey respondentsData management 16%

Enterprise data management 15%

Enterprise information management 11%

Information management 11%

Master data management 11%

Data or information governance 8%

Business intelligence or data warehousing 7%

Data or information architecture 5%

Other 15%

Figure 2. Based on 61 respondents.

What do you call the coordination of data management disciplines in your organization?We don’t have a formal name for it. 66%

We have a name for it. 34%


Technology and business drivers are making the need for UDM more urgent.

8 TDWI RESE ARCH


USER STORY SUCCESS DEPENDS ON THE RIGHT TEAM, SOLUTIONS, MANDATE, AND PLAN.

“As a consultant, I regularly assist our clients with enterprise data management, by explaining the four

things I have found that you need for success,” said Dennis Sweeney, an information management

specialist with Ingenix Consulting, which focuses on the healthcare industry. “First, you need a team

structure that’s conducive to enterprise data management. We recommend a center of excellence, which

can be either physical or virtual, depending on the client’s corporate culture. Second, you need an

established inventory of data management people and solutions. Third, there has to be a high-profile

executive consistently telling everyone they have to participate in enterprise data management. And finally,

there has to be a realistic plan, whereby everyone involved can do their day jobs effectively, while

also collaborating.”

The State of Unified Data Management

UDM Adoption RateThis discussion of UDM and related matters makes you wonder how many user organizations are actually doing it. To find out, this report’s survey asked respondents to “rate the level of formal coordination of the data management disciplines in your organization.” Furthermore, the question requested separate answers for today and in three years. (See Figure 3.)

Today. Survey results show that user organizations are indeed coordinating diverse data management practices today. For the most part, the coordination is at a moderate, low, or very low level. Very few respondents (11%) report coordination at a high or very high level.

In three years. Survey respondents anticipate stepping up the formal coordination of the data management disciplines, such that most (90%) will be doing it at a moderate, high, or very high level within three years. By that time, few organizations plan to be doing UDM at a low or very low level.

The survey shows that UDM indeed exists and some form of it is already practiced in the majority of organizations surveyed. Cross-team coordination and collaboration are base requirements for UDM, which also has loftier goals in IT-to-business alignment. Users, on the whole, are already doing this at a moderate level and plan to move up to a high level within three years. This means growth for UDM, which in turn should amount to firmer support for data-driven business goals.

Benefits of UDMUDM can result in many benefits for data, the enterprise, and the management of each. To get a sense of which benefits are more likely to be realized than others, this report’s survey asked: “Which of the following would improve in your organization if you improved the coordination of multiple data management practices?” The most likely benefits (seen at the top of Figure 4) are those most often selected by survey respondents, and the likelihood of a benefit declines as the list proceeds downward.

UDM enables better business decisions and strategies (65%). Unifying data management practices tends to also unify data, which is good for decision making and strategy development that’s based on data. This is consistent with results charted in Figure 7 and Figure 9 where respondents named BI as the business and technology practice that needs UDM most. Furthermore, the term “trusted data” came up in many of the user interviews conducted for this research. According to interviewees, UDM is instrumental in building confidence in data and decisions based on data, which furthers the cause of fact-based decision making.

Most users already do UDM today, and they

will do more within three years.

UDM benefits range from better

business decisions and performance to

improvements in data’s quality, integration, and

governance.

www.tdwi.org 9

The State of Unif ied Data Management

UDM helps focus data management where enterprise goals need it most. At least, that’s the perception of respondents who ranked highly the business leverage of data assets (41%), business-to-IT alignment per data (37%), and data-driven corporate objectives (19%).

UDM contributes to business performance and execution (35%). Likely benefits may also apply in related areas, such as cross-unit business processes (28%), customer experience and service (26%), and operational efficiency (20%).

UDM improves the condition of data. This is especially apparent with data quality (56%), because success with data quality usually requires that diverse technology teams collaborate. But such cross-team collaboration can also lead to consistent data definition and usage (52%), which may be attained through master data and its management (39%) and data standards (23%).

UDM makes sharing data easier. Therefore, UDM yields likely benefits for data integration (51%), data sharing across business units (34%), and metadata and its management (33%).

UDM assists with some governance activities, but not others. These include data governance and stewardship (51%), data auditability (26%), and compliance relative to data (21%). Even so, some data traits often associated with governance—namely data security (12%) and data privacy (7%)—ranked low in the survey, which makes them unlikely benefits of UDM.

UDM fosters integrated data models (17%). Users noted in interviews that UDM also involves the unification of data models. Often, it’s about standards for data models that lead to consistent data definition and usage, sometimes in the context of MDM. In these cases, data models lay a good foundation for data integration and BI.

UDM is unlikely to boost developer productivity. Some of the least likely benefits of UDM (according to users’ responses) include the reuse of data management work (13%), productivity of data management teams (12%), and synergies among data management teams (10%).

With UDM in mind, rate the level of formal coordination of the data management disciplines in your organization.

Very High2%

14%

High9%

40%

Moderate32%

36%

Low34%

8%

Very Low23%

2%


TODAY

IN THREE YEARS

10 TDWI RESE ARCH


Barriers to UDMUDM has its benefits, as we just saw. Yet, it also has its barriers. Again, to get a sense of which barriers are more likely encountered than others, this report’s survey asked: “In your organization, what are the top potential barriers to coordinating multiple data management practices?” The most anticipated barriers (seen at the top of Figure 5) are those most often selected by survey respondents, and the likelihood of a barrier declines as the list proceeds downward.

UDM is most often stymied by turf issues. These include a corporate culture based on silos (61%) and data ownership and other politics (60%).

Which of the following would improve in your organization if you improved the coordination of multiple data management practices? (Select 10 or fewer.)

Business decisions and strategies 65%

Data quality 56%

Consistent data definition and usage 52%

Data governance and stewardship 51%

Data integration 51%

Business leverage of data assets 41%

Master data and its management 39%

Data integrity 38%

Business-to-IT alignment per data 37%

Business performance and execution 35%

Cost of data management 35%

Enterprise data architecture 35%

Data sharing across business units 34%

Metadata and its management 33%

Cross-unit business processes 28%

Customer experience and service 26%

Data auditability 26%

Data standards 23%

Compliance relative to data 21%

Operational efficiency 20%

Data-driven corporate objectives 19%

Integrated data models 17%

Data freshness or timeliness 15%

Reuse of data management work 13%

Data security 12%

Productivity of data management teams 12%

Views of the business via data 12%

Synergies among data management teams 10%

Data privacy 7%

Development standards 7%

Other 1%

Figure 4. Based on 1,561 responses from 179 respondents (8.7 average responses per respondent).

www.tdwi.org 11

The State of Unif ied Data Management

UDM won’t get to first base without proper leadership. Therefore, UDM suffers when there’s a lack of governance or stewardship (44%), a lack of business sponsorship (42%), or unclear business goals for data (28%). In related areas, inadequate budget for data management (31%) or the lack of a compelling business case (19%) can also be barriers. As with many data management programs (such as data governance and BI), UDM needs an executive sponsor who can provide leadership and a mandate that prevents squabbles among owners of data and applications.

UDM is tough to pull together across multiple teams. The distribution of data management over multiple organizations (28%) and the independence of data management teams (23%) constitute considerable barriers. These are exacerbated by the fact that data management professionals are leery of consolidating or reorganizing data management teams (20%).

UDM is unlikely when the data has problems. These problems include poor master data or metadata (32%) and poor quality of data (24%). In a related area, inadequate data management infrastructure (28%) may also be a problem.

UDM slows down without appropriate tool support. As explained earlier, UDM is mostly about coordinating user-oriented best practices. Yet, UDM’s cause can suffer due to tool issues, such as existing tools not conducive to UDM (20%) and poor integration among data management tools (14%). But these tool issues appear at the bottom of Figure 5, meaning that the probability of a barrier is low.

UDM barriers range from turf issues and inadequate leadership to problems with data and software tools.

In your organization, what are the top potential barriers to coordinating multiple data management practices? (Select six or fewer.)

Corporate culture based on silos 61%

Data ownership and other politics 60%

Lack of governance or stewardship 44%

Lack of business sponsorship 42%

Poor master data or metadata 32%

Inadequate budget for data management 31%

Data management over multiple organizations 28%

Inadequate data management infrastructure 28%

Unclear business goals for data 28%

Poor quality of data 24%

Independence of data management teams 23%

Consolidation/reorganization of data management teams 20%

Existing tools not conducive to UDM 20%

Lack of compelling business case 19%

Poor integration among data management tools 14%

Other 4%

Figure 5. Based on 857 responses from 179 respondents (4.8 average responses per respondent).

12 TDWI RESE ARCH


USER STORY CENTRALIZED DATA MANAGEMENT CAN BE A VARIATION OF UDM.

“Our corporate business strategy is to leverage the strength we get from the diversity of our products,”

said the director of enterprise data management at a well-known consumer products firm. “The catch

is that each product is managed through a fairly independent business unit. We have preserved this

valuable independence and allowed each unit its own data management solutions. Yet, some data must

comply with our enterprise data model that’s defined by a central body and enforced via master data

management. This way, we’ve centralized data management on an as-needed basis—without the literal

centralization and relocation of data—to unify management and process across the business units.

Related to centralized data management, our firm is aggressively adopting [a] centralized [IT infrastructure

library] (for better global standards) and portfolio management (to reduce the diversity of application

brands and their disparate data models).”

Organizational Issues for Unified Data Management

Points of Cross-Functional Coordination and CollaborationIn most organizations, achieving UDM demands a number of adjustments to current technology solutions and business processes. As a consequence, UDM has a few organizational requirements.

Team collaboration. Let’s recall that many organizations manage data in isolated silos strewn across the enterprise, using a variety of tools and teams. A certain amount of coordination among these teams is inevitable, because their solutions interact. For example, data integration tools regularly call data quality tools; data federation tools connect to operational databases; and data quality and master data management solutions often require changes to other, related data management solutions. Although this is a good start, UDM demands much more. Ideally, the combination of all data management solutions involved should form a recognizable architecture—even if it’s loosely federated—that’s held together by agreed-upon methods for the deep tool and platform interoperability that UDM assumes. Getting to this point usually demands the creation of a new, broader team structure, such as a central competency center, a data stewardship program, or a data governance board.

Cross-functional activities. This has many meanings. On the technical side, an implication of UDM is the coordination of multiple data management teams, each with a different discipline. On the business side, UDM demands that multiple business units coordinate data-driven, cross-unit business processes. Though UDM looks like a technology practice, its purpose is to enable business-to-IT alignment relative to data management, which is also cross-functional. As the subtitle of this report says, UDM is “a collaboration of data disciplines and business strategies.”

Organizational Structures for UDM’s CoordinationIt’s unlikely you need an organizational structure devoted to UDM. As we’ll see, UDM overlaps with other data-driven programs and organizational structures that you probably already have. So it may be best to make UDM a set of requirements and goals for these other structures. With all that in mind, it’s up to users to determine to which pre-existing teams or programs they should take their UDM requirements.

UDM is inherently collaborative and cross-

functional.

www.tdwi.org 13

Organizational Issues for Unif ied Data Management

To provide some guidance in this area, this report’s survey asked: “Which of the following organizational structures do you feel are best equipped to successfully coordinate diverse data management teams?” (See Figure 6.) The leading groups identified by survey responses are the data warehouse or BI team (55%), the enterprise data architecture group (54%), and the data governance board or committee (54%). Other organizational structures likely to handle UDM requirements successfully include data stewardship programs (43%), data integration competency centers (32%), and the CIO’s office (28%).

Note that most of these are meta-organizations. In other words, bodies that reach across multiple teams to enable (perhaps even enforce) some form of coordination across the teams. This is especially true with data architecture groups, governance boards, stewardship programs, and central IT management. Other teams can be meta-organizational in that their influence reaches other teams, whether formally or informally, as is typical of a data warehouse or BI team. The point is that UDM inherently reaches across technical teams, and so some kind of meta-organizational support is a critical success factor.

The Role of Data Governance in UDMUDM’s goal of unification is worth pursuing, but getting there involves a lot of change that must be managed. Furthermore, many of the changes demand collaboration and coordination on multiple levels, including multiple technical teams, multiple business units, and the alignment of both camps. At this level of complexity, the challenge reaches beyond technology and business—UDM becomes an organizational challenge. Hence, UDM success requires assistance from a meta-organization that can enable and manage the coordination.

UDM requires a lot of change, and data governance (DG) manages change very well. Many organizations are addressing the coordination challenges of UDM and similar practices by creating organizational structures for various kinds of governance. These may be relatively high-level, like corporate governance or IT governance. However, DG is a specific form of governance that’s most relevant to UDM.

Which of the following organizational structures do you feel are best equipped to successfully coordinate diverse data management teams? (Select five or fewer.)

Data warehouse or BI team 55%

Enterprise data architecture group 54%

Data governance board or committee 54%

Data stewardship program 43%

Data integration competency center 32%

CIO’s office 28%

Team of business analysts and other power users 28%

Central IT management 22%

Database administration group 11%

Other 5%


DG gives UDM procedures for change management, among other things.

14 TDWI RESE ARCH


Data governance can be a critical success factor for UDM. Any organization contemplating a UDM implementation should consider creating a process that involves DG or a similar meta-organizational structure. Many firms create a DG program before attempting to use data management tools and practices that require cross-functional coordination and change, as is the case with data quality, metadata management, and master data management. UDM has similar cross-functional requirements, and so a DG program can be a critical success factor for UDM.

USER STORY DATA STEWARDSHIP PROGRAMS AND COMPETENCY CENTERS ARE COMMON FOUNDATIONS FOR UDM.

“We’re in the process of defining a data management organization,” said a director of data management.

“We’re still working on the details, but it’s going to pull together data stewards, data architects, and lots of

data management practitioners, along with their metadata, data models, data profiles, and so on. I hope

to make data stewardship a priority, where each steward is defined as a data owner who can drive data

management work according to business needs. Our data management organization will provide shared

services—akin to a competency center—but with an enterprise perspective.”

Business Strategies Supported by UDM

Some Strategic Business Initiatives Need UDMA key assertion of this report is that UDM must go beyond its base requirement of coordinating diverse data management disciplines by also directly supporting the data requirements of strategic business initiatives. Any initiative worth its salt will include metrics and reports for measuring success, and the effect of UDM needs to be part of that measurement. Otherwise, any practice such as UDM can fail to prove value and be de-emphasized over time.

To determine which business initiatives are prime candidates for UDM’s support, this report’s survey asked: “Which business initiatives in your organization do you think would benefit most from UDM?” (See Figure 7.) The order of initiatives determined by survey responses reveals priorities for UDM’s support of strategic business initiatives.

Business intelligence (87%). By a long shot, BI is respondents’ highest priority for UDM support. This is consistent with the survey responses illustrated in Figure 4, where respondents identified better business decisions and strategies as the leading potential benefit of UDM. Closely related to BI, performance management (39%) is also a likely candidate for UDM support.

IT-to-business alignment (44%). This priority makes perfect sense, because UDM focuses on discovering, defining, and satisfying data requirements relative to business initiatives. In other words, UDM is a great way to handle the data portion of IT alignment.

Operational excellence (42%). In today’s data-driven business processes, achieving excellence requires a full arsenal of data management techniques, which UDM can muster. UDM can also assist with important areas within operations, such as customer relationship management (30%) and enterprise resource planning (24%).

Governance (36%). In heavily regulated industries, governing the organization and its data is both a survival ploy and a competitive differentiator. Likewise, UDM assists with data concerns in related initiatives, such as compliance (26%) and risk management (22%).

BI, operations, and governance are

commonly supported by some form of UDM.

www.tdwi.org 15

Business Strategies Suppor ted by UDM

UDM Has the Potential to Be Highly StrategicClearly, some strategic business initiatives need UDM, assuming UDM can do the challenging job of aligning data management work to the priorities of these initiatives. To test whether user organizations are achieving this alignment, this report’s survey asked respondents to “rate how well data management solutions are aligned to strategic business goals.” (See Figure 8a.) Most respondents report attaining a moderate alignment (43%), followed by others with a low (26%) or high (21%) alignment. This proves that aligning data management solutions with strategic business goals is certainly possible. Furthermore, the vast majority of users have done it, and they’ve achieved a respectable level of alignment.

So far, we’ve established that UDM is commonly supporting strategic business initiatives and strategic business goals. But does that make UDM strategic? In general, we don’t automatically think of supporting mechanisms (as most data management teams and their solutions are) as strategic. But they can be, if they support initiatives and goals whose successes are considered key to enterprise survival and growth.

To test perceptions of UDM’s strategic status, this report’s survey asked respondents to “rate how strategic (in terms of being critical to business’s primary goals) UDM could be.” A whopping 59% reported that it could be highly strategic (see Figure 8b), whereas an additional 22% felt it could be very highly strategic. Few survey respondents said that UDM is not very strategic (5%) and no one felt it’s not strategic at all (0%).

Which business initiatives in your organization do you think would benefit most from UDM? (Select five or fewer.)

Business intelligence 87%

IT-to-business alignment 44%

Operational excellence 42%

Performance management 39%

Governance 36%

Customer relationship management 30%

Compliance 26%

Enterprise resource planning 24%

Risk management 22%

IT centralization 12%

Mergers and acquisitions 12%

Corporate reorganizations 8%

Partner programs 4%


Most users already do UDM at a respectable level.

Rate how well data management solutions are aligned to strategic business goals in your organization.

Very high 2%

High 21%

Moderate 43%

Low 26%

Very low 8%

Figure 8a. Based on 179 respondents.

Rate how strategic (in terms of being critical to business’s primary goals) UDM could be in your organization.

Very highly strategic 22%

Highly strategic 59%

Moderately strategic 14%

Not very strategic 5%

Not strategic at all 0%

Figure 8b. Based on 179 respondents.

UDM is strategic when it directly supports strategic business initiatives and goals.

16 TDWI RESE ARCH


This indicates that, in the perceptions of survey respondents, UDM has a strong potential for high strategic impact. By extension, UDM is indeed strategic (despite its supporting role) when it is fully aligned with and satisfying the data requirements of strategic business initiatives and strategic business goals.

USER STORY MOST DATA MANAGEMENT WORK SHOULD SUPPORT STATED BUSINESS GOALS.

“Enterprise data management requires enterprise-level goals relating to data to be both adequately

defined and clearly communicated first, to be sure that we supply information management work that

actually supports the users’ goals,” said Michael Enstrom, the enterprise business intelligence architect

for the University of Wisconsin System. “Otherwise, you may end up with an enterprise information

environment that, for example, has high-quality data or great connectivity, but may still not be what’s

needed functionally. Or it may not be effectively used, due to a lack of accuracy, either in defining

requirements or in identifying all of the correct stakeholders. Without the up-front work to adequately

define both functional and quality-of-service requirements, you may end up putting a lot of time

and effort into perceived improvements that are not going to provide what the business really needs.

Productive decision making depends on accurate, useful information being available both at the time of

decision and afterwards, when analyzing whether those decisions were effective, in terms of meeting the

organization’s goals.”

Data Management Practices Coordinated by UDMA fundamental assumption of this report is that there are many distinct data management practices, and some of them need coordination with others. To dig deeper into this assumption, we need to first list the practices, prioritize them, and break the practices into categories. These actions will reveal what the available data management practices are, which are most open to coordination via UDM, and how diverse data management disciplines relate to each other. Furthermore, this information enables us to construct a framework that users can apply to designing a UDM program or selecting tools for UDM.

Disciplines Prioritized by UDM NeedAs a way of prioritizing data management practices, this report’s survey asked, “In your opinion, which of the following have the most pressing reasons for being unified with other data management practices?” (See Figure 9.) The resulting list puts available data management practices in order by the likelihood of their participation in UDM, with the highest likelihood at the top.

Some data management practices are more open

to UDM’s coordination than others.

www.tdwi.org 17

Data Management Practices Coordinated by UDM

In your opinion, which of the following have the most pressing reasons for being unified with other data

management practices? (Select 10 or fewer.)

Core Data Management Practices

Business intelligence 71%

Data quality 58%

Data warehousing 51%

Master data management 48%

Data governance 47%

Data integration 45%

Enterprise data architecture 42%

Supporting Data Management Practices and Infrastructure

Metadata management 36%

Data stewardship 35%

Application integration 30%

Data modeling 24%

Extract, transform, load (ETL/ELT) 23%

Reporting 22%

Predictive analytics 22%

Data profiling 20%

Process management and integration 17%

Web services, data services, and SOA 16%

Information lifecycle management 14%

Data synchronization 12%

Data federation 12%

Data glossaries 12%

Low Priorities for UDM Today

B2B data exchange 10%

Operational database administration 8%

Cloud-based data management 8%

Complex event processing 6%

Enterprise search 4%

Database migrations 4%

Text analytics (unstructured data) 3%

Database replication 1%

Figure 9. Based on 1,329 responses from 179 respondents (7.4 average responses per respondent).

BI/DW combine into one practice

18 TDWI RESE ARCH


BI and DW are the highest priority for coordination via UDM.Business intelligence (71%) and data warehousing (51%) are at or near the top of the list of data management practices in Figure 9, making them the highest priority for UDM, according to survey responses. (Since most organizations practice BI hand-in-hand with DW, we’ll consider them one practice here.) As proof of BI and DW’s tight association with UDM, let’s recall that Figure 4 named BI as the top beneficiary of UDM, Figure 6 ranked the BI and DW team as the one most likely to field UDM successfully, and Figure 7 identified BI as the business initiative most in need of UDM.

The hegemony of BI and DW makes perfect sense in this context. After all, they are the very epitome of UDM, in that BI and DW coordinate multiple data management practices. Furthermore, BI and DW professionals collaborate with business people to ensure that their data management work satisfies enterprise requirements for data, reports, and analyses. We can easily argue that BI and DW’s success depends on technical coordination and business collaboration.

BI and DW also provide an established model for UDM. One of the users interviewed for this report said: “Our enterprise data management program takes what we learned in data warehousing and applies it more broadly to data management outside of data warehousing, especially in the operational database area, plus sharing data as an asset, in general.” For many organizations, BI and DW are a starting point for UDM, which then spreads into a wider enterprise scope. This helps explain why BI and DW professionals are often involved in UDM, as well as other unifying programs, such as DG, data stewardship, and data architecture.

Although UDM has a prominent association with the analytic systems of BI and DW, it is also relevant to operational applications and operational data. For example, survey data shows that UDM can involve an even mix of both analytic and operational systems (37% in Figure 10). Or it can take a more exclusive approach by involving mostly analytic systems (35%) or mostly operational systems (20%).

UDM focuses on a short list of core data management disciplines.When it comes to coordination via UDM, not all data management practices are created equal. Typically, there are primary practices that form the core of a UDM program or similar effort. These are distinguished from secondary practices that play more of a supporting role in UDM. The core practices are usually high-profile, high-priority practices that require immediate attention and ample resources. They may also inherently demand coordination and collaboration. Furthermore, most people think of the core practices as solutions, which makes them a greater priority than supporting practices.

The UDM portfolio divides into core and supporting practices.

To what types of IT systems is UDM or its equivalent applied in your organization? (Select only one.)

Even mix of both analytic and operational systems 37%

Mostly analytic systems (e.g., BI and DW) 35%

Mostly operational systems (e.g., databases for ERP, CRM, and financial apps) 20%

None of the above 8%


BI and DW provide a model for UDM.

BI and DW are the very epitome of UDM.

www.tdwi.org 19


The six most common core practices in UDM are visible at the top of the prioritized list in Figure 9, namely BI and DW (71% and 51%, respectively), data quality (58%), MDM (48%), DG (47%), data integration (45%), and enterprise data architecture (42%). Survey results aside, anecdotal evidence from TDWI also points to these as core practices, in that all are topics of very popular courses offered by TDWI Education, and all come up prominently in interviews and surveys conducted by TDWI Research. The core practices of a real-world UDM program can vary. But they are usually a subset of the ones listed here.

There are good reasons that UDM’s core disciplines need coordination and collaboration:

• Data integration and quality have a deep symbiosis that needs coordination. Data integration ferrets out data quality issues, whether problems to be fixed or opportunities to be leveraged for improvement. Likewise, data quality reaches more data and systems that need improvement when it employs data integration’s numerous interfaces.

• Some practices aren’t effective without guidance from the business. As one interviewee put it: “Data quality is special, because it never exists on an island. It has to be a response to business pains resulting from poor quality, and its improvements need to be guided by the business. Master data management is pretty much the same situation.”

• Much of UDM’s coordination work is best handled by data governance. Truth be told, DG is not a data management practice per se, because it’s more about processes and rules for handing data than about the actual handling of data. Yet, DG is indispensable, because its collaborative platform for governance can also be applied to data management practices that reach across multiple IT systems or business units–as all core UDM practices do.

• Some practices involve changes to many systems, which DG helps with. In particular, data quality and master data management usually require changes in the applications and databases to which they are applied. The change management processes built into any good DG program can be used to propose and manage cross-system changes resulting from UDM practices.

• Data architecture involves influence on many databases and data management solutions. In the old days, data architecture was about modeling one database at a time. Nowadays, it’s about making incremental improvements to numerous databases, for the sake of data model and metadata standards, broader data sharing, compliance, and information lifecycle management. To exert its influence, enterprise data architecture needs collaborative venues, such as UDM and DG.

• All core practices touch the same data as they repurpose it for their discipline. If for no other reason, this is why core data management practices need coordination via UDM.

UDM also depends on secondary disciplines that play supporting roles.Most of UDM’s supporting practices are seen in the middle of Figure 9, ranging from metadata management to data glossaries. Supporting practices and infrastructure pieces aren’t as high-profile or as sexy as the core practices, but they still provide essential functionality—without which data management wouldn’t be possible:

• Metadata is everywhere, so metadata management is, too. An interviewee explained it this way: “Metadata is one of the biggest challenges, because it’s everywhere, modeled differently, and often hidden. Yet it’s something you have to figure out for every individual data management practice, plus synchronize across unified practices.” This is why metadata management (36%) is the most common UDM supporting practice. Sometimes it’s so prominent that it’s more like a core practice.

UDM’s six core practices represent high-profile, mission-critical solutions.

Supporting practices are low-profile but essential.

20 TDWI RESE ARCH


• Data modeling is a subset of more practices than you might realize. For example, in the world of data warehousing, a lot of data modeling for warehouse data structures is done within a data integration tool when defining the targets of a data flow. Likewise, specialized approaches to data modeling are commonly found in tools for master data management and data quality. Of course, you may prefer to work with a dedicated data modeling tool. This makes data modeling (24%) a very common supporting practice.

• Managing data effectively is tough when you don’t know the current state of data. That’s why data profiling (20%) has risen from relative obscurity 10 years ago to its current status as a common support practice. Many users prefer to profile data manually, which is time-consuming, error-prone, and hard to repeat consistently. Luckily, a few dedicated data profiling tools are available from vendors. Even better, data profiling capabilities are now built into most leading data quality and data integration tools.

UDM needs an infrastructure for unified design and deployment.But this doesn’t mean that UDM has its own autonomous infrastructure. Instead, UDM depends on the servers, development GUIs, and connectivity of the data management tools it’s unifying, plus shared enterprise infrastructure for data integration, application integration, message oriented middleware, various types of services and buses, and service-oriented architecture (SOA).

• Application integration and similar platforms can provide valuable infrastructure. For example, a solution built atop a data management tool usually employs the tool’s built-in interfaces. But the solution may connect to infrastructure for application integration (30%), especially when data should be handled by the logical layer of an application. Likewise, when data is handled within a process, data management solutions may tap into infrastructure for process management and integration (17%). In a similar vein, data management’s quest for new interfaces and broader infrastructure is also leading it to Web services, data services, SOA (16%), and cloud-based data management (8%). This illustrates that UDM’s infrastructure is not a monolith; it’s an amalgam of interfaces and services supported by its constituent tools, plus certain enterprise integration platforms.

• Unified designs need unified design tools. It’s a given that UDM will involve multiple tools, servers, and hand-coded solutions. A mature UDM program will seek integration among multiple data management design tools, to foster consistency across solutions. Likewise, it will seek interoperability among deployed servers, plus connectivity with enterprise operational and analytic applications. From this, we see that—when taken to an extreme—UDM also unifies designs, design tools, and deployed server-based solutions, not just data management teams and their business sponsors.

Database administration and similar tasks are rarely coordinated via UDM.This includes operational database administration (8%), plus tasks in operational data integration, such as data synchronization (12%), B2B data exchange (10%), database migrations (4%), and database replication (1%). All these are clustered near the bottom of Figure 9, which denotes their low priority for UDM. This is probably due to how most database administration and maintenance work is done in isolation of other data management practices, although it regularly involves collaboration with application developers.

Database administration and unstructured data

are low priorities for UDM.

UDM depends on its constituent tools and

enterprise platforms for shared infrastructure.

www.tdwi.org 21


UDM has little room for unstructured data.At the very bottom of Figure 9 we find enterprise search (4%) and text analytics (3%). Both have a strong foothold in BI and DW, but are rarely associated with other data management practices. Most organizations still consider them (along with content management and document management systems) as part of the world of unstructured data, which has very few intersections with the structured data that dominates most data management practices today.

A General Framework for UDMIn the preceding discussion, we established that UDM includes three distinct groups of technical components: core practices and their tools, supporting practices and their tools, and infrastructure for unified designs and interoperable servers. Put all three together, and the result is a general framework for UDM, as illustrated in Figure 11.

Realize that your UDM framework may vary (as compared to Figure 11), based on your organization’s needs or maturity. For example, if enterprise data architecture is not yet a priority, it may not be one of the core disciplines. Many organizations expend so much effort on metadata management that it’s more of a core discipline than a supporting one. Search, text analytics, or content management may be important supporting or even core disciplines in organizations that handle valuable textual information. Likewise, data security and privacy are paramount for some businesses, and are therefore parts of the framework. If UDM has an exclusive focus on operational systems, it may exclude BI and DW. In short, the actual content of the UDM framework can be unique for every organization.

Regardless of the practices and infrastructure included in your UDM program, the UDM framework shown in Figure 11 includes most of the components most organizations would coordinate via UDM, arranged in a typical hierarchy. Use this framework to guide your planning and predict your future.

Core and supporting practices, plus shared infrastructure, constitute a UDM framework.

Your UDM framework will vary, but this one can still help you plan.

Unified Data Management

Figure 11. One possible form of TDWI’s UDM framework.

Core Practicesfor UDM

BIand DW

DataQuality

MasterData

Management

DataGovernance

DataIntegration

EnterpriseData

Architecture

SupportingPracticesfor UDM

MetadataManagement

DataStewardship

DataModeling

DataPro�ling

DataFederation

DataGlossaries,

Etc.

Infrastructurefor UDM

• Interoperability among multiple data management design tools and servers• Connectivity with enterprise operational and analytic applications• Use of interfaces and buses for data integration, application integration, SOA, etc.

22 TDWI RESE ARCH


USER STORY UNIFYING DATA CAN BE AS IMPORTANT AS UNIFYING TEAMS AND TOOLS.

“We met many of the goals of enterprise data management through two forms of unification,” said Darren

Taylor, the VP of integrated business systems at Blue Cross and Blue Shield of Kansas City. “First, post

Y2K, we felt encumbered by the distribution of data across many systems. So, starting around 2004, we

began an initiative to consolidate and integrate data from the dozens of disparate systems and sources

into an enterprise data warehouse. The solution is actually a hybrid cross between a data warehouse and

an operational database. Having most of the data in one place and in a fully integrated structure enables

us to perform operational BI and provide many low-latency 360-degree views for both BI and operations.

“Second, we unified data management teams and solutions through a competency center. It includes data

management professionals for data integration, quality, metadata, modeling, and architecture. It also

includes a group of business analysts who ensure that most data management work supports real-world

business needs. Between the enterprise data warehouse and the competency center, our data and its

management are unified in a way that supports the fast pace and visibility into fresh information that our

business requires.”

Vendor Platforms for UDM

Defining UDM Suites and PlatformsBy now, you’ve probably noticed that UDM unifies many things. It unifies the technical efforts of diverse data management teams, it unifies the goals of technology and business groups, and it unifies data itself (sometimes metadata and master data, too). Now we need to examine yet another level of unification, namely the unification of software tools and platforms for data management.

An organization of any size or sophistication will use multiple tool types for data management, simply because there are multiple types of data management tasks, such as BI, data quality, data integration, MDM, and the other practices discussed in this report. Furthermore, the tools employed by users may be from several vendors, as well as hand-coded or home-grown. All this diversity can be coordinated at an organizational or team level. But a large or mature UDM program will also need unification at the tool level, which requires that data management tools integrate and interoperate at appropriate points.

Software vendors that produce data management tools have noted users’ need to integrate tools and are supplying the demand. For example, a number of vendors have collected numerous tools that enable data management practices. The vendor may build or acquire such tools. Either way, the product portfolios of several vendors have grown, as they fill up with more tools and functions that enable diverse data management tasks. The firms that sponsored this report are all good examples of software vendors that are building out their product portfolios of data management tools. The sponsors are ASG, DataFlux, Informatica, SAP, Talend, Teradata, and Trillium Software.

The good news is that leading vendors now have more tools to offer to user organizations. The slightly bad news is that integration and interoperability are still somewhat rudimentary among the tools of some vendor platforms, and that’s an impediment to unifying data management.

Several vendors have assembled suites of

diverse tools.

UDM unifies tools, too, not just user practices

and business goals.

www.tdwi.org 23


Another way to sum up the situation is that several vendors have assembled suites of tools, and the suites are impressive in the number and diversity of data management tools (including tools for most of the core and supporting practices previously discussed). But a suite differs from a platform, in that a platform assumes rich integration and interoperability among its tools. Data management tool vendors have worked hard in recent years to improve the situation by increasing integration and interoperability among tools, though usually just within a single vendor’s portfolio of tools (rarely across tools of competing vendors). In other words, these vendors are actively extending their product portfolios into data management suites, plus evolving the suites into integrated platforms. This is good news, because it makes the vendors’ product portfolios much more conducive to UDM.

A Framework for UDM PlatformsTo understand what vendors offer today (and will probably offer in the near future), we can employ TDWI’s UDM framework. As illustrated in Figure 11, the framework organizes user-oriented practices in data management into core and supporting levels. Since most of these practices are available as tool types from vendors, we can use the same framework to describe vendor offerings.

• Tool types for UDM core practices. As a reminder, the six core practices of UDM (as seen in the UDM framework of Figure 11) are BI/DW (the two considered as one practice), data quality (DQ), master data management (MDM), data governance (DG), data integration (DI), and enterprise data architecture (EDA). Most of these user practices are enabled by tools from multiple vendors. An exception is EDA; TDWI is unaware of any tool devoted to this practice. And DG is a bit of an exception. A few DG tools exist, such as the Governance, Risk, and Compliance (GRC) application from SAP. And some vendors with DQ or DI tools (such as DataFlux, Informatica, and Trillium) are building out their stewardship functions to evolve them into DG tools.

• Tool types for UDM supporting practices. Common supporting practices include metadata management, data stewardship, data modeling, data profiling, data federation, and data glossaries. Some of these are enabled by independent tools (especially data modeling), but all are commonly seen as functions built into larger tools and platforms. For example, a few stand-alone data federation tools are still on the market, although most users turn to the federation capabilities built into their BI/DW platforms or DI tools. Likewise, data profiling tools are available separately, but most users rely on the profiling functions built into their DQ, DI, and MDM tools.

Tool types vary among UDM suites. Ideally, a vendor’s UDM suite or platform will support many of the core and supporting UDM practices via dedicated tools or embedded functionality. As you can imagine, the list of supported practices varies quite a bit among vendor product portfolios. Some vendors offer two or three tools, whereas other vendors are obviously aiming for a comprehensive platform of many tools.

A UDM suite needn’t support all core and supporting practices. It’s important to set realistic expectations. We shouldn’t expect any vendor to support every imaginable tool type in data management. In fact, many vendors are quite successful by sticking with a short list of tool types based on their core competency, customer needs, and market opportunities.

Vendors are now evolving their suites into integrated platforms that are conducive to UDM.

The core and supporting practices of the UDM framework are all enabled by tool types of the same name.

24 TDWI RESE ARCH


Examples of UDM PlatformsSince the firms that sponsored this report are all good examples of software vendors that offer a UDM platform, let’s take a brief look at each. The sponsors form a representative sample of the vendor community. Yet their UDM platforms are quite diverse, illustrating multiple approaches to UDM3.

ASG ASG, Inc. offers ASG-Rochade (among other products), an independent metadata repository that’s tool-agnostic and suited for broad enterprise use, managing metadata for many IT systems from a shared, central source. Leveraging metadata management as a core competency, ASG is producing a suite of data management tools and solutions, each built atop ASG-Rochade as a common platform. This makes sense, because metadata management is by far the most common supporting practice for UDM. And two of the tools released from this product line enable popular UDM practices, namely ASG-MDM (master data management), ASG-Data Governance, ASG-Model Management, and ASG-metaGlossary (a data glossary).

DataFlux DataFlux has shown a deep commitment to UDM, which they call enterprise data management. For years, DataFlux has offered a mature DQ suite, and more recently they’ve built out many of its stewardship functions to evolve them toward data governance. DataFlux is a subsidiary of SAS, and a couple of years ago the two executed a reorganization that moved SAS’s DI products to DataFlux. This has helped them deepen the integration between DQ and DI tools. These tools, of course, also have tight integration with SAS’s DW, BI, and analytic tools. Together they form a rich set of data management tools.

Informatica Informatica’s tagline reads “the data integration company,” and their definition of DI is very broad, encompassing multiple forms of DI, DQ, MDM, profiling, stewardship, data services, changed data capture, unstructured data processing, B2B data exchange, cloud data integration, information lifecycle management, complex event processing, and so on. In effect, DI is Informatica’s umbrella term for UDM, and their DI platform addresses both supporting and core practices for UDM, all built into a common platform. Furthermore, Informatica was instrumental in defining the competency center, which has become a common organizational structure for unifying data management practices, especially DI, DQ, and BI.

SAP The product portfolio of data management tools from SAP is fairly comprehensive. With the combined solutions that came from the Business Objects acquisition, SAP offers BI, DW, MDM, DI, DQ, DG, profiling, process integration, data marts, metadata management, text analytics, information lifecycle management, and content management. SAP (and Business Objects before them) has stated a deep commitment to UDM—which they prefer to call enterprise information management. And they’ve taken action on the commitment by deepening the integration and interoperability of products in the newly expanded portfolio and beyond to deliver open data management solutions that can support any heterogeneous data environment.

3. The vendors and products mentioned here are representative, and the list is not intended to be comprehensive.

www.tdwi.org 25


Talend Talend has a platform that follows a common lifecycle progression seen with UDM. Many user organizations start with a DI tool (whether for BI/DW or operational use), then complement it with a DQ tool to fix the data problems that DI typically exposes. MDM is a common third tool, and data profiling is required for DI, DQ, and MDM. Talend offers a tool for each of these four UDM practices, thereby satisfying one of the most common combinations of tool types. Furthermore, all four tools are built atop a shared platform, with all development GUIs integrated into Eclipse, a unified metadata repository, and only one server to deploy. Based on open source, the entire package comes at a modest price.

Teradata Teradata is famous for its world-class data warehousing Teradata Database software and the hardware platforms and appliances with which it’s packaged. But don’t forget that Teradata also offers several data management tools, applications, and services. For example, Teradata offers master data management software with optional accelerators for product and customer data. Other examples include Teradata Metadata Services, Teradata Data Quality Rules Manager, and Teradata Warehouse Miner (which includes Teradata Profiler). These products run on Teradata Warehouse, fully leveraging its speed and scalability. And Teradata Professional Services has long experience helping organizations with data management, including tools from many vendors.

Trillium Software The Trillium Software System supports the entire DQ lifecycle for any type of data, as well as DG and MDM functions. The main tools are TS Quality (for DQ), TS Discover (data profiling), TS Insight (a data monitoring dashboard), and TS Director (connectivity to leading packaged applications and integration buses). MDM functions and location geocoding are built into Trillium’s DQ suite, as are interfaces that integrate Trillium with leading MDM solutions from other vendors. This is a true platform in that all Trillium tools share rules, metadata, profiles, development artifacts, services, and interfaces.

USER STORY A USER’S DREAM: ONE TOOL FOR DESIGNING AND DEPLOYING ONE SOLUTION.

“Ideally here’s what I need, although I know it’s not available yet,” a senior data integration specialist told

TDWI. “Imagine one data flow, the kind of data flow we’ve all learned to design using our data integration

and data quality tools. But the data flow isn’t exclusively about data integration or data quality. The flow

has steps for those, plus lots of other data management tasks, especially MDM, data federation, changed

data capture, data warehouse dimensional loads, service bus integration, and all the other stuff I have

to design and deploy. Furthermore, imagine that the design tool where I build the data flow has—in one

developer GUI—the support functions I need to build this ‘mega data flow,’ like shared metadata, data

profiles, data models, a data glossary, and functions for governance and stewardship. This one tool

for designing and deploying one solution is what I need to survive the confluence of data management

techniques. And it would give my company the simple view into data management that it needs for data

governance, compliance, and business integration.”

26 TDWI RESE ARCH


The Future of UDM Best Practices and Tool TypesThere will be more data management tools. Vendors will continue to build and acquire more tools of the types listed in the core and supporting layers of the UDM framework. It’s important to note that not all vendors currently support all the best practices and tool types listed here as required or desirable for a UDM suite or platform. Therefore, there’s plenty of room for vendors to expand their product portfolios in response to competitive pressure, customer needs, and their own creativity. Hence, you should think of every UDM platform as a work in progress.

Data management tools will integrate and interoperate better. After all, from a tool standpoint, this is what UDM is all about. Don’t forget that this involves two different tool areas: integration of GUIs for development or administration and interoperability among servers and other enterprise platforms.

Younger data management best practices and tool types will mature. We’ve barely seen the first generation of DG tools and practices. As this best practice evolves, so will its tools. Likewise, MDM is a bit further down the road to maturity, but MDM practices and tools will also improve dramatically.

There will also be more practices and tool types coordinated via UDM. In other words, completely new tool types will arise to enable new practices. For example, the modern best practice of enterprise data architecture has grown tremendously in the last five years. Yet EDA practitioners do it with a handful of supporting tools (say, for data modeling, profiling, metadata management, etc.). Expect the vendor community to soon meet the demand for dedicated EDA tools.

Users’ definition of data management will expand, thus expanding the scope of UDM. And this will bring more tool types to data management. For example, most of us stubbornly manage data as a static entity, while ignoring the fact that much data exists in a continually moving process. To correct this omission, some users already tap enterprise infrastructure for workf low or process management and integration. Eventually, these techniques will be a more prominent part of data management.

Similarly, many user organizations are slowly realizing that unstructured and semi-structured data should not be segregated from structured data, since all these data types describe the same business entities. Expect text analytics, text mining, and various forms of search (and possibly content and document management) to have more intersections with the structured data that dominates data management today.

Finally, UDM needs—and will eventually pull into its coordination—many other tools and practices that already exist, such as those for portfolio management, information lifecycle management, and database administration.

Vendors will offer more tools conducive to UDM,

and the tools will have tighter integration and

interoperability.

Existing UDM practices and tools will mature,

and new ones will join them.

The number of practices and tools coordinated via UDM will continue

to grow.

www.tdwi.org 27

Recommendations

RecommendationsAlways remember UDM’s two complementary goals. You need to achieve balanced success as you coordinate diverse data management disciplines and support strategic business objectives.

Keep UDM focused on best practices from a technical user’s viewpoint. This is where most of UDM’s coordination takes place, although it also concerns data-driven corporate objectives and the concerted use of data management tools.

Practice UDM at an appropriate level. Never consider coordinating all your data management work via UDM. Instead, opportunistically select combinations of practices that will give the organization a noticeable improvement. The most common combinations are pairs, as with data integration and data quality or DG and MDM. Expect the level of UDM to increase as you coordinate more data management efforts.

Prepare to deepen your UDM efforts. TDWI survey data shows that UDM usage will increase significantly over the next three years. Your peers in other user organizations are doing it, so you should, too.

Adopt UDM for the business benefits. These include better decisions and strategies, leverage of data assets, business-to-IT alignment per data, and data-driven corporate objectives.

Improve your data through UDM. Technology benefits of UDM include improvements in data quality, consistent data definitions, master data and its management, metadata and its management, and data standards.

Give UDM proper leadership. UDM needs strong business sponsorship and clear business goals. Else it may succumb to barriers due to organizational silos, data ownership, and other politics.

Foster cross-functional activities. On the technical side, coordinate multiple data management teams. On the business side, multiple business units need to coordinate their data-driven, cross-unit business processes. Bring these two together to enable business-to-IT alignment relative to data, which is also cross-functional.

Enlist DG to manage the many changes UDM will invoke. A good DG program will have processes and procedures for proposing, approving, and policing changes to information systems. Use DG to push through UDM’s changes in a public and consensus-driven way.

Execute data management solutions mostly with core practices. After all, these enable the bulk of any data management solution, and so are highly visible and usually considered your metric for success.

Don’t underestimate the importance of supporting practices and shared infrastructure. Without them UDM isn’t possible. Increasing your reliance on support practices will probably increase the success of your core practices.

Look for vendor tools that are conducive to UDM. These are data management tools of various types that integrate at the appropriate points with other tools, in both development and deployment environments.

Consider vendor platforms for UDM. A single-vendor approach simplifies the coordination of multiple tools, although a best-of-breed approach to UDM can also be successful.

Balance UDM’s complementary goals and stay focused on user best practices.

Start by unifying pairs of practices, and expand into more practices over time.

Do UDM for the improvements to the business and its data.

Succeed with leadership and cross-functional consensus.

Know the roles of core and support practices, plus shared infrastructure.

Use tools that support the integration and interoperability that UDM demands.

Research Sponsors

ASGwww.asg.comASG is the industry-leading provider of enterprise metadata management solutions. ASG-Rochade® is a powerful metadata repository that manages information about data and systems across the enterprise, uncovering complex relationships between information technology assets, facilitating what-if analysis, and increasing the productivity of business and IT. ASG-metaGlossary™ is a business glossary solution that provides a governed and contextually rationalized vocabulary between business and IT to help proactively address data governance and compliance mandates. Founded in 1986, ASG provides world-class solutions to more than 85 percent of the world’s largest companies, enabling them to reduce costs, improve business-service delivery, and reduce risks.

DataFluxwww.dataflux.comDataFlux enables business agility and IT efficiency by providing innovative data management technology and services that transform data into a strategic asset. A wholly owned subsidiary of SAS (www.sas.com), DataFlux helps organizations manage critical aspects of data through unified technologies and expertise that provide the benefits of data quality, data integration, and master data management.

Teradata Corporation www.teradata.comTeradata is the acknowledged global leader in data warehouse innovation and analytical solution development. Every day we raise our customers’ intelligence to higher levels, making them more focused and competitive by gathering enterprise information and extracting actionable insight.

Teradata elevates enterprise intelligence by giving every decision maker the insight required for smarter, faster decisions. We add value and reveal opportunity across more dimensions than any competing solution.

In every industry and geography, our technologies and expertise make the difference. Simply put, Teradata solutions make companies smarter and give them the competitive advantage to win. Informatica

www.informatica.comInformatica is the world’s number one independent provider of data integration software. Thousands of organizations rely on the Informatica Platform to obtain timely and trusted data for their most important business initiatives. With Informatica, enterprises gain a competitive advantage from all their information assets to grow revenues, increase profitability, further regulatory compliance, and foster customer loyalty. Informatica delivers a comprehensive, unified, open, and economical platform to reduce IT costs and expedite the time to address data integration needs of any complexity and scale.

SAPwww.sap.comSAP is the world’s leading provider of business software. Today, more than 92,000 customers in more than 120 countries run SAP® applications—from the needs of small businesses and midsize companies, to global organizations. The SAP business software portfolio includes solutions for enterprise resource planning, business intelligence, enterprise information management, and related applications. SAP solution portfolios support the unique business processes of more than 25 industries.

Talend www.talend.comTalend is the recognized market leader in open source data management. After three years of intense research and development investment, and with solid financial backing from leading investment firms, Talend revolutionized the world of data integration when it released the first version of Talend Open Studio in 2006.

Talend’s solution portfolio includes data integration (operational data integration and ETL for business intelligence), data quality, and master data management (MDM) and covers all facets of data management.

Unlike proprietary, closed solutions, which can only be afforded by the largest and wealthiest organizations, Talend makes data integration solutions available to organizations of all sizes, and for all integration needs.

Trillium Software www.trilliumsoftware.comEnterprise data quality ensures that accurate, consistent, and complete information is always available to drive higher productivity, decrease costs, increase revenues, and improve ROI. Trillium Software lets businesses implement a disciplined approach to Total Data Quality across the enterprise and eradicate data quality defects.

Harte-Hanks Trillium Software, the global leader in Enterprise Data Quality, provides technologies and services that deliver global data profiling, cleansing, enhancement, linking, and governance for CRM, ERP, supply chain management, data warehouse, e-business, and other enterprise applications.

Trillium Software technology and solutions help global companies manage growth and change. We serve major global leaders in many industries including financial services, pharmaceutical, retail, manufacturing, high technology, hospitality, and government.

1201MonsterRoadSW

Suite250

Renton,WA98057

T 425.277.9126

F 425.687.2842

E [email protected]

www.tdwi.org

TDWI Research provides research and advice for BI professionals worldwide. TDWI Research focuses exclusively on BI/DW issues and teams up with industry practitioners to deliver both broad and deep understanding of the business and technical issues surrounding the deployment of business intelligence and data warehousing solutions. TDWI Research offers reports, commentary, and inquiry services via a worldwide Membership program and provides custom research, benchmarking, and strategic planning services to user and vendor organizations.

TDWI RESE A RCH

a collaboration of data disciplines and business...

Documents