When Data Governance Turns Bureaucratic: How Data Governance Police Can Constrain the Value of Your Multidomain Master Data Management Initiative

Dan Power

Over the past several years, we’ve seen rapid adoption of multidomain master data management (MDM) technology in a variety of industries, especially pharmaceuticals, biotechnology, financial services, insurance, high tech, and medical device manufacturing.

Companies continue to seek new ways to increase revenues, cut costs, and improve regulatory compliance. Leveraging a multidomain MDM system to create a “single source of truth,” with consolidated, clean and consistent business data such as customer, product, supplier, and employee—also known as master data—has provided companies with a sustainable competitive advantage and a dramatic return on investment. According to Gartner’s 2009 “Magic Quadrant for Master Data Management of Customer Data,” the market for MDM-related software and services continues to grow at double-digit rates, even in generally uncertain economic times.

But for those companies that haven’t yet implemented multidomain MDM, a number of business initiatives may be hamstrung because critical master data remains scattered over a large number of different applications, systems, and databases. This situation, with “islands of information” containing inconsistent and duplicate master data, leads to key processes in the enterprise being based on “dirty data.” This makes it difficult to process orders correctly, get invoices paid promptly, introduce new products on time, keep customer satisfaction and retention levels high, and improve overall levels of agility and innovation. In turn, companies with dirty data experience higher costs and reduced productivity, with most of the impact hidden within an organization’s fixed costs.

Multidomain master data management effectively solves these problems. It combines a multidomain MDM system with integration, data quality, and enrichment capabilities, supported by a new data governance organization and new processes for managing master data. Bringing the critical master data into the MDM system, cleansing it centrally, and then sharing it back out to the rest of the enterprise, results in master data that is accurate, complete, timely, and consistent. Clean master data leads to improvements in the organization’s marketing, sales, finance, customer service, and other core operational areas.

white paper

Sponsored by

Over time, solving the common problems associated with fragmented master data gives the enterprise an opportunity to progress to higher levels of MDM and data governance maturity, and to attain additional benefits such as increased revenue, reduced costs, and improved compliance.

This evolutionary progression, from using fragmented data to using consolidated master data, from having little or no governance organization and processes to enterprise-wide orchestrated data governance, and from trying to operate with master data chaos to master data order, takes time. Companies progress through several common developmental stages, such as the transition from reactive to proactive data governance.

Reactive Data Governance DefinedMost companies implementing multidomain master data management systems have initially used a “reactive” style of data governance. Popular “front office” applications, such as customer relationship management (CRM) and “back office” applications, such as enterprise resource planning (ERP) are used to authoring the master data, such as customers, products, suppliers, employees, etc.

Data movement tools then move the new or updated master data into the multidomain MDM system. It cleanses, matches, and merges the data to create or update the “golden record,” and then synchronizes that back to the original system, other enterprise applications, and perhaps a data warehouse or business intelligence / analytics system.

As many as 80% - 90% of companies implementing multidomain MDM technology start with this “coexistence” architecture. But because the CRM and ERP applications remain the “Systems of Entry”, while the MDM system’s role is limited to being the “System of Record”, some issues persist, and some of the biggest promises of master data management remain unrealized.

[ 2 ]

Create Master Data Consume Master Data

Source Systems Target SystemsData Steward

MDMBusiness User

CRM

Business User

CRM

Business User

ERP

Business User

ERP

Jon Q Jones

J QuincyJones

ExceptionException

ExceptionException

Exception

JonathanQuincy Jones

• Business users create unreliable data in multiple systems, which is inconsistent and duplicate• Few data stewards flooded with volumes of unreliable data to fix• Delays the use of reliable data across the enterprise

Reactive Data Governance Falls Short on Multidomain MDM PromiseIn the above scenario, business users continue to use and rely on their Systems of Entry (and live with its shortcomings), and the enterprise typically creates a small group of data stewards who work primarily with the System of Record.

The CRM and ERP applications generally used as the Systems of Entry usually don’t enforce all of the attributes required by the MDM system or System of Record. So the data stewards often are forced to perform time-consuming research in order to populate important fields in the master record that are often left blank upon initial creation in the Systems of Entry.

More importantly, Systems of Entry typically don’t have the sophisticated searching and matching capabilities needed to rigorously prevent entry of duplicate records. Duplicates can be recognized and merged after the fact in the multidomain MDM system as the System of Record, but it’s better to fix problems or prevent them at the source.

A good rule of thumb is that it’s much more expensive to fix data quality issues after the fact than to fix them at the source or prevent them from happening in the first place.

So when business users create a duplicate new record, the duplication becomes an issue for the MDM system and the data stewards to correct and manage. This factor and the research required to populate important but empty fields can cause a lot of time lag and back-and-forth between the business users and the data stewards, and make it difficult to achieve Service Level Agreements (SLAs), data quality targets, efficiency goals, and overall return on investment.

In some cases, because people may perceive that the data stewards will take care of any issues, business users may become even less concerned with accurate, consistent data entry, paradoxically causing the reactive data governance strategy to contribute to lower levels of data integrity.

Reactive Data Governance Impinge On Business PerformanceThe reactive style of data governance, which usually involves using batch-oriented integration to synchronize master data from the MDM system to traditional “front office” CRM applications and “back office” ERP applications, can impact several areas of the business.

The time lag introduced by the batch integration and reactive data governance approach may lead to scenarios where the business continues to operate on duplicated, incomplete and inaccurate master data. This in turn reduces the ability of the multidomain MDM initiative to achieve its expected business goals of delivering the right data to the right people at the right time. After the expectation has been set that data will be clean, accurate, and timely, time lags introduced by batch integration can be frustrating.

[ 3 ]

The reactive style of data governance, with a downstream group of data stewards being responsible for cleansing, de-duplicating, correcting and completing critical master data, may lead to a perception of “data governance as bureaucracy.”

Reactive data governance can also lead to end users perceiving the data stewardship team as the “data quality police,” with a corresponding negative perception of bureaucracy and delay, and of master data that is still dirty. This will also make it harder for the MDM initiative to achieve all of its expected benefits, and may lead to higher overall data management costs.

The risk with this approach is that the organization may end up with, at least in part, the “worst of both worlds” – having invested in a MDM initiative but only realizing some of the potential benefits of having clean, accurate, timely and consistent master data available throughout the enterprise.

So what are the options?

Options to Reactive Data GovernanceThe reactive style of data governance is not inevitable. In early implementations, the tools used with a multidomain MDM system were not quite ready for prime time. They were aimed at data stewards. They weren’t business or end user-focused.

Because of these limitations, organizations were forced into one of two choices: continue the status quo, or develop custom “front ends” which were user-friendly enough for the average end-user to create and update master data.

The burden of creating custom front ends, however, can be considerable. In addition to the other aspects of defining, designing, developing and deploying a MDM system, a significant additional task, that of creating the custom front end, has to be added.

This can increase the initial cost of multidomain master data management substantially. And more importantly, according to “Software Maintenance” by Gerardo Canfora and Aniello Cimitile, “software maintenance consumes 60% to 80% of the total life cycle costs”—so for every $1 spent on initial development, roughly $4 in maintenance costs are incurred over the lifetime of this type of application.

[ 4 ]

There are three approaches to moving beyond the reactive style of data governance.

1. Users Enter Data Directly into the Multidomain MDM System: Users enter data directly into the MDM system using a user-friendly front end, but their new records and updates to existing records are held in a staging or holding area until reviewed and certified by a data steward. Only then are the insertions or updates accepted into the MDM system for the full cleanse, match, merge and publishing of the “best record” out to all of the other enterprise applications. This method is better than having a completely different application, like a CRM or ERP system, serve as the Systems of Entry, but it can still introduce lags and inefficiencies. Despite these shortcomings, using a staging area does resolve most problems of not enforcing entry of important attributes or not having a thorough search-before-create. Additionally, since we’re not at the mercy of how a legacy application or a modern CRM or ERP application handles the data entry function, we have also shortened the timeline considerably by not having the batch-oriented data movement of the more reactive approach.

2. Users Enter Data that is Routed Directly to the Multidomain MDM System: The new records or updates are entered outside, but are routed immediately to the MDM system for automated cleansing, matching, and merging. Anomalies or exceptions are routed to data stewards’ queues, enabling fewer stewards to support more end users. This is an improvement on the first proactive approach since we’re leveraging the MDM system’s business rules, data cleansing and matching capabilities, and only requiring a steward to look at insertions or updates that pop out as an exception to the cleanse, match and merge process.

3. Users Enter Data Using a Data Governance-Specific Front-End: The third approach is to allow direct entry into the multidomain MDM system by end users, but using a front end specifically designed for the proactive style of data governance. Screens can be set up specifically for end user data entry, and you can take advantage of all of the automation, data cleansing, business rules, searching and matching that a full-featured MDM system allows. So it’s not necessary to enter data into a staging area in the MDM system first, and you don’t need a separate workflow application outside of the system.

The latest and greatest Systems of Entry for master data, coupled with a robust MDM system, which includes strong searching, matching, data cleansing and business rule enforcement capabilities, makes it safe for the average end user to enter new or updated records directly into the MDM system. An effective Systems of Entry also allows data stewards to be more productive—they can spend their time researching and correcting legitimate issues since they don’t have to manually examine every new or updated master record.

[ 5 ]

Proactive Data Governance DefinedSo how do we move towards a more proactive architecture and data governance style? The first requirement is that we start authoring data directly in the multidomain MDM system, decoupling data entry from the traditional CRM and ERP systems. When the System of Entry and the System of Record are one and the same, the application architecture is simplified. The CRM and ERP systems become consumers of master data only—they no longer originate it.

But in order to achieve this valuable simplification, a flexible, user-friendly interface is needed. It’s helpful to be able to create different versions of the user interface targeted at different groups of business users (from casual to expert), while still having a complete data stewardship console where data stewards can work on issues requiring human judgment, while tracking data quality metrics and resolving anomalies.

The role of the multidomain MDM system itself changes, from being a passive recipient and cleanser of data entered or updated elsewhere, into being both the original System of Entry and the System of Record. Once new or modified records pass the internal data governance rules, the MDM system publishes those certified records via real-time or near real-time middleware to the CRM and ERP systems, as well as any data warehouses or analytic systems. Where a real-time or near real-time feed is not needed, new and changed records can be queued up for synchronization to other enterprise systems via a batch integration.

This shift eliminates a major source of complexity as well. Instead of having the MDM system in the center of a complex web of source systems on the left and consuming systems on the right, the MDM system becomes the source system, and the other applications and databases in the enterprise become consuming systems. So, roughly half of the system integration workload goes away, and the job of mapping source systems and their individual, idiosyncratic ways of allowing data entry back to the MDM system goes away as well.

It seems like a radical step, but it’s really a continuation of a longstanding trend. When enterprise application suites first became popular, companies assumed their new CRM or ERP system would be the single source of truth. But over time, companies fell victim to proliferating systems and databases. So no one front office or back office system had the entire set of master data.

So if you’re going to add a multidomain MDM system and acknowledge that CRM and ERP systems are not designed to manage master data, why not take the next step and remove their ability to create, update or delete master data, instead allowing those systems to read and transact with master data only?

[ 6 ]

[ 7 ]

How Proactive Data Governance HelpsThe proactive model places more emphasis on business users being the owners of the master data, not just entering it into the System of Entry. Rather than just key in a few critical fields and letting the data stewards worry about the central issues of accuracy, completeness, timeliness, and consistency, the accountability for these goals shifts to the business users, who need the data to be accurate, complete, timely, and consistent for their day-to-day operations.

Having more accountability in the business for the quality of master data has a big impact on the consumption of master data. The end-to-end process for creating a new customer or product, and modifying it when needed, is more visible and straightforward, so consumers of master data are more likely to trust it. Rather than being a black box, where newly created records are entered in an incomplete, dirty or duplicated state but “magically” come out of the MDM system complete, clean and de-duplicated, people using the data will have better information on the audit history and lineage of the data, which will drive up the business users’ trust levels in the master data.

Also, the number of data stewards (and their role) will change. Since the end users will be empowered to enter new records directly in the MDM system, there should be less need for data stewards to act as the “data quality police.” The MDM system itself will enforce the important business rules on completeness, accuracy and consistency.

Once users are no longer dependent on the CRM and ERP systems (with their quirks) to perform the initial entry and updating of master data, the data stewards can focus on managing exceptions and monitoring important metrics used for measuring the master data’s quality, availability, security, and usefulness.

Create Master Data

Consume Master Data

Source System

Target Systems

Data Steward

MDMBusiness User

BusinessUser

BusinessUser

CRM

Business User

ERP

JonathanQuincy Jones

JonathanQuincy Jones

• Business users proactively create reliable data in one system, which is consistent and unique

• Minimizes data steward workload

• Reliable data is immediately available for use across the enterprise

Business Benefits of Proactive Data GovernanceThe business benefits of multidomain MDM derive from having the best, cleanest source of data, exposed to the widest group of people in the enterprise, while still having data quality tools and stringent rules in place to avoid entry of inaccurate, incomplete, or inconsistent data.

The first benefit of proactive data governance is getting the master data right at the source. With a rigorous “search before create” function in place and robust business rules for ensuring critical fields are either populated from approved lists of values or validated against third party data, the initial level of quality for new records will be much higher.

Master data management efforts are usually focused on either the “get it clean” or “keep it clean” aspects of data quality.

The “keep it clean” side of master data management is much easier if the initial level of data quality in the MDM system is very high, and if you’re not continually polluting the system by bringing in inaccurate, incomplete or inconsistent data from the CRM or ERP source systems.

The proactive style of data governance also effectively eliminates any time lags between the initial entry of a new master record and its certification and publishing via middleware to the rest of the enterprise.

Proactive data governance, supported by a user-friendly front-end, enables data entry directly into the multidomain MDM system, which applies all of the typical business rules for cleansing, matching and merging data. This approach also allows data stewards to publish updates via an enterprise bus to the rest of the organization as soon as the initial data entry goes through the cleanse, match, and merge process.

The proactive data governance approach eliminates the perception of “data governance as bureaucracy,” since the authoring of master data has been pushed upstream to the business users, leaving the data stewards in a less intrusive role where they won’t be a bottle-neck to critical business processes such as order management or invoicing.

Sales and marketing benefit because marketing campaigns can be done more quickly and cost-effectively, with no upfront data remediation needed prior to launching a campaign. Finance benefits, since all of the data elements needed for a new customer will be captured at once, and the process for adding a new customer can include pulling third-party content and calculating a credit limit, then passing that information back to the ERP system.

Customer service reps without direct access to a MDM system typically have to search several systems to find the information they need to take action. This makes it tough to provide a high level of service when an impatient customer is on the phone. When all information is stored in the MDM system and accessible through an efficient, user-friendly front end, customer service reps will be able to access all of the data needed for each customer interaction, as well as being able to author new data when necessary.

[ 8 ]

By making MDM the System of Entry as well as the System of Record, you’re essentially maintaining the data in a “zero latency” state, where it is ready for any anticipated usage scenario in the enterprise, and the data synchronized to the CRM and ERP systems should be at the highest levels of cleanliness, accuracy, timeliness, and consistency.

Proactive Data Governance—Lessons Learned from LeadersOrganizations that have evolved into the proactive style of data governance have reported some common lessons around relationship management, history, workflow, and security.

Relationship Management

MDM should be the system of record for not only master data, but also the relationships among the master data. It becomes the central place to gain a 360-degree view of how the data from different systems relate to each other. For example, the multidomain MDM system relates the sales order from an order management system to the invoice in accounts receivable. These relationships or hierarchies are visually displayed within the user interface that interacts directly with the MDM system data. The user interface can also be used for viewing the relationships among master data and editing them directly in the MDM system Thus, MDM becomes the System of Entry for relationships as well.

History

When you’re accepting a new record or an updated record from an outside system like a CRM system, you may be limited in tracking the history of that record by what is allowed by the outside application. When MDM is both the System of Entry and the System of Record, sophisticated tracking of the audit history and lineage of the data is possible. It’s even possible to display the changes to the core master record over time, showing the insertions and updates by various users and processes in a dynamic timeline view, allowing every change in every attribute to be tracked and displayed.

Workflow

A configurable front-end enables you to design and implement basic workflow functions, so end users can enter new master records. But, those new records may require an approval step by a data steward before they are fully accepted into the multidomain MDM system and published out to the rest of the enterprise.

Another application of workflow is in queuing of tasks for data stewards. Exceptions to matching or automatic merging of duplicate records are routed to the appropriate data steward.

Advanced features allow for escalation of issues to the appropriate person and automatic rerouting to a backup person when a user is on vacation. This eliminates the time spent on inquiries about the status of new or changed records by providing direct visibility into specific workflow steps, and elapsed time for these processes.

[ 9 ]

[ 10 ]

Security

The user interface needs to be configurable, with different job roles having different levels of access and permissions. Some data elements that can help a data steward resolve a discrepancy may not be appropriate for everyone in the enterprise to see. Furthermore, even within a job role like data steward, you might want different levels of security, with the more senior people able to perform more actions on a broader set of records. Also, you may need to segregate access, so data stewards in Germany are not allowed to see French customer records, for example.

When using a CRM or ERP system external to MDM as the System of Entry, the security model of that application imposes limits in terms of who has clearance to perform which operations to which records. Once you move the entry and maintenance of master records directly into the multidomain MDM system, you have far more fine-grained control of the security of that data, down to the individual attribute or field level.

ConclusionMoving from reactive to proactive data governance simplifies the enterprise architecture, provides for faster throughput in certifying new or updated master records and makes it easier overall to publish them to the organization. A primary benefit is that the initial level of quality for new records is much higher. Proactive data governance also pushes the authoring of master data back upstream to the business users, which both increases accuracy and lowers costs while producing data at the highest levels of cleanliness, accuracy, timeliness and consistency.

Where Proactive Data Governance Works Best

What holds companies back from the proactive data governance approach? It’s mainly a question of where they are on the data governance maturity scale. It’s difficult for a company to jump directly from the far left side of the maturity model—where they have no central multidomain MDM system and no data governance organization or processes —to the far right side of the scale, where they have a robust data governance process plus a modern MDM system and integration architecture.

Typically organizations evolve their data governance approach over time. For example, once the initial MDM system is up and running, and some of the expected benefits take longer to materialize, or limitations of the reactive approach become obvious, you can plan to take down the ability to author records in the original source systems and migrate that function directly into the MDM system.

It may be appropriate to switch to the proactive data governance approach after upgrading the company’s integration or middleware capabilities (for example, adding an integration tool capable of handling real-time updates), or as part of a major upgrade to the existing CRM or ERP systems, because that may be a good time to introduce business process changes that might be needed.

[ 11 ]

When to Migrate from Reactive to Proactive

Metrics will drive the business case for migrating from the reactive to proactive styles of data governance.

Ask yourself questions like the following, and try to quantify the answers in terms of investments of time, effort and expense:

• How long does it take to onboard a new customer?

• How many different steps are involved?

• How many times does the average new record get touched before being accepted into the multidomain MDM system?

• How many duplicate records are still being created in the source systems (and then merged in the MDM system) because of the limitations of those source systems?

• How many data stewards are needed to support the enterprise?

• Do master records seem to get into a “change, change back” cycle as two different groups of users attempt to enforce two different sets of business rules?

• Are important aspects of the master data “falling through the cracks” between the source systems and the MDM system?

• Is the process of maintaining the integration between various source systems and the MDM system becoming a burden?

• Do users complain about having to wait between entering a new record in the CRM system and having it become available in the ERP system?

• Are there funding issues for data governance because it’s seen as overhead or a bureaucracy?

After answering these questions, it should be apparent whether you’ll be able to make the case for migrating to a more proactive data governance approach. At that point, you can plan the migration process in detail, setting it up as a separate project or perhaps integrating it into another related project.

When to Start Out with Proactive Data Governance

Some situations call for starting out immediately with proactive data governance, such as when you’ve got multiple CRM systems and ERP systems that would require integration with the multidomain MDM system in order to allow them to continue to operate as Systems of Entry, or when your current source systems are very brittle or hard to maintain or modify.

In those cases, bite the bullet and plan from the beginning for proactive data governance. Some organizations have thousands of end users authoring master data directly in the MDM system, with a team of data stewards backstopping them, catching anomalies, resolving low scoring matches, manually merging duplicate records when needed, etc.

Another scenario is when you think you’re going to wind up using the proactive data governance approach in the end. Why go through the hassle of creating bidirectional integration from your source systems to the multidomain MDM system? You’re probably better off jumping straight to empowering end users to author master data.

About The AuthorDan Power is the founder and president of Hub Solution Designs, Inc., a management and technology consulting firm specializing in master data management (MDM) and data governance. He has 23 years of experience in management consulting, enterprise applications, strategic alliances and marketing at companies like Dun & Bradstreet, Deloitte Touche Tohmatsu, Computer Sciences Corporation, eCredit and Parson Consulting. Power speaks frequently at technology conferences and advises clients on developing & implementing high value MDM and data governance strategies.

About Hub Solution Designs, Inc.Hub Solution Designs, a global management and technology consulting firm, specializes in developing and executing high value master data management and data governance strategies. Through recognized thought leadership, an excellent reputation and a strategic network of partnerships, the firm delivers successful projects to Fortune 1000 clients, who are its best references.

About Informatica MDMInformatica MDM empowers business users to improve their operations with reliable views of critical master data distributed across data sources. The award-winning solution provides comprehensive, unified, open, and economical Master Data Management (MDM) on a single platform. It enables customers to manage multiple data domains and architectural styles and unifies all MDM requirements—data integration, profiling, quality, and master data management—on the same platform. Informatica MDM provides open data integration to all heterogeneous applications and data sources. Informatica MDM delivers faster time-to-value, lower TCO, and superior ROI because it can be rapidly implemented and is easily configured to quickly accommodate ever changing business needs.

About InformaticaInformatica Corporation (NASDAQ: INFA) is the world’s number one independent leader in data integration software. The Informatica Platform provides organizations with a comprehensive, unified, open, and economical approach to lower IT costs and gain competitive advantage from their information assets. Nearly 4,000 enterprises worldwide rely on Informatica to access, integrate, and trust their information assets held in the traditional enterprise and in the Internet cloud. Visit www.informatica.com.

[ 12 ]

[ 13 ]

[ 14 ]© 2010 Informatica Corporation and Hub Solution Designs, Inc. 7126 (04/08/2010)

Informatica CorporationWorldwide Headquarters, 100 Cardinal Way, Redwood City, CA 94063, USAphone: 650.385.5000 | fax: 650.385.5500 | toll-free in the US: 1.800.653.3871www.informatica.com

Hub Solution Designs, Inc.188 Whiting Street, Suite 3, Hingham, MA 02043-3844 USAoffice: 781.749.8910 | fax 781.735.0318 | www.hubdesigns.comblog.hubdesigns.com | info@hubdesigns.com