Brotherhood of Master Data - The MDM Cult Series Part 1

Dears,

The reason why the title has the world “cult” is to know that MDM is still not fully explored technology and successful execution of the MDM projects remains the authority of little knowledgeable brotherhood who indeed keep this secret very close to the professional network. This is an attempt to unleash the knowledge of MDM to wide group of audience whom I feel would make the best use of this information in their MDM Initiatives.

This article is designed to give a business-centric view of the master data management (MDM) concept and what challenges will be faced by organizations of various sizes. Business stakeholders within an organization will gain an understanding of the needs of their organization and will be empowered to define the initial MDM vision for their organization. This vision will facilitate the initial decision necessary to implement a master data management solution within their organization.

Different Challenges based on size of Organization

As any business grows, the management of this data is a critical driver to their success. Every acquisition brings in new sets of master data to be merged with the existing business. Although MDM is important to organizations of all sizes, the size of an organization plays a crucial role in how to approach the implementation of MDM.

Many small businesses do not consider their master data a problem to be concerned with. After all, the spreadsheets they currently use to manage their Product List work great and the accounting system is the only location that the chart of accounts needs to exist in. In my experience this is the easiest and cheapest time to handle the master data management problems. The number of stakeholders for each dataset is small. The number of systems that rely on each dataset is very small. This is the time to implement a master data management strategy that can grow with the business. Each new system implementation will benefit from the single source of all of master data.Mid-size organizations have a number of dependent systems for each set of master data, so system integration starts to become important. The number of stakeholders for each silo of master data is relatively small. These groups may still work in small teams efficiently. Effective master

data controls and an owner for each of the master data entities must be defined. These owners, usually called data stewards, are responsible for managing their domains as new systems are integrated into the organization.Large organizations have a number of challenges when implementing a comprehensive MDM solution. They are large enough that there are several stakeholders for each silo of master data. Many systems rely on these same models of master data. At this size, coordination of data is a central concern and requires the input of many different stakeholders.Conglomerates are the most complex MDM challenges. While they may be smaller in overall size in term of employees, assets, or revenue than their large organization brethren, the distinguishing characteristic of these organizations is the breadth of their products offerings or the diversity of their businesses. Typically these organizations have a diverse offering that makes tracking their master data very challenging. With a significantly diverse product offering, being mindful of how the different businesses interact is extremely important. Also, many industries have specific regulatory hurdles with regards to customers and products that may not be readily known by the organization as a whole.

Why current systems are ineffective master data applications

In many applications, especially ERP systems, master data is created and stored as a requirement of these process systems. Some companies may even call the teams that manage the central data for the ERP systems the master data management group. Although this group is a great place to start to source the new roles of true master data management, these systems do not provide many of the features required to properly manage master data for the entire organization.Some common limitations of these MDM strategies are:

Limited ability to version this master data Inefficient methods of exporting this data into other applications Master data is specific to the functions of the system that manages it and doesn’t readily satisfy

the requirements of other applications that need to consume it Inability to properly store hierarchies or change hierarchies as business requirements change Limited or no ability to model relationships between different data groups

Limited ability to version this master data

Functional systems require master data to run their specific operations for the organization. Their chief consideration is the most current general ledger, cost centers, organizational entities, and products. Companies spend thousands of hours and hundreds of thousands of dollars to re-organize their sales team. Invariably, a large portion of this time and money is spent mapping the old business units to the new structure.

Inefficient methods of exporting data to other applications

Large, business-wide applications are heavily customized for each organization. These systems provide limited ability to transfer data out of the master data systems. Most systems have some export mechanism that resembles a query language with output of text files. The ability to transform this data with system tools during the export process is very limited if it exists at all. It is also very difficult to export changes within a specified period of time. Due to the lack of versioning, it is very unlikely that master data transactions will be available.

Master data is skewed to the functions of the system it is in

These systems have been customized to provide tailored processes to the organization. In the process of customizing these systems, many of the strategies used to customize these processes revolve around making modifications to the master data stored. These changes may work well for their intended function, but as we will see ,storing data in a function-dependent manner makes it less usable to the rest of the organization’s systems.

Inability to properly store hierarchies

Some ERP/CRM solutions tout the ability to store master data hierarchically. In actuality this is usually managed by placing multiple identifiers into an attached attribute. By giving each character in this attribute special meaning, a surrogate-derived hierarchy can be formed in any subscribing reporting engines. This is a messy solution that tends to scale poorly. As a company grows, each of these character sets can become overextended, creating complex interim solutions. Changing the hierarchy requires changing the identifiers of all records, which can be prohibitively expensive.Proper hierarchy storage should allow for both derived-data hierarchy relationships and arbitrary parent-child relationships.

Limited or no ability to model relationships between different data groups

Many solutions are not designed to allow relationships to be made between two disparate data groups. Managing products per customer or products per salesperson can be difficult, if not impossible, as the systems may be working with a small subset of the overall corporate data set.

Watch this space for the second part of this Series

Loving P&CDC*

Brotherhood of Master Data Management -MDM must be process-Independent

Dears,

Welcome to the second part of this series.Large ERP systems are designed to manage all of the master data tailored for their system's needs. In this regard they are highly effective, but master data needs to be stored separately from the processes that use the data. As systems evolve over time, one of the easiest ways to modify complex system functionality is to modify the data to solve the problems. Many systems will have multiple customer IDs that map to the same customer to meet some custom reporting needs.

Another issue with storing master data in a process-oriented system is the need to store transactional history. As transactions are created, each is tagged with a combination of account, customer, product, cost center, and so on. These tags must be kept to maintain referential integrity in the system. These histories are like shackles to your master data, requiring multiple custom fields to maintain open and closed statuses.

Master data systems should be agnostic to the uses of this data. This approach keeps these records clean of any attempt to circumvent the programming of a production system. By eliminating the need to maintain ancient accounts for transactional history, MDM systems can provide clean representations of each master data set. Different methods of implementing master data managementA master data management solution can have many different looks. It is very rare to see a large organization implement a corporate-wide MDM solution in one project. Most of these projects

grow organically as different group associated with the project spread the word of the cost and time savings that MDM enables. There are a number of different factors that contribute to the style of implementation that is chosen. Some of these factors are:

Level of the organization behind this initiative Structure of the current organization Structure of the current functional systems Complexity of the systems to be integrated Size of the organization Degree of internal pain attributed to master data issues

Level of the organization behind the initiativeA true enterprise-wide MDM implementation requires the highest level of an organization

to underwrite the project. If data integration pains are felt at a lower functional level of the organization, single-dimension MDM solutions are a good fit. Once success is achieved for this one area, more centralized support for expanding the implementation may be found.Size of the organizationHow many people within the organization are dependent on this master data set? How many records need to be stored for each data set? These questions help an organization to determine the type of solution to implement. Once an organization reaches a certain size, implementing an enterprise-wide MDM solution in one project becomes unfeasible. A phased approach may be more prudent.Structure of the current organization

Is the company large and centrally located? Will multiple organizational units need to be synchronized? Will the internal corporate culture create drag on the implementation of any new solution? As much as size matters, the current structure of an organization matters more. Structure of the current functional system

When evaluating each system to integrate into an MDM solution, a number of structural elements can affect the decision-making process. Are all customers' records reflected in the system to be integrated? Do multiple records for the same customer provide some functional benefit to the system? Can these customer records be aggregated in the master data management solution? Freeing master data from process systems can allow for better data quality.Complexity of the systems to be integrated

It is important to evaluate the complexity of each system to be integrated. How critical is the system to the business? What are the best methods to import/export master data from the system? Will the master data stored have a high correlation factor to other systems within the organization?Degree of internal pain attributed to master data issues

As is the case with any IT project, how large a problem the current process creates for the organization is directly related to the amount of resources that are brought to bear on alleviating the issue. Without the financial incentive to optimize master data management, many organizations will choose less costly methods of data integration.

The MDM Cult Series-Part 3-MDM Inception Recipe

Dears,

We have a variety of MDM Application Suites catering to Product, Customer, Site, Activity, Supplier..etc .Out of this whole bunch two highly successful types of targeted MDM solutions are customer data integration (CDI)/Customer Hub and product information management (PIM) / Product Hub solutions.

Customer Data Integration / Customer Hub

Customer data integration solutions center around the integration of an organization’s customer data. These solutions are highly integrated with a company’s CRM and ERP systems. Due to the nature of combining so many disparate sources of customer data, match merge and duplicate removal algorithms are critical areas of data integration within the CDI subtype. As this name implies, most of these solutions are focused on integration and require additional internal processes to maintain these systems.

PIM) / Product Hub solutions

Product information management solutions provide an organization with product-centric management. These solutions typically focus on managing, correlating, and merging product data as bills of materials and online catalogs.

Due to the limited scope of these solutions, many organizations can implement them in a relatively short time. The limited scope also keeps the number of stakeholders that must reach a consensus to a minimum. Quick wins and a narrow scope cause many companies to implement these solutions, ignoring the serious limitations these solutions provide from the perspective of organizational master data management.Neither of these solutions is designed to be applied to all of the data sets within the organization. An organization can implement a number of separate solutions to create coverage of all their possible master data sets. Implementing these multiple solutions reduces the number of possible integration points but maintains data silos. Cross-dimensional relationships are difficult, if not impossible, to manage.

Working with ITIt is critical for business owners taking on the challenge of MDM to work well with the organization's Information Technology group. While many of the MDM management tasks required for sustained success rely heavily on the business users themselves, management of the technologies associated with the data integration routines will need to work well with the IT current processes.

Implementing in a phased approachNow that we have detailed all the possible techniques to implement master data management within the organization, let us lay out some efficient ways to implement MDM within an organization. Each of these processes can provide an organization a reasonable path to grow MDM through the organization. An organization’s structure will have a significant impact on which method will be the most feasible.

Single Dimension Build OutMany organizations come to realize that master data management is needed in their organization through one significant business driver. Commonly, these issues revolve around only a few distinct dimensions within the business. These pains within an organization provide the perfect location to start the MDM process. A few common characteristics that will assist in this approach are:

Single dimension being affected Low resistance to a new system of entry for this dimension Minimal additional stakeholders required for complete implementation Central data management (at least for the dimension in question)

For this example, let us discuss a fictitious company called Fabrikam, Inc. This company spends at least 20 hours per month reconciling changes between seven systems that rely on account-related information. New accounts are created by three different groups. The Accounts Payable group creates a new account for each new supplier. Accounts receivable creates a new account for each new customer. All other account changes are handled by the financial controller. These changes and all the corresponding attributes must be propagated to all seven systems. Difficulties arise because many of these accounts are created a few months before a balance is shown in them. If a system is not in sync at that time, reports will not balance and it is difficult to determine where the error is coming from.

Phased Approach – Before

After reviewing their options, the finance and IT departments agreed that this problem was a commonly recurring issue with the master data within the organization. The IT department was able to identify at least six places within the organization where critical company master data needed to be synchronized. While the time and monetary resources were not available to implement an enterprise-wide MDM solution in the next quarter, it was determined that the solution provided should have the ability to scale across the organization.

Three months later, the initial implementation was a success. All three account creators were using the MDM solution to create new accounts. All of the account structures were being propagated to the seven internal systems. The company spends less than one hour a month reconciling issues between any of the financial systems. After seeing the initial success of the finance department's implementation, the sales organization is preparing a project to leverage the MDM system to manage its active customers. The accounting group will be able to leverage this data when creating the accounts receivable data as well.

This project is prepared to grow organically throughout the organization, each group taking advantage of the efficiencies learned in early phases of the MDM implementation. As the project grows into other areas of the business, it is imperative that clear ownership is determined. These "data stewards" will be discussed in more detail in the next white paper.

Phased Approach – After

Complete enterprise MDM implementations

Complete MDM solutions require the entire lifecycle of the master entities to be managed from within the master data management solution. Controlling the entry of the master data allows the enterprise application to proactively manage the quality of the data. Although an enterprise implementation will be both the system of entry and system of record for all master data entities, it may still require mapping data to other applications.

It would be naive to think that any company will be capable of getting all of their systems to use the exact same set of data. Some transformations will still be required to run the process systems. This does not mean that every defining characteristic of an entity is managed within the solution. Those defining attributes unique to an organization's system operation should be managed within the source system where they have relevance. The enterprise solution should provide a broad range of entry points to be a viable option as the system of entry.

There are four techniques for implementing master data management within an organization. These methods differ in the amount of control they exert over the master data they manage. All of these techniques rely on reliable data integration solutions.In the following section, a number of acronyms will be used within system illustrations. These acronyms are described below:

Acronym Long Name Description

SOE System of Entry

Primary point of data entry. This may be direct entry or through services that update the data in virtual real time.

SOR System of Record

Most, if not all systems, receive their data from this source. When conflicts arise, this system is considered primary.

MDISMaster Data Integration Services

Data cleansing and integration processes that provide automated methods for some of the following activities: segmentation, aggregation, transformations, match/merge, and grouping.

IM Identity Mapping

To eliminate repeated integration, identity maps should be used to manage surrogate key relationships. These may be one-to-one or one-to-many.

BI Business Intelligence Technology and applications dedicated to the

analysis and presentation of business information.

CRMCustomer Relationship Manager

Customer management software.

ERP Enterprise Resource PlannerSoftware system that serves all areas of a business enterprise.

DW Data Warehouse Repository of electronically stored data.

S1,S2 … Additional SystemsThese are just placeholders for company-specific applications that are yet to be defined.

Master Data Registry In registry implementations, each system remains in control of its own data. All system

data records are mapped in the master data registry. Data maps show the relationships between the keys of two different systems. These keys can be mapped in two different ways:

One to one: Every record in the main system will have only one corresponding record in the secondary system.

One to many: Every record in the main system will have one or more corresponding records in the secondary system.

These mappings provide the data integration applications a reliable way to compare related items. At any time, different systems can be compared and cleansed. Although this technique provides important mapping information to the organization, any new items in any system will lead to data inconsistencies within the solution requiring a very complex data management story.Figure 1: Master Data Registry

Data aggregation solutionsIt is very common for initial MDM implementations to implement this type of watered-

down approach. In many cases the applications and systems used for this technique are identical to the more advanced techniques listed below. The major missing factor in the solution is control. It is very difficult for an initial MDM project to get all of the necessary stakeholders to relinquish control of their data to a new product immediately. Another system, usually the most critical business transaction application, remains the system of record and system of entry. Integration processes transfer data from this initial source to the MDM application. This master data may be enhanced by the MDM application itself, but a majority of the important information is still imported from the more entrenched system. These systems may be more entrenched due to a number of factors, including importance to the business, amount of time spent cleansing the data,

current process, or even perception of value. Data will then be propagated to other systems. Data controls will be limited as the source system will not be designed to account for any other system’s requirements. Despite the limitations inherent with this method, this is actually a good method for bringing quick wins to an organization with MDM. Many stakeholders are reluctant to give up the security of their traditional data management processes. By showing early benefits and demonstrating application reliability, an organization can develop trust in the MDM application as the system of record and system of entry.Using this technique for the initial implementation, risk to the mission-critical application can be mitigated effectively. Less critical applications can begin to source their master data from the MDM application, solving any integration issues that arise without major ramification to the organization. Integration processes can be tested and modified in an iterative fashion.Figure 2: Data Aggregation

System-of-record-only implementationsThese implementations give complete control of the master data sets to the MDM

application. Other systems provide the initial data to be imported into the MDM system, but, unlike the data aggregation solution, the flow of data from this System of Entry is bidirectional. New records are transferred into the MDM application for integration. Any discrepancies in the data defer to the MDM application, which is the system of record.

These implementations still require a degree of data integration and ongoing cleansing as elements may come from both the source system and the MDM application. Also, many times this system of entry only has the ability to detect data issues directly related to the initial use. For instance, any customer information that is not stored in the CRM solution will not be available to determine complete data quality.

Figure 3: SOR only (Hub)

Complete enterprise MDM implementations

Complete MDM solutions require the entire lifecycle of the master entities to be managed from within the master data management solution. Controlling the entry of the master data allows the enterprise application to proactively manage the quality of the data. Although an enterprise implementation will be both the system of entry and system of record for all master data entities, it may still require mapping data to other applications.

It would be naive to think that any company will be capable of getting all of their systems to use the exact same set of data. Some transformations will still be required to run the process systems. This does not mean that every defining characteristic of an entity is managed within the solution. Those defining attributes unique to an organization's system operation should be managed within the source system where they have relevance. The enterprise solution should provide a broad range of entry points to be a viable option as the system of entry.

Figure 4: Enterprise Solution