Department of Orthopedic Surgery

Improving the Data Audit Process at the Michigan Arthroplasty Registry Collaborative

Final Report

To: Rochelle Igrisan, MBA, MSN, RN, Senior Project Manager

April Richmond, BSN, RN, Senior Clinical Information Analyst

Richard Hughes, PhD, Co-Director of MARCQI

Thomas Zheng, PhD, MS, Expert Statistician

Mark P. Van Oyen, PhD, IOE 481 Faculty Instructor

Mary Duck, Michigan Medicine Liaison

From: IOE 481 Project Team #4

Pranita Balusu, Industrial & Operations Engineering

Brandon Brathwaite, Industrial & Operations Engineering

Klaudia Mastela, Industrial & Operations Engineering

Date: December 12th, 2017

Table of Contents

EXECUTIVE SUMMARY 3Background 3Opportunities 3Methods 3Findings 4Conclusions and Design Recommendations 4

INTRODUCTION 5

BACKGROUND 6Key Issues 7Goals and Objectives 7Project Scope 7

DATA & METHODS 8Interviews and Surveys 8Data Collection 8Data Audit Observation 8Literature Review 9

Alternate Methods for Billing Audits 9Data Analysis 9Recommendations 9Deliverables 9

FINDINGS & CONCLUSIONS 9Interviews and Surveys 10Billing Audit Observation 10Literature Review 11

Alternate Methods for Billing Audits 11Excel Formulas & Functions 12

Data Analysis 12

DESIGN METHODS, CONSTRAINTS, AND STANDARDS 12Design Goal 12Design Process 13

Constraints 13Standards 14Pugh Matrix 14Excel Design Process 15

DESIGN RECOMMENDATIONS 15Excel Template 15

1

User Manual 16Possible Improvements 16

EXPECTED IMPACT 16

REFERENCES 17

APPENDIX 18

2

EXECUTIVE SUMMARY

The Michigan Arthroplasty Registry Collaborative Quality Initiative (MARCQI) collects data on hip and knee replacement surgeries in the state of Michigan. Information from operations is audited regularly to ensure completeness and accuracy of data. MARCQI is continuously looking to improve the data audit process to reduce audit times and improve identification of inconsistencies in the data. The team from IOE 481 was asked to understand and provide recommendations for the data audit process. The scope of this project included the billing audit process for MARCQI auditors. The team performed design work, developed an Excel template, wrote a user manual, and conducted a training session with the auditors. This report summarizes the team’s findings, conclusions, recommendations and deliverables for improving the billing audit process.

BackgroundMARCQI consists of 60 participating hospitals which each performing at least 200 hip or knee arthroplasties annually. Data for these operations is constantly collected by Clinical Data Abstractors and recorded in the MARCQI database. Then, four auditors trained at the MARCQI coordinating center in Ann Arbor perform annual audits on the data. These consultants must compare the data from two different sources - the hospital sites and the MARCQI database - and ensure there are no inconsistencies between the datasets. The current process for conducting these audits, which are called billing audits, has little automation and is extremely time consuming.

OpportunitiesTo identify different ways for MARCQI to improve the efficiency of the billing audit process, the team looked into different methods of automating the audit. Specifically, the team researched the use of Excel in comparing data sheets and the use of other software, such as R, Python, and SAS. After comparing the benefits of all the possible programs to use for billing audits, the team decided to use Excel to build a template for MARCQI auditors to use for billing audits. Although other programs provide greater operating speeds and the capability to conduct audits on large datasets, Excel is able to handle datasets as large as all the MARCQI hospitals and provides a more user-friendly interface. Thus, the team decided to use Excel to improve the billing audit process.

Methods● Interviewed and surveyed the Senior Clinical Information Analyst, Senior Project

Manager, and MARCQI auditors. The team interviewed the Senior Clinical Information Analyst, who is one of the auditors, and the Senior Project Manager to gain a better understand of MARCQI’s goals, the audit process’ current state, and the billing audit’s inefficiencies. Moreover, the team gathered results from a survey on the current billing audit process from all of the auditors to understand specific quantitative information, such as the average time it currently takes for each audit and the average number of errors an auditor can expect to see in each audit.

● Collected data at weekly team meetings. At each weekly meeting, the team gathered MARCQI documents and datasheets to understand the current billing audit process and

3

the various fields of data MARCQI collects from hospital sites. The Senior Clinical Information Analyst provided the team with historical data sets of data from 2015-2016.

● Observed a billing audit. The team organized an observation session with the Senior Clinical Information Analyst to fully understand the current billing audit process and identify the primary areas for improvement.

● Conducted a literature review. The team conducted literature searches on various methods of data comparison using different tools, such as Excel, Python, R, and SAS. After exploring these opportunities and then deciding to use Excel for the project, the team researched different methods of data comparison within Excel to identify the most efficient format to use for the Excel template.

● Analyzed collected data. The team reviewed all the data gathered from interviews, surveys, observations, literature reviews, and datasets. The team used the data analysis to consolidate the information into specific requirements and targets for the Excel template.

● Developed recommendations. Using the data analysis to guide them, the team next developed specific recommendations that addressed the concerns and pain points identified by the auditors. These recommendations are included in the Excel template and in written instructions in a User Manual provided in conjunction with the template.

● Designed Excel template. The team used the recommendations to dictate the design of the Excel template. The template was provided as one component of the final deliverable along with the User Manual and an in-person training session.

FindingsTable 1 provides a summary of the team’s findings after analyzing the data from various methods the team used throughout the project.

Table 1: Summary of findings from various methods.

Method Finding

Interviews and Surveys ● Data audits currently take an average of 11 hours per site

● Auditors would like audits to take an average of 2 hours per site

Billing Audit Observation ● Primary pain point is that audits are extremely manual and repetitive

● Secondary pain point is the inconsistent formatting of data

● There is no standardized process or instructions sheet

Literature Reviews ● Excel provides sufficient functionality for billing audits

● Can use conditional formatting to highlight differences

● Can use sort and filter methods to identify inconsistent and missing cases

4

Conclusions and Design RecommendationsAfter analyzing the data, the team then drew conclusions and used these conclusions to determine the most effective way to design the Excel template so that the time for each billing audit would be reduced. These conclusions and recommendations are summarized in Table 2.

Table 2: Summary of conclusions and design recommendations.

Conclusion Design Recommendation

The primary pain point, which is an effect of the repetitive and manual nature of billing audits, can be solved by adding automation to the process.

The Excel template should automate the billing audit process by:

● Consolidating two sheets of data into one to facilitate comparison

● Highlighting specific fields and differences between various sheets using conditional formatting

● Providing auditors with the option to sort and filter cases that are correct to focus solely on the incorrect cases

The overall process would be more straightforward and the auditors would be better able to address any issues that come up with more thorough and descriptive instructions.

The Excel template would be complemented with:

● A User Manual that thoroughly describes the functionality of the Excel template

● A User Manual that includes any possible errors that may occur and how to handle them

● An in-person training session to provide auditors with a clear and real-time example and clarify any initial questions or concerns

Copies of the Excel template and User Manual are provided in the appendix.

5

INTRODUCTION

The Michigan Arthroplasty Registry Collaborative Quality Initiative (MARCQI), a statewide consortium of hospitals that is funded by Blue Cross and Blue Shield of Michigan and the Blue Care Network, was established in 2012 to improve the quality of care given to patients who receive total hip or knee replacements. To accomplish this, MARCQI collects ~153 elements of data on information such as patient demographics, patient risk factors, processes of care, and outcomes for every hip and knee replacement surgery that is done in the 60 partner hospital sites [1]. The data is then audited by MARCQI employees at regular periods to ensure that data is complete and accurate and that each data element is interpreted with consistency across all participating hospitals.

The current data audit process MARCQI has in place for participating sites has some inefficiencies. Because of these inefficiencies, the audits are taking too much time and auditors are not able to perform as many audits as they would like. MARCQI is concerned they are not identifying all inconsistencies in data since they are only able to conduct audits annually. Therefore, the Project Manager would like to know whether there is a way to improve the data audit process to better ensure data quality and reduce the time required for each audit. The Project Manager has asked an IOE 481 team from the University of Michigan to identify potential process improvements. The team has been asked to observe the data audit process and the audit preparation process to analyze both the length of time and thoroughness of each step. The team’s analysis was used to recommend process improvements to MARCQI to ensure more economical and accurate data audits. The following report presents the IOE 481 team’s methods, findings, conclusions, and final recommendations for this project.

BACKGROUND

The primary goals of MARCQI are to improve the quality and safety of care given to total knee or hip arthroplasty patients, analyze registry data to identify the poorly performing new technologies in total joint arthroplasty, and show both patients and purchasers that the hospital sites under MARCQI are improving the value of arthroplasty services [1]. The coordinating center, housed at the University of Michigan, ensures that these goals are met by overseeing the quality of data collected and then conducting research using this data. In order to collect data, MARCQI requires that each hospital site hires Clinical Data Abstractors, or CDA’s, who contribute data weekly to the MARCQI database on all of the total knee and hip replacement procedures performed. The coordinating center then conducts two different types of audits on all the participating sites to ensure that the data is accurate, complete, and interpreted in the same manner across all the sites.

One type of audit MARCQI conducts is the on-site audit, for which auditors go to each hospital site and compare the data the CDA’s input into the MARCQI database with the data in the

6

hospital database. These audits are now conducted every other year. The more frequent audit that MARCQI conducts is the billing audit. The billing audits are done annually and compare the charges made by the hospital billing department with the data entered into the MARCQI database. According to the information provided by the Senior Clinical Information Analyst, one of the four auditors at the MARCQI coordinating center, in interviews, data comparisons are done on Excel using the VLOOKUP function, but take at least a few hours per site, which is too much time for the current capacity of the data auditors. However, this type of audit is important as it ensures two facets of data quality. First, the billing audits ensure completeness of data since the billing information received from the hospital billing departments includes every surgery performed. MARCQI compares this information to the data put into the MARCQI database to check that the CDA’s have recorded all the cases completed at the hospital. Second, billing audits ensure that various data elements, such as patient name and procedure code, are recorded accurately.

The intent of this project is to explore the billing audit process of the MARCQI audit process and provide recommendations on how to make the process more efficient.

Key IssuesThe following key issues are driving the need for this project:

● All necessary information may not be wholly or accurately captured with current auditing practices, sometimes resulting in questionable data

● Areas of inefficiency currently exist in the billing audit process, resulting in longer audit times

● Audit frequency is low, increasing the need for shorter audit times in order to conduct a sufficient amount of audits to ensure data quality

Goals and ObjectivesTo recommend process improvements that increase the efficiency of MARCQI’s billing audit process, the team completed the following tasks:

● Interviewed MARCQI auditors, including the Senior Clinical Information Analyst, and the Senior Project manager

● Obtained data samples from MARCQI the Senior Clinical Information Analyst and the Senior Project Manager

● Researched Excel formulas to identify more efficient ways of comparing two sheets● Analyzed data collected from observations

Using the information gathered from these tasks, the team was able to:● Determine reasons for inefficiencies in the steps of the billing audit process● Recommend improvements that can reduce time required for audits or ensure

effectiveness in finding inaccurate or incomplete data by designing an Excel template to be used for the billing audit process

Project ScopeThe team focused solely on understanding and improving the process for billing audits. Such audits involve the use of Excel data sheets, and the team’s final deliverable is an Excel template to use while conducting billing audits. The following table gives an overview of the specific components of MARCQI’s audit process that are in and out of scope:

7

Table 3: Project Scope

In Scope Out of Scope

● MARCQI’s billing audit process○ Excel template for billing

audits○ Tutorial and user manual for

template

● On-site audit process● Any other processes involved in

MARCQI’s audit process

DATA & METHODS

This section presents the team’s plan for completing the objectives with a primary goal of improving the billing audit process. The team acquired data through interviews and surveys, observations of MARCQI data and the billing audit process, and literature searches. Data analysis, recommendations, and final deliverables were developed based on this methodology.

Interviews and SurveysThe team interviewed the Senior Project Manager and the Senior Clinical Information Analyst associated at MARCQI during weekly meetings. Questions focused on the billing audit process in its current state across all sites and the on-site audit process at Michigan Medicine, including the planning, execution, and follow-up phases of an audit. The team also asked questions to identify the inefficiencies and pain points currently associated with MARCQI billing audits. Finally, the team constructed surveys using Qualtrics survey software and distributed them to the Senior Clinical Information Analyst at Michigan Medicine, who shared the surveys with the three other MARCQI auditors. The survey questions asked are included in the Appendix.

Data CollectionThe team met weekly with the Senior Clinical Information Analyst, who has been trained at the Ann Arbor coordinating center, to acquire an understanding of the audit process in its current state. The team also acquired a data sample of a previous audit from the Senior Project Manager and the Senior Clinical Information Analyst. This consisted of two historical sets of billing data for one site with a one year span of data from 2015-2016. One set consisted of data recorded by the site that was sent directly to the MARCQI office, with approximately 4700 rows of data. The other dataset was retrieved by the nurse coordinators from the MARCQI database corresponding to the site and consisted of approximately 3900 rows of data. Acquiring this data allowed the team to understand common trends and issues associated with past MARCQI auditing, data collection, and data reporting.

8

Billing Audit ObservationThe team organized and observed a mock audit at the coordinating center with the Senior Clinical Information Analyst to gain first hand exposure to the billing audit process in its current state. Additional pain points in the process that were not captured in weekly interviews and surveys were identified through the observation. This observation assisted in determining the priority of the functionalities implemented in the team’s Excel template for data audits.

Literature ReviewThe team obtained MARCQI literature, supplied by the initiative’s Co-Director and the Senior Project Manager, which provided an overview of MARCQI and detailed its operations. The team also conducted online research for technical papers and articles detailing methods and techniques which would assist in improving the billing audit process.

Alternate Methods for Billing AuditsDuring meetings with the MARCQI Co-Director and Expert Statistician, the team was provided with methods for automating the billing audit using other analytics and programming softwares such as Python, R, and SAS, which were then compared to Excel. The advantages and disadvantages of using Excel compared to these other programs were explored, and further discussed in detail in findings.

Excel Formulas and FunctionsAfter deciding that Excel would provide the best user interface, the team conducted additional literature searches on various methods of data comparison in Excel. These included methods of identifying and methods of highlighting differences in data between two different sheets or between two different rows.

Data AnalysisDuring data analysis, the team analyzed the audit process in its current state using data obtained from interviews, surveys, observations, and the literature review. Data analysis tools consisted of Qualtrics and Excel, which were used to specify targets for time improvements. However, the most useful components of the data analysis resulted in a list of requirements and targets for the design of the Excel template. The most problematic aspects of the billing audit process were identified based on the data.

RecommendationsDuring the recommendation phase, the team developed suggestions for improving the audit process. Areas of improvement for the audit process, such as audit efficiency, audit time, audit quality, were identified. An explanation of the steps that can be removed and/or automated to make the audit process faster was developed based on the steps taken during the development phase of final deliverables.

DeliverablesThe team next developed deliverables showcasing the proposed improvements to the billing audit process. The team finalized an Excel template with functions that automate various elements of the billing audit process. A User Manual was constructed in conjunction with the template to provide a description for use of the program and to allow for standardization of the new processes that are adopted. An in-person training session was also organized with the Senior

9

Clinical Information Analyst and MARCQI auditors for further instruction on use of the program.

FINDINGS & CONCLUSIONS

Based on the described methods, the team discovered the following findings regarding the billing audit process. The surveys, observation, and literature review were the primary source of these findings. Based on the findings from the methods, the team also drew the following conclusions regarding the billing audit process.

Interviews and SurveysThe team received the results of the Qualtrics surveys distributed to the Senior Clinical Information Analyst and the other three auditors for a total of 4 responses. The team gained insight regarding the average amount of time required to complete an audit and the most common data fields that contain errors during audits. The survey results show that billing audits often take an average of 11 hours for one site but the desired time for each audit is 2 hours. As the billing audit in its current state is a very manual process, however, there was a great deal of variation in length of audits. This is due to the vast differences in the size of participating MARCQI sites, which range from performing a minimum of 200 arthroplasties to several thousand annually [1]. A graph showcasing the correlation between number of cases in a site and the average hours to complete an audit is included in the appendix.

After asking survey respondents to identify the most common errors in billing data, the results indicate that the most common error is a case that is completely missing from either the site data of MARCQI dataset for a MARCQI institution, accounting for 55% of errors. The remaining 45% of errors are due to differences in the entries of the 7 data fields that nurse coordinators across site and MARCQI sets. These results revealed that, in addition to identifying differences in entered data, efficiently locating cases that are completely missing from either of the data sources for billing audits can significantly improve the billing audit process.

From the survey, the team realized that the time consumption of the billing audit needs to be reduced. Since 55% of the issues come from completely missing cases, the team decided that there should to be a way to distinguish between cases which have an inconsistency within the 7 data fields and the cases which are possibly missing. To do this, approximate matches had to be identified and inconsistencies between those matches were to be identified. Cases which did not had an identical or approximate match would be flagged as possibly missing.

Billing Audit ObservationThe team’s observation of a mock billing audit at the Ann Arbor coordinating center led to important discoveries regarding the current MARCQI billing audit process. First, the team discovered what the auditors use to assess the accuracy of the billing audits. The mock session revealed that although there are several columns of data that are captured in both the site and MARCQI datasets, the nurse coordinators are only concerned with comparing 7 columns of data: Last Name, First Name, Date of Birth, Surgery Date, the first 5 characters of the Procedure Code, Date of Admission, and Date of Discharge. Checking that there are no differences in data entry across site and MARCQI sets for these 7 fields is the primary indicator of the accuracy of reported data.

10

The team also found additional pain points in the billing audit process. Another obstacle encountered by the Senior Clinical Information Analyst during the mock audit was the inconsistent formatting across site and MARCQI datasets for corresponding arthroplasties. This is due to the current lack of a standardized process for performing billing audits. Currently, MARCQI provides a guideline for full-time nurse consultants to follow when auditing data in Microsoft Excel, which includes steps such as using the VLOOKUP function to manually compare the site and MARCQI datasets corresponding to the same hospital. Issues arise when grammatical errors in each dataset result in the VLOOKUP not properly capturing all cases, due to slight differences in data entry procedures that exist across MARCQI sites. Thus, auditors then perform a lengthy process of manually checking two Excel sheets against one another to identify whether errors exist in the site or MARCQI data.

An initial conclusion of the observation was to remove the waste of having to keep two spreadsheets open simultaneously and going back and forth between the two by consolidating the data into one set. The auditors would start by concatenating the relevant data fields into one field then use VLOOKUP to remove all identical cases. The remaining cases are manually looked over for inconsistencies. This was done by looking for a similar case in one set to match with each remaining case in the other set. If a similar match was found, each of the seven fields were compared to determine the inconsistency. If a similar match was not found, the case was marked as missing within one of the data sets.

After determining to combine the two data sets, the next issue was the manual comparison of the remaining cases. Searching for a similar case and then determining a difference between the two cases was the most time consuming issue. Automating the comparison of different cases significantly decreases the time of the billing audit from the current state. By automating the comparison, this also allows for automating identification of cases which are possibly missing from either of the two data sets.

After attempting to do a billing audit using the current instructions, the team realized that there were missing explanations for certain processes such as formatting columns. A User Manual was designed to provide a standardized and extensive explanation of the new billing audit process. This includes not only steps for the template, but also how to proceed when problems arise as well as tips regarding Excel functionality that makes the work more efficient.

Literature ReviewThe literature review was a factor in deciding on the design methods for the deliverable, covered in more details in the Designs Methods, Constraints, and Standards section of this report, and research into formulas and functions to use in the template once the design method was chosen.

Alternate Methods for Billing AuditsAfter several discussions with the MARCQI Co-Director and Expert Statistician regarding methods for automating billing audits with Python, R and SAS, the team developed findings based on the functionality of these programs, restrictions imposed upon MARCQI, and comparisons to Excel. Ultimately, the programming oriented nature of these softwares results in more robust functionality and less computational limitations. Excel has the options of coding with VBA, using macros, and using functions and formulas, which provides limited flexibility.

11

When considering computing speed and flexibility, other programs such as Python, R, and SAS may be better than Excel, but the users and future issues were also taken into account. The current process is performed using Excel, so the users have experience with it and would be more likely to adopt a template made in a program they feel comfortable with. The team also has experience with Excel making it more likely to create a template using Excel. Ultimately, due to IT restrictions within MARCQI, the user-friendly interface of Excel, and auditors’ familiarity with Excel compared to these solutions, the team decided to remain within Microsoft Excel for all recommendations and deliverables. The selection process for the template program is covered in more details in the Designs Methods, Constraints, and Standards section of this report.

Excel Formulas and FunctionsThe team performed research on various methods that could be implemented in the deliverables and recommendations after narrowing the scope of the project to improving the billing audit process within Microsoft Excel. Technical articles were found online which detailed methods for automating different comparisons in the software. These methods included comparing multiple columns of data for matches and differences and highlighting them [2], comparing data from two different Excel files for differences [3], and automatically removing duplicate data found within an Excel file [4].

Once the team had decided the design method would be Excel, the review provided information which supported functions and formulas as the best method for a design. VBA and other coding within Excel was determined as a possible cause of future issues and would slow down the program, so the template was designed with only functions and formulas. MARCQI’s data sets can have thousands of cases, so the program has to be able to handle the size without slowing it down significantly which could be an issue with adding coding to the Excel template. Comparing multiple columns of data for matches and differences could be done with IF and COUNTIF statements. Then, highlighting significant information could be done using conditional formatting to call out the inconsistencies. Although it was possible to remove duplicate data, the team found alternatives which allowed the distinguishing which data had duplicates and which had inconsistencies. This was done in case the auditors needed to reference those cases.

Data AnalysisAfter coming to the conclusions mentioned in the previous sections, the team was able to create an Excel template as well as a User Manual. The Excel template includes combined datasets from both the site and MARCQI data and identifies how many duplicate cases are present, if a case has an approximate match, and calls out inconsistencies in approximate matches and potentially missing cases. The template was first created using a sample of fictional patients with known outcomes to know if the template was addressing expected issues.

After a functioning template was designed, the team observed a mock billing audit performed with the template and then used the obtained historical data and a secure laptop to test the template themselves. The trial run brought issues with formatting and understanding the data to the team's attention. Formatting was addressed with the auditor to understand the usual format of received data and how it is normally dealt with. From this information, the team created standardized procedures to deal with the issues which were then included in the User Manual. After making adjustments to the template, the team observed another mock billing audit

12

performed with the updated template. This provided feedback from the auditor about changes which would make the template easier to use by the auditors. The feedback from the final trial run was researched and implemented where possible. A User Manual was then created based on the final Excel template.

DESIGN METHODS, CONSTRAINTS, AND STANDARDS

Design GoalThe final deliverable of the project is where the primary design challenge lied. While the project did not involve much data collection or process mapping, a template that the auditors at MARCQI can use when they conduct billing audits was designed as the final deliverable. This deliverable was intended to automate the process of comparing fields in the audit so the auditors can reallocate time to more meaningful and less repetitive tasks.

Design ProcessSince the deliverable was so broadly defined, the first step of the design process was to identify the tool or software to use for the design of the template. In order to select this tool, there were specific constraints and standards that had to be followed, which are outlined in Table 4 and then described in detail.

Table 4: Constraints and Standards Matrix

Category 1 2 3

Constraints Limits on scope C-1 C-2

End user requirements C-3

Development and implementation C-4 C-5

Standards Confidentiality and protection S-1 S-2 S-3

Stakeholder expectations S-4

ConstraintsC-1. Financial constraints - MARCQI, which is funded by Blue Cross Blue Shield of Michigan, has a limited budget to spend. Thus, there are certain tools that could facilitate the process of the billing audit that cannot be installed on the computers of all the auditors, such as Excel add-ins, due to costs associated with purchasing licenses and softwares.

C-2. Hospital IT department constraints - The IT department at Michigan Medicine has specific and strict guidelines that limit installation of additional softwares onto computers that access Michigan Medicine’s network. Moreover, the IT department restricts the use of web-based tools when accessing patient data.

13

C-3. Accessibility for auditors - The auditors who will be using the final deliverable are not familiar with all computer languages or tools, such as Python. This limits the scope of tools and softwares to those that the auditors are comfortable using and updating in the future if necessary.

C-4. Time constraints - A significant constraint is the amount of time available to complete the Excel template. Time must be taken into account because this limits the number of times the template can be tested to identify possible bugs; since the team is not able to account for all exceptions that may occur, the aspects of the template that cover the greatest number of cases had to be prioritized.

C-5. Data availability constraints - Due to MARCQI’s confidentiality standards, a limited amount of MARCQI data sets was available to analyze while building the template and to test the template on. Thus, a trial set of data was created by the team for testing purposes, which may result in a template that isn’t as accurate with MARCQI data.

StandardsS-1. HIPPA - In accordance with HIPPA, the data sets received from MARCQI have limited patient fields than the full data sets to maximize protection for patient information.

S-2. MARCQI confidentiality - In addition to HIPPA, MARCQI’s own confidentiality rules dictate the protection of patient identifiers received in data sets and that the identities of specific hospital sites are not revealed either; like in MARCQI’s collaboration with hospitals and external partners, hospital sites are not to be referred to by name.

S-3. Data access standards - The high importance of maintaining patient and hospital site confidentiality in MARCQI also dictates the devices the data sets can be accessed on; the team was limited to working on designated computers when using MARCQI data.

S-4. Blue Cross Blue Shield expectations - MARCQI must comply with Blue Cross Blue Shield’s policies and practices at all times since Blue Cross is the sponsor of MARCQI. Thus, as consultants for MARCQI, the team was also subject to the same best practices and expectations. For example, confidentiality and accurate reporting are two expectations for Blue Cross.

More standards could not be found after searching on OSHA, MiOSHA, NIOSH, ASTM, ANSI, SAE, Mil-Specs, etc. Web searches using the UM library online resources were performed from November 7, 2017 to November 14, 2017.

Pugh MatrixThe following Pugh matrix shows the decision process behind setting specific objectives and requirements for the design of the final deliverable template. After narrowing it down to two tools that would provide a deliverable design with the necessary functionality, the tool that would be the simplest for the end users (the auditors) was selected based on the criteria outlined in Table 5.

Table 5: Pugh Matrix

14

Excel Python

Criteria Weight Score Weighted Score Weighted

Effectiveness in audit time reduction

5 5 25 5 25

Time to design and implement

4 5 20 3 12

User acceptance 4 5 20 2 8

Ease of training users 4 4 16 2 8

Scalability to MARCQI level data

2 3 6 5 10

Total 87 73

The goal of the project was to design a template that makes the billing audit process as simple and efficient as possible for the data auditors so they can dedicate time to less repetitive tasks that cannot be automated. Since the auditors are the end users, auditor satisfaction was prioritized over other criteria to ensure that the deliverable would be useful for MARCQI. According to the Pugh Matrix, Excel was the better option for the deliverable template.

Excel Design ProcessWhen approaching the Excel design, the team was given seven fields that were used by MARCQI in the comparison of the data from the two different data sources, the data from the billing department at the hospital and the MARCQI database from CDAs at the hospitals. The main point of the billing audit is to make sure the data is accurate and fully represented, so the Excel had to identify mistakes between the two datasets and where a full data records was missing from either one of the data sets. To do this, current methods already used within the current billing audit process, such as concatenate to identify identical matches, were combined with new methods, such as if statements to identify approximate matches and differences within approximate matches.

DESIGN RECOMMENDATIONS

Based on the findings and conclusions from the data, the team recommends using the Excel template and User Manual designed for the billing audit throughout this project.

Excel TemplateThe first step in designing a new billing audit process was determining a way to standardize and automate the work as much as possible which the team recommends an Excel template to be used. Initially the billing audit process did not use a template, but rather instructions on how to perform the audit using Excel.

15

The Excel template had 3 prepared tabs: one tab for the MARCQI data, one tab for the data provided by the billing department, and one tab for the combined data and processing the data. The two tabs for the different data sets contained columns for the source of the data, last name, first name, date of birth, surgery date, procedure code, additional procedure codes, admission status, admit date, and discharge date. The seven fields excluding additional procedure codes and admission status are important to have matching in both sets.

Once the data is combined, the template had a column which computes the number of cases within the combined data with the same seven fields. This field determines the number of exact matches, so any cases with only one case is either missing from a data set or has inconsistencies between the two sets.

The data is then sorted according to number of cases and then last name, which allows the auditors to easily see which cases are causing issues. The template also has formulas to determine which set the missing data is missing from and which fields are inconsistent with an approximate match. Highlighting the differences and identifying the missing cases significantly decreases the amount of time auditors spend on manual identifying issues. The auditors then review the remaining cases to see if the cases have inconsistencies due to additional procedure codes or if they were correctly identified. Although there is still manual work in the process, the amount of manual work is significantly decreased.

User ManualA User Manual provides standardization and issue resolution for the Excel Template, which provides steps for procedures which were not addressed in the original instructions. The User Manual includes an introduction to the template, instructions on performing a billing audit, how to address possible issues, and tips for using Excel, see Appendix B.

Possible ImprovementsDue to the time constraints of this project, the team was unable to explore and apply more features for the template. In the future, there are areas for improvement. One feature that would make the process more effective would be comparing cases to more than just the case above and below it. Another feature would be comparing procedure codes to the additional procedure codes listed or creating individual cases for each additional procedure code.

EXPECTED IMPACT

In performing this project, the team utilized qualitative and quantitative methods to analyze the audit process. As a result, the recommendations provided maximized quality and efficiency of the billing audit process that occurs at MARCQI. Specifically, these recommendations will result in:

● Shorter billing audit times leading to reduced labor costs● Standardized processes● Complete and accurate MARCQI dataset

16

REFERENCES

[1] R.E. Hughes, B.R. Hallstrom, H. Zheng, J. Kabara, M. Cowen, R. Igrisan, S. McPhail and A. Richmond, “Michigan Arthroplasty Registry Collaborative Quality Initiative (MARCQI) Report: 2012-2016,” University of Michigan: Ann Arbor. Memo. 2017.

[2] S. Cheusheva, “How to compare two columns in Excel for matches and differences,” Ablebits, August 26, 2015. [Online]. Available: https://www.ablebits.com/office-addins-blog/2015/08/26/excel-compare-two-columns-matches-differences/

[3] S. Cheusheva, “How to compare two Excel files for differences,” Ablebits, February 25, 2016. [Online]. Available: https://www.ablebits.com/office-addins-blog/2016/02/25/compare-two-excel-files-sheets/

[4] “Remove duplicates in Excel,” Excel Functions. Accessed October 23, 2017. [Online]. Available: http://www.excelfunctions.net/remove-duplicates-excel.html

17

APPENDIX

Figure 1: Survey data showing how hours for a billing audit increase with the number of cases in a site.

Sample size: 4; Source: IOE 481 Team; Collection Period: November 2017

18

Figure 2: Survey data showing what percentage of errors are found in each field. Sample size: 4; Source: IOE 481 Team; Collection Period: November 2017

Figure 3. Screenshot showing all the fields present in the Excel template developed by the team for improving the billing audit process.

19

Billing AuditUser Manual

For

20

Table of ContentsINTRODUCTION 2

TEMPLATE DESCRIPTION 2

CONDUCTING THE BILLING AUDIT 2

Preparing the Data 2

Arranging the Data 3

Formatting the Data 3

Importing the Data (Two Step) 4

Importing the Data (One Step) 6

Sorting the Data 7

Processing the Data 8

Understanding the Columns 8

Checking the Data 8

Procedure Code Inconsistencies 8

Possibly Missing Cases 9

Preparing for CDAs 9

21

INTRODUCTION

MARCQI’s billing audit’s purpose is to create an accurate and complete dataset of hip and knee surgeries that meet MARCQI’s criteria. These audits are performed by the auditors annually for each participating hospital. The billing audit is used to identify inconsistencies between the dataset from the billing departments of participating hospitals and the MARCQI dataset from the CDAs at these hospitals. This is done by comparing the patient’s last name, first name, date of birth, surgery date, procedure code, admit date, and discharge date. Each case may have an identical match between the two datasets, inconsistencies within the seven patient fields between the two datasets, or may be missing from one of the datasets. This template is used to identify this for all the cases. The auditors then contact the CDAs with the inconsistencies to follow up. After the CDAs address the inconsistencies, the MARCQI dataset is complete and accurate.

TEMPLATE DESCRIPTION

The template is used to automatically identify inconsistencies between the billing department dataset and the MARCQI dataset. This is done by identifying how many identical cases there are between the two sets and then focusing on only the cases without a match. After sorting the data by how many cases there are and last name, it compares each case to the case above and below it to see if it is an approximate match. An approximate match means that there are five out of the seven fields that match then draws attention to the fields with inconsistencies by highlighting the mismatched data cells. If a case does not have an approximate match with a case above or below it, it is considering to be missing and identifies which dataset it is missing from depending on the source of the case. In this situation, the case may actually be missing from the other dataset, may have additional procedure codes that can explain the missing case, or may have more than two inconsistencies with the actual matching case. This is done through the utilization of IF formulas, COUNTIF formulas, and conditional formatting.

CONDUCTING THE BILLING AUDIT

Preparing the DataBefore combining the data for the template to process it, the data must be in proper formats and order to transfer correctly. This includes preparing it before copying it into the template tabs. This is to be repeated for both the billing department dataset and MARCQI dataset.

Arranging the Data1. Check if the order of the columns matches the order shown below. If the data matches,

move on to the next section, otherwise continue to Step 2. The column headings do not

need to have the same name, but the order of data should be the same. If there is no data

for a certain column, insert a column as a place saver.

22

2. To remove unnecessary columns:

i. To delete the column: Click above the column, this should select the entire

column. Right-click and select “Delete.”

ii. To move the column: Click above the column, this should select the entire

column. Right-click and select “Cut.” Then select the column where you want to

place the old column. Right-click and select “Insert.”

3. To rearrange necessary column:

i. Click above the column, this should select the entire column. Right-click and

select “Cut.” Then select the column where you want to place the old column.

Right-click and select “Insert Cut Cells.” This will insert the cut column in the

new spot and shift everything else to the right.

4. To add necessary columns:

i. Click above the column where the necessary column should be, this should select

the entire column. Right-click and select “Insert.” This will insert a new column

to the left of the selected column.

Formatting the Data1. Check if date columns include only the date or date and time. If it includes date and time,

there are two options.

i. Select the column. Change cell format to Date. If this does not work try the other

option.

ii. Contact the hospital billing department and insist on a new file without the times.

2. Adjust Procedure Code to only include the first 5 characters of the code:

i. Click above the column, this should select the entire column.

ii. Go to the “Data” tab on the top.

iii. Click on “Text to Columns.”

iv. Choose the “Fixed width” option. Click on the Next button.

v. Create a break line at the desired width by clicking at the desired position. Double

click on any unwanted lines. Click on the Next button.

23

vi. Choose the “General” column data format option for the wanted section. Select

any unwanted sections and choose the “Do not import column” option for those.

vii. Click on the Finish button. Note: the column heading will also get cut, do not

worry about that.

Importing the Data (Two Step)1. Select the first row of data in source file.

2. Press CTRL+SHIFT+DOWN ARROW. This will select all of the data until the end of

the dataset.

3. Press CTRL+C or right-click and select “Copy” to copy.

4. Open the template tab corresponding to the source of the data.

5. Select the first empty cell under the Last Name column.

24

6. Press CTRL+V or right-click and select “Paste” to paste.

7. Select the cell with the inputted source.

8. Double-click the box in the bottom right of the cell. This will fill the cells below it with

the same value until the last dataset value.

9. Select the new complete first row of data.

10. Press CTRL+SHIFT+DOWN ARROW.

11. Press CTRL+C or right-click and select “Copy” to copy.

12. Open the “Combined” tab.

13. Select the first empty cell under the Source column.

14. Press CTRL+V or right-click and select “Paste” to paste.

15. Repeat steps 1 through 12 with the other dataset.

16. Select a cell in the Source column.

17. Press CTRL+DOWN ARROW. This will take you to the last full cell in that column.

18. Select the next empty cell in the Source column.

19. Press CTRL+V or right-click and select “Paste” to paste. Both datasets should now be

imported in the combined tab.

25

Importing the Data (One Step)1. Select the first row of data in source file.

2. Press CTRL+SHIFT+DOWN ARROW. This will select all of the data until the end of

the dataset.

3. Press CTRL+C or right-click and select “Copy” to copy.

4. Open the template tab for combined data.

5. Select the first empty cell under the Last Name column.

6. Press CTRL+V or right-click and select “Paste” to paste.

7. Select the first empty cell in the source column. Type either “Site” or “MARCQI” which

matches the source of the data.

8. Double-click the box in the bottom right of the cell. This will fill the cells below it with

the same value until the last dataset value.

9. Repeat steps 1 through 4 with the other dataset.

10. Select a cell in the Source column.

11. Press CTRL+DOWN ARROW. This will take you to the last full cell in that column.

26

12. Select the next empty cell in the Last Name column.

13. Press CTRL+V or right-click and select “Paste” to paste.

14. Return to the last full cell in the Source column by selecting a cell in the column then

pressing CTRL+UP ARROW.

15. Select the next empty cell in the source column. Type either “Site” or “MARCQI” which

matches the source of the data.

16. Double-click the box in the bottom right of the cell. This will fill the cells below it with

the same value until the last dataset value. Both datasets should now be imported in the

combined tab.

Sorting the Data1. Go to the “Home” tab on the top.

2. Click on the Sort & Filter button.

3. Select “Custom Sort...”

4. Press OK. The custom sort levels should be already inputted. If they are not, the image

below shows what it should look like.

5. Click on the Sort & Filter button again.

6. Select “Filter.”

7. Filter the “Number of Cases” column to only show those with the value 1, ones with no

match.

27

Processing the DataUnderstanding the Columns

➔ Number of Cases: this column counts how many identical cases there are with the same

seven fields. You will only be focusing on cases with a value of 1 in this column.

➔ Approximate Match?: this column compares each case to the case above and below it. If

the case matches in at least 5 out of the 7 fields and is from two different sources, it

outputs a “Yes” and there is considered to be an approximate match. If it does not have at

least 5 matching fields or is from the same source to the case above or below it, it outputs

a “No.”

➔ Missing: this calls out which cases are possibly missing and uses the source of the case to

output which source it may be missing from. These should still be double checked for

additional codes.

➔ There are other columns that are hidden which contain formulas that help these outputs.

Checking the DataAfter sorting and filtering the data, check to make sure everything is correct. This will mostly related to the cases with procedure code issues and those that are possible missing.

Procedure Code Inconsistencies 1. Check if the MARCQI case of the approximate match has Additional Codes.

2. If there are additional codes and it matches, change both cases Number of Cases to 2.

This will filter them out as cases that are not concerning.

3. If there are no additional codes, check the MARCQI dataset, either through the other data

file or from the MARCQI tab if you performed the two step import.

i. Sort the data by Last Name, if it isn’t already filtered by last name.

ii. Filter the data to only show cases with Additional Codes. This is done by Filtering

and deselecting the (Blanks) box.

iii. Either manually look for the patient name or press CTRL+F to perform the search

function and search for the last name.

iv. If there are additional codes and it matches, change the case’s Number of Cases to

2.

v. If there are no additional codes, there may be an actual inconsistency with

Procedure Code.

28

Possibly Missing Cases1. Check if there is a similar case above or below it to check the additional codes if

improperly shown as missing.

2. If there are additional codes and it matches, change the case Number of Cases to 2. This

will filter them out as cases that are not concerning.

3. If there are no additional codes, check the MARCQI dataset, either through the other data

file or from the MARCQI tab if you performed the two step import.

i. Sort the data by Last Name, if it isn’t already filtered by last name.

ii. Filter the data to only show cases with Additional Codes. This is done by Filtering

and deselecting the (Blanks) box.

iii. Either manually look for the patient name or press CTRL+F to perform the search

function and search for the last name.

iv. If there are additional codes and it matches, change the case’s Number of Cases to

2.

4. If there are no additional codes within the combined tab or the MARCQI set, the case is

missing.

Preparing for CDAs1. Follow the directions to sort and filter the data under the “Sorting the Data” section. This

will only show the cases with 1 Number of Copies, meaning only those which are still

issues.

2. Select the top row with column headings. Select the columns you would like to send the

CDA.

3. Press CTRL+SHIFT+DOWN ARROW. This will select all of the data until the end of

the dataset.

4. Press CTRL+C or right-click and select “Copy” to copy.

5. Open a new Excel file which will be sent to the CDAs.

6. Press CTRL+V or right-click and select “Paste” to paste.

29