This is a sound textbook for Information Technology and MIS undergraduate students, and MBA graduate students and all professionals looking to grasp a fundamental understanding of information quality. The authors performed an extensive literature search to determine the Fundamental Topics of Data Quality in Information Systems. They reviewed these topics via a survey of data quality experts at the International Conference on Information Quality held at MIT. The concept of data quality is assuming increased importance. Poor data quality affects operational, tactical and strategic decision-making, and yet error rates of up to 70%, with 30% typical are found in practice (Redman). Data that is deficient leads to misinformed people, who in turn make bad decisions. Poor quality data impedes activities such as re-engineering business processes and implementing business strategies. Poor data quality has contributed to major disasters in the federal government, NASA, Information Systems, Federal Bureau of Investigation, and most busineses. The diverse uses of data and the increased sharing of data that has arisen as a result of the widespread introduction of data warehouses have exacerbated deficiencies with the quality of data (Ballou). In addition, up to half the cost of creating a data warehouse is attributable to poor data quality. The management of data quality so as to ensure the quality of information products is examined in Wang. The purpose of this book is to alert our IT-MIS-Business professionals to the pervasiveness and criticality of data quality problems. The secondary agenda is to begin to arm the students with approaches and the commitment to overcome these problems. The current authors have a combined list of over 200 published papers on data and information quality.

Introduction to INFORMATION QUALITY

By Craig Fisher Eitel Lauría Shobha Chengalur-Smith Richard Wang

AuthorHouse

Copyright © 2011 Craig Fisher, Ph.D., MARIST College, Eitel Lauría, Ph.D., MARIST College, Shobha Chengalur-Smith, Ph.D., SUNY Albany, and Richard Wang, Ph.D., MIT
All right reserved.
ISBN: 978-1-4685-3028-5

Contents

Preface............................................................................xiChapter 1..........................................................................1Information Systems and Impacts of Poor-Quality Data...............................1Chapter 2..........................................................................27Total Quality Management...........................................................27Chapter 3..........................................................................49The Multiple Dimensions of Information Quality.....................................49Chapter 4..........................................................................79Information Products and Total Data Quality Management.............................79Chapter 5..........................................................................108Statistics.........................................................................108Chapter 6..........................................................................153Controlling Information Product Quality............................................153Chapter 7..........................................................................178Measuring and Tools for Assessing Data and Information Quality.....................178Chapter 8..........................................................................215Decision Support and Business Intelligence.........................................215

Chapter One

Information Systems and Impacts of Poor-Quality Data

Information systems (IS) help organizations get, process, store, manage and report information so that everyone can do their jobs as efficiently and effectively as possible. IS organizations exist in every large corporation, government entity, and educational institution in the United States. Although these systems have had many names—electronic data processing, data processing, information systems, information services, information processing, data centers, information technology—the key words are data and information. The purpose of IS has always been to provide data and information to those who need it.

However, even after 40 to 50 years of IS experience, organizations are often less—rather than more—satisfied with the information they are getting. Experience in developing IS includes many years of improving techniques and processes to assure quality results. Some techniques are: structured programming, data validity checks, check digits, test and conversion strategies, inspections, and computer-assisted software engineering tools. Processes include application development methodologies often modeled after engineering development processes, joint application development, various levels of integration tests and user acceptance tests, rapid application development (RAD) methodology, use of various models such as data flow diagrams, and entity relationship diagrams. Object-oriented approaches, including a unified modeling language, are now being used. These can be found in modern textbooks on systems analysis and design or management information systems.

Even with all of the improvements in processes and techniques, complaints of poor-quality data and bad information are frequent. Many of the processes and procedures mentioned in the previous paragraph take time and are full of errors. As a result, people bypass what seems to them a bureaucratic, time-consuming process. For example, the rapid development processes called RAD allow the developer to skip many of the bureaucratic steps. Another factor that contributes to poor-quality data is end user computing (EUC). Although one of the primary goals of modern Windows-type facilities is to make it easy for end users to quickly build systems, this does not guarantee quality. Are organizations relying too much on EUC? Compare the training of an end user with an IS student. The IS student takes at least one semester of database development in which normalization and referential integrity are covered in detail. Many end users are not even aware of these data quality (DQ) techniques.

End-user-created databases are generally not easily integrated into the corporate IS. Data definitions, the abbreviations and names of the data, the system architecture, the software, and the interfaces involved vary so much from user to user and department to department, that any attempt to create an integrated database or data warehouse seldom ends satisfactorily. The existence of good development procedures is not effective if people can simply bypass them, whether through ignorance or design. An overall approach which includes treating information as products, modeling information management systems, building information product maps, and applying total quality management (TQM) approaches, will be discussed in later chapters.

It could be that modern day information systems are so complex and cumbersome that accidents and errors cannot be avoided. For example, Fleet Bank launched a $38-million customer relations management project to pull together customer information from 66 source systems. The project failed in less than three years because it was too difficult and time consuming to understand, reconcile, and integrate data from so many sources.

Home Depot's IS supports over a million products in approximately 2,300 stores with many databases. Home Depot is attempting to tie together thousands of its software applications, stores, and systems. Information systems consist of people, procedures, databases, systems hardware and software, and computer application programs. The complexity of the systems, and of its components, may lead to DQ problems. Clearly, a very comprehensive approach to quality is necessary, and this will be covered in the remaining chapters.

Information Systems

This chapter is focused on examples of problems and their effects on people, organizations, the Federal Government, and society in general. We will summarize the effects after we distinguish between data and information.

Data Versus Information

There is an old saying that "one person's data is another person's information." Several textbooks distinguish between data and information. Data is defined as isolated facts devoid of meaning. Information is defined as processed data that has meaning because of relationships established with other data. For example, if a retailer sold 10 dresses, that might be an interesting fact but doesn't have much meaning by itself. However, if that retailer sold 100 dresses yesterday and 10 today, then more meaning can be derived. Meaning is enhanced by establishing relationships with other data. Additional data might include comparisons of this retailer's sales to those of a competitor; or comparisons of sales over intervals, such as sales per month, per quarter, or per year. It might also be important to know how many dresses were sold during a sales promotion or during an employee incentive program to sell, for example, 100 dresses. Another view might be the rate of sales. If sales were very high—10 dresses in 30 minutes—that might make the manager take action that would not be taken if the rate of sales was very low. Depending...