Traffic Data Quality Measurement

Final Report

To
Office of Highway Policy Information
Federal Highway Administration
U.S. Department of Transportation
Washington, D.C.

In Association with
Cambridge Systematics Inc.
Texas Transportation Institute

Battelle Memorial Institute

September 15, 2004

• Disclaimer

TABLE OF CONTENTS

• ACKNOWLEDGMENTS
• LIST OF ACRONYMS

• EXECUTIVE SUMMARY
  • Introduction
  • Framework for Data Quality Measurement
  • Guidelines for Data Quality Measurement
  • Conclusions and Recommendations

• 1.0 INTRODUCTION
  • 1.1 Background
  • 1.2 Project Objectives and Scope
  • 1.3 Organization of Report

• 2.0 RESEARCH APPROACH
  • 2.1 Define Traffic Data Quality Metrics
  • 2.2 Prepare a Framework for Assessing Traffic Data Quality
  • 2.3 Prepare Guidance on Data Quality Assessment
  • 2.4 Beta Testing and Review of Guidelines

• 3.0 FRAMEWORK FOR DATA QUALITY MEASURMENT
  • 3.1 Define Data Consumers
  • 3.2 Define Quality Measures
  • 3.3 Establish Acceptable Data Quality Targets
  • 3.4 Calculate Data Quality Measures
    • 3.4.1 Accuracy
    • 3.4.2 Completeness
    • 3.4.3 Validity
    • 3.4.4 Timeliness
    • 3.4.5 Coverage
    • 3.4.6 Accessibility
    • 3.4.7 Composite Data Quality Measures
    • 3.4.8 Level of Detail for Data Quality Measures
  • 3.5 Identify Data Quality Deficiencies
  • 3.6 Assign Responsibility and Automate Reporting
  • 3.7 Perform Periodic Assessment
  • 3.8 Case Studies

• 4.0 GUIDELINES FOR DATA QUALITY MEASUREMENT
  • 4.1 Introduction
  • 4.2 Establishing Acceptable Data Quality Targets
    • 4.2.1 Travel Demand Modeling
    • 4.2.2 Air Quality Conformity Analysis
    • 4.2.3 Congestion Management Systems
    • 4.2.4 Highway Performance Monitoring System (HPMS)
    • 4.2.5 Permanent Count Station Reports
    • 4.2.6 Safety Studies
    • 4.2.7 Traffic Simulation
    • 4.2.8 Program and Technology Evaluation
    • 4.2.9 Ramp Signal Coordination
    • 4.2.10 Traveler Information
    • 4.2.11 Pavement Management Systems
  • 4.3 Quantifying Data Quality Targets
  • 4.4 Level of Effort Required for Traffic Data Quality Assessment
  • 4.5 Specifications and Procedures for Using Metadata for Reporting Data Quality
  • 4.6 Guidelines for Data Sharing Agreements
    • 4.6.1 Review of Data Sharing Agreements
    • 4.6.2 Data Quality Provisions in Data Sharing Agreements
    • 4.6.3 Model Data Quality Sections of Data Quality Agreements

• 5.0 BETA TESTING OF FRAMEWORK AND REVIEW OF GUIDELINES
  • 5.1 Framework Review Comments
  • 5.2 Guidelines Review Comments
  • 5.3 Other Comments

• 6.0 CONCLUSIONS AND RECOMMENDATIONS

• 7.0 REFERENCES

List of Appendices

• Appendix A: AUSTIN CASE STUDY
• Appendix B: PITTSBURGH, PENNSYLVANIA CASE STUDY
• Appendix C: OHIO DEPARTMENT OF TRANSPORTATION CASE STUDY

List of Tables

• Table ES-1. Types of Data Consumers and Applications
• Table ES-2. Data Quality Targets
• Table ES-3. Level of Effort Estimates for Traffic Data Quality Assessment and Reporting
• Table ES-4. Standards for Data Transfer Agreements
• Table 3-1. Types of Data Consumers and Applications
• Table 3-2. Illustration of Composite Data Quality
• Table 3-3. Traffic Data Quality Summary
• Table 3-4. Case Studies
• Table 4.1. Draft Data Quality Requirements for Planning, Engineering, and Operations Applications
• Table 4.2. Level of Effort Estimates for Traffic Data Quality Assessment and Reporting
• Table 4.3. Standards for Data Transfer Agreements
• Table A.1. Completeness Statistics for Original Source Data
• Table A.2. Completeness Statistics for Data Archive
• Table A.3. Completeness (Availability) Statistics for Traveler Information
• Table A.4 Validity Statistics for Original Source Data
• Table A.5. Validity Statistics for Archive Database
• Table A.6. Validity Statistics for Traveler Information
• Table A.7. Traffic Data Quality "Scorecard" for Austin Case Study
• Table B.1. Completeness Statistics for I-Minute Original Source Datav
• Table B.2. Completeness Statistics for 5-Minute Summary Data in Data Warehouse
• Table B.3. Completeness (Availability) Statistics for Key Route Travel Times
• Table B.4. Validity Statistics for I-Minute Original Source Data
• Table B.5. Validity Statistics for 5-Minute Summary Data in Data Warehouse
• Table B.6. Validity Statistics for Key Route Travel Times
• Table B.7. Traffic Data Quality Summary for Pittsburgh Case Study
• Table C.1. Comparison of Manual and ATR Counts for Vehicle Classes and Volumes
• Table C.2. WIM Calibration Report from DOT
• Table C.3. Accuracy Tests of Five Detectors from a Vendor
• Table C.4. Summary of Completeness Measures
• Table C.5. Summarizes the Validity Measures
• Table C.6. Coverage for Continuous Counts
• Table C.7. Coverage for Short-Counts
• Table C.8. Data Quality Summary

List of Figures

• Figure ES-1. Structure of Data Quality Assessment Framework
• Figure 3-1. Structure of Framework
• Figure 3-2. Illustration of Completeness and Validity Measures
• Figure 4.1. Example of Data Quality Documentation Using ISO 1915
• Figure 4.1. (contd.) Example of Data Quality Documentation Using ISO 1915
• Figure 4.2. Example Language for Specifying Minimum Data Quality Criteria in a Data Sharing Agreement
• Figure A.1. Simplified Austin Case Study Mapped to National ITS Architecture
• Figure A.2. Data Flows and Data Consumers in Austin Case Study
• Figure A.3. Sample of Original Source Data for Austin
• Figure A.4. Accuracy of Speed Values in Original Source Data
• Figure A.5. Accuracy of Hourly Traffic Volumes in Archive Database
• Figure A 6. Accuracy of Route Travel Time Values in Traveler Information
• Figure B.1. Data Flows and Data Consumers in Pittsburgh Case Study
• Figure B.2. Accuracy of Hourly Traffic Volumes in Archive Database
• Figure B.3. Accuracy of Route Travel Time Values in Traveler Information
• Figure C.1. Data Flows and Consumers – Ohio Case Study
• Figure C.2. Accuracy Test of Detector #3 (Difference between manual and detector counts)