1 Introduction
Working with stakeholders across the country, the Federal Highway Administration’s (FHWA) Road Weather Management Program (RWMP) has undertaken a study to identify performance measures that can be used to evaluate the success of its products and activities in achieving key program goals. This project identified performance metrics that convey the progress of FHWA’s RWMP activities in addressing the program objectives established under the Safe, Accountable, Flexible, Efficient, Transportation Equity Act – A Legacy for Users (SAFETEA-LU) Section 5308, Road Weather Research and Development Program, within Subtitle C – Intelligent Transportation System Research. These SAFETEA-LU goals include:
- Maximize the use of available road weather information and technologies.
- Expand road weather research and development efforts to enhance roadway safety, capacity, and efficiency while minimizing environmental impacts.
- Promote technology transfer of effective road weather scientific and technological advances.
The challenge lies in establishing meaningful, understandable, and practical measures of performance that can help evaluate the social, scientific, and organizational benefits expressed in the SAFETEA-LU goals and achieved through RWMP products and activities.
This report summarizes the work undertaken and the results generated to identify a set of metrics appropriate for implementation based on 1) their relevance to the RWMP, 2) endorsement by the stakeholders, 3) availability of data, and 4) resources needed to support implementation.
The high level SAFETEA-LU goals were identified by the U.S. Congress in 2005 to help focus the research and development activities of the RWMP. The RWMP has developed a roadmap to define and guide their current and future activities. Some of the important RWMP initiatives underway include the following:
- Environmental Sensor Station (ESS) Siting Guidelines. This project will produce and support consistent guidance for state and local agency personnel responsible for procuring, siting, operating, and maintaining weather stations along the highways.
- Maintenance Decision Support System (MDSS) and Maintenance Operations Decision Support System (MODSS). The MDSS is a multi-year program to prototype and field test advanced decision support systems for use by winter road maintenance managers in order to enhance the efficiency and cost-effectiveness associated primarily with the timing of treatments and the choice of material types and amounts to use for snow removal and ice control. MODSS is an extension of the MDSS that seeks to apply decision support technologies for weather-responsive maintenance and traffic operations throughout the year.
- Clarus. This is a multi-year nationwide initiative to assure the provision of timely and high quality weather information to transportation operators and system users. Drawing from a host of available weather data sources, Clarus integrates this information in a common format and converts it to valuable road weather information.
- Weather Responsive Traffic Management (WRTM). The RWMP is engaged in several research studies to evaluate the macroscopic and microscopic impacts of weather on traffic operations and traffic flow, and develop models and strategies for traffic management during inclement weather conditions. A concept of operations for weather responsive traffic management has been drafted that addresses road weather impact mitigation strategies, including data collection, assessment of weather impacts on roadway networks, operational strategies to control traffic during adverse weather, and future research needs. The MODSS initiative described above includes various WRTM concepts.
- Traffic Management Center (TMC) Weather Integration. This initiative seeks to identify strategies for the effective integration of weather information in the day-to-day TMC operations, and assist TMC managers and operators in developing and implementing strategies for their agencies. One of the current activities is developing a weather integration self-evaluation and planning guide.
- Vehicle Infrastructure Integration (VII). VII benefits derive from vehicles wirelessly connected with each other and with roadside equipment in order to enable a host of safety, mobility, and commercial applications. VII offers the opportunity to know much more about traffic and roadway conditions, including weather, than ever before. Vehicles equipped with VII technology will be able to anonymously send and receive information that includes travel time, environmental conditions, and weather information. This information will lead to improved traffic signal control, traveler information, transportation planning, and reduced costs for system operations. The RWMP has worked with the federal research laboratories to develop processing algorithms that enable the data collected through VII to provide relevant road weather conditions encountered by vehicles. These processing algorithms permit the aggregated VII data to be combined with other weather observations to produce an enhanced depiction of the weather status of the roadway environment.
- Fundamentals of Road Weather Management Training Course. This training course developed by FHWA and available through the National Highway Institute is designed for traffic, emergency, and maintenance managers, as well as safety engineers and others involved in highway operations and maintenance. A web-based version of the course is also now available.
Performance measurement can be defined as a process of assessing progress toward achieving predetermined goals and objectives, including information on the efficiency with which resources are transformed into goods and services (outputs), the quality of those outputs (how well they are delivered to clients and the extent to which clients are satisfied), and the results or benefits of a program activity compared to its intended purpose (outcomes), and the effectiveness of government operations in terms of their specific contributions to program goals and objectives.
Performance measurement has been conducted relatively informally across the range of road weather programs and agencies throughout the country. For this study, FHWA needed to identify relevant measures of performance for evaluating road weather management products and services. The ultimate objective is to operationalize the selected measures using appropriate data to determine RWMP accomplishments and progress in meeting the goals of SAFETEA-LU pertaining to Road Weather Research and Development.
Figure 1. Link between goals, objectives,
and performance measures
The importance of performance measurement and evaluating the effectiveness of road weather management strategies cannot be overstated because weather has significant impacts on transportation safety, mobility, and productivity. Thus, the purpose of this study is to identify meaningful, relevant, and practical output and outcome metrics that address the SAFETEA-LU objectives through FHWA’s RWMP activities. Several quantitative and qualitative performance metrics were identified and analyzed to measure program results in the following areas:
- Traffic management (including traveler information)
- Maintenance management
- Emergency management
- Transit management
- Transportation system performance
- Driver performance
The links between SAFETEA-LU goals, RWMP objectives and outputs and outcomes are illustrated in Figure 1. The identified performance measures will use indicators to describe the efficiency and effectiveness of products and services within the RWMP. Output measures tend to be quantitative indicators of operational efficiency, such as tons of materials applied to a freezing road surface or the miles of roadway plowed over a period of time. Outcome measures represent impacts or benefits achieved from program activities that tend to be more difficult to quantify, such as the reduction in travel time or travel costs that can be attributed to the use of a decision-support tool. Outputs link most directly back to the inputs, while outcomes relate more to the programmatic goals. Both output and outcome metrics serve as valuable indicators of program performance.
The performance measures study was initiated in late 2006 by FHWA using a contractor team. The study included the following elements that are described in more detail in this report:
- Conduct literature review. An extensive search and review was conducted of over 150 documents, focusing on those that address RWMP objectives associated with traveler safety, mobility, operational efficiency, information integration, and technology transfer for road weather management. While close attention was placed on surface transportation programs, performance measures were also examined in other weather programs of the federal government, such as those being used by the National Oceanic and Atmospheric Administration (NOAA) and the Federal Aviation Administration (FAA).
- Identify hypotheses and associated measures of effectiveness (MOEs). FHWA and its contractor team identified several objectives and hypotheses related to each of the three primary goals of the SAFETEA-LU Section 5308 program, and identified about 120 preliminary output and outcome measures that relate not only to those goals but also to the products and activities of the program.
- Conduct stakeholder workshop. All this information was organized in a matrix and presented to a representative group of stakeholders during a workshop sponsored by FHWA in April 2007.
- Circulate Request for Information to stakeholders. The performance measures were refined based on the recommendations from the workshop, and more than 60 outcome measures were distributed nationally for comments and recommendations through a Request for Information (RFI) survey sent out to 253 public and private sector stakeholders.
- Recommend selected measures for implementation. The measures were then reviewed and prioritized in terms of relevance to the RWMP, endorsement by the stakeholders, data availability, and ease of implementation.
