Executive Summary

Introduction

This report presents the results of the Federal Highway Administration (FHWA) evaluation of the Idaho Transportation Department’s (ITD) integration of their Road-Weather Information System (RWIS). The ITD RWIS project received approximately $396,400 in funding through the FHWA Intelligent Transportation System (ITS) Integration Program.

The ITD RWIS project was selected for evaluation because it held significant potential to generate lessons learned and other findings that would be useful to other RWIS operators and those planning for such systems. In particular, this project offered an opportunity to study the potential benefits and impacts on public travel and state highway maintenance practice associated with the integration of ITD RWIS data with non-transportation weather data, and the improved information access. The findings of this evaluation could provide useful insights to USDOT’s ongoing Clarus initiative which intends to “demonstrate an integrated road weather observation network and data management system to reduce the effect of adverse weather on all road users and operators.” Clarus—the Nationwide Surface Transportation Weather Observing and Forecasting System—aims to leverage state investments in Environmental Sensor Stations (ESS) to minimize weather impacts on safety, mobility and productivity.

The RWIS integration project was intended to solve the following four major problems faced by the ITD:

Lack of a Consolidated, Internet-Accessible ITD RWIS User Interface – Data from two different brands of ITD RWIS were accessed separately via two different proprietary user interfaces. Data from the two different systems were in different formats. Many ITD field maintenance sheds did not have convenient access to the data (dial-up connections were used).
Inability to Integrate Various Brands of Environmental Sensor Stations (ESS) into a Single RWIS – ITD operated two separate RWIS, one for each brand of ESS. (ESS, which are also known as weather stations and RWIS stations, are the field components of an RWIS.) It was not possible to integrate information from the different brands of ESS and therefore ITD felt confined to one of the two existing ESS brands when expanding coverage.
Lack of Public Access to RWIS Information – Public information on weather and road conditions was limited to the ITD “Road Report” phone line and website (http://164.165.237.41/apps/roadreport/), which did not include RWIS information.
Inadequate ESS Coverage – ITD operated about 34 RWIS sensors statewide and there were many critical gaps in coverage.
The RWIS Integration project addressed these problems by establishing a single RWIS user interface and data format, integrating data from existing brands of ESS and weather information from other sources (such as the National Weather Service) and capable of incorporating data from additional ESS; establishing a new public RWIS webpage as part of the existing ITD Road Report traveler information website; and integrating, via a weather data sharing consortium, data from a large number of ESS operated by other organizations within Idaho and in bordering states. In addition to solving the four major problems noted above, the ITD Project Manager intended that the improved access to data would increase the utilization of RWIS in winter maintenance decision-making on the part of ITD maintenance personnel.

Evaluation Process

The evaluation of the ITD RWIS integration project employed a before-and-after study approach. The before (pre-deployment) data was collected and analyzed in 2001 and 2002. The RWIS project became operational in November 2002. Collection of after (post-deployment) data focused on the second winter season of operation from October 2003 through May 2004. The post-deployment evaluation was delayed one year in order to allow ITD maintenance personnel to be trained and to gain experiences with the integrated system.

The evaluation included the following analyses which evaluated each of the major components of the ITD RWIS Integration project:

Interviews with ITD maintenance personnel and other ITD personnel associated with the project.
Website utilization by travelers and ITD maintenance personnel.
Accident and winter maintenance resource utilization impacts which could result from any improvements in winter maintenance practices associated with the project.
Survey of public website users.
Interviews with commercial vehicle operators

Findings

Evaluation findings are summarized below in three major areas: Road-Weather Integrated Data System (RWIDS) as an ITD maintenance information resource; RWIDS as a traveler information resource; and RWIDS as a data integration platform); etc.

RWIDS as an ITD Maintenance Information Resource

RWIDS represents an important evolutionary, but not a revolutionary, advance for ITD winter maintenance decision-making. The RWIDS project has been very successful in its objective of making available a much more convenient, consolidated source of ITD RWIS data, and a vast wealth of additional weather data, to ITD maintenance personnel. Personnel with good access to the Internet in areas where RWIDS coverage is good find the site very useful. However, even the RWIDS proponents view it as “just another tool in the tool box” that has incrementally enhanced their decision making, rather than as a revolutionary improvement.
The usefulness of the current version of RWIDS varies widely for ITD maintenance personnel throughout the state, and can be enhanced. The system is much less useful in areas with few or no ESS, and there are still significant portions of the state with no stations. Other factors that impact the utility of RWIDS include lack of high-speed Internet access in many areas; scarcity of maintenance shed personal computers in some areas; little to no prior experience using Internet weather information and particularly ESS data in maintenance decision-making. To some extent this last factor is related to the variation across Foreman Areas and Districts in overall attitudes toward technology and new practices.
ITD maintenance personnel are generally comfortable synthesizing discrete weather data to draw conclusions and make decisions. Most ITD maintenance personnel that were interviewed expressed little interest in expert systems for maintenance decision support. They don’t feel that the technology is sufficiently sophisticated to effectively deal with what they perceive as a very complex and changing decision-making context. They also expressed concerns that ESS coverage and reliability are not sufficient to support such systems now or in the foreseeable future. It was clear that the interviewees tend to associate more synthesized information and suggested actions with an “automated” system. Therefore, their distrust of decision support does not necessarily imply that they would be likely to reject more synthesized information. In fact, several maintenance personnel cited the synthesized weather summaries prepared by NWS employees as among the most useful type of weather information to help coordinate shift changes.
Most ITD maintenance personnel are interested in a wide range of data, but are most interested in the same basic information popular with the public: cameras and basic current and forecasted weather information.
RWIDS might not generate quantifiable improvements in roadway safety or the utilization of ITD winter maintenance resources. There was no statistically significant change in statewide winter weather-related accident rates during the one-year period after implementation of RWIDS. The absence of measurable, traceable improvements in safety is not unexpected given the many exogenous factors; the modest, incremental improvement in decision-making associated with RWIDS; the fact that not all maintenance personnel utilize RWIDS; and the inherently conservative and relatively inelasticnature of many ITS winter maintenance resource commitment decisions.
Although demonstrable safety improvements are lacking, travelers perceive that their safety is enhanced through the use of the RWIDS webpage. Eighty percent of on-line survey respondents who used RWIDS agreed that the information helped them better prepare for road-weather conditions and 76 percent of the respondents indicated that the information helped them drive more carefully.

RWIDS as a Traveler Information Resource

The addition of the RWIDS webpage accounts for a sizable percentage of the overall increase in Road Report website utilization. Utilization of the Road Report website in general has risen consistently and dramatically in the year following implementation of the RWIDS webpage (169%). About half (47%) of the new user sessions included RWIDS viewing. Further, 85 percent of all sessions that included RWIDS viewing featured only viewing of RWIDS content. A full 25 percent of post-RWIDS Road Report sessions feature viewing of only RWIDS information.
The traditional Road Report information, fed by ITD maintenance foremen reports, is not “replaced” by the RWIDS website. Although “RWIDS-only” user sessions account for a significant portion of the overall growth in Road Report website utilization (about 47%), most of the growth, and most of the total current utilization is for traditional Road Report information (the road closure and conditions information provided by maintenance foremen).
RWIDS has probably not reached its full potential as a public information resource, since it has not been marketed or integrated. The ITD has not marketed the RWIDS webpage. Although significant levels of usage have occurred as result of users happening across the new webpage, and presumably through some word-of-mouth, it seems likely that marketing could significantly increase usage. Also, since it is clear that users still value—and separately consult—the traditional Road Report closure and condition information, that overall Road Report usage might be further enhanced if the RWIDS information was integrated with the traditional information.
Public RWIDS users, including commercial vehicle operators, are most interested in camera images and basic current and forecasted weather information.

RWIDS as a Data Integration Platform

RWIDS has successfully leveraged a wealth of data from other organization’s ESS. The RWIDS webpage provides access to approximately 120 ESS operated by other organizations, including other state department of transportation ESS. Most maintenance personnel find these other stations useful, although they prefer ITD operated ESS.
Other organization’s ESS supplement but do not necessarily replace ITD ESS. Data from other organizations’ stations provide a low-cost means to greatly enhance data coverage. However, even in the case of Idaho which acquired access to a large number of such stations, such stations do not necessarily address all, or even the most critical (hot spot), data coverage gaps. Further, although they provide some value, other organizations’ stations do not provide a standard, full compliment of information. For example, very few non-transportation department stations provide pavement data. Non-transportation agency stations are typically not ideally sited to support roadway maintenance-related data collection. Finally, other organizations may not maintain their stations at levels needed to support public traveler or highway maintenance information needs.
RWIDS has allowed ITD to pursue more cost-competitive ESS procurements. Since completion of the RWIDS data platform, two sets of new ITD ESS have been successfully integrated into the system. The project has allowed the ITD to select from among a wider, more cost-competitive range of offerings and thereby significantly reducing the costs of new stations. The ITD estimates that the cost to deploy four sites using one of the two new brands of stations is about 12 percent lower than the bid submitted by the primary “incumbent” RWIS vendor. In that particular procurement, the flexibility provided by RWIDS resulted in a savings of about $30,000, or about $7,500 per site.

Recommendations

Continue to fill gaps in ESS coverage. Other organizations’ ESS do not include pavement data and are often not positioned to effectively meet ITD RWIS needs. It is recommended that ITD pursue plans to implement additional ITD ESS and to evaluate adding pavement sensors to key ESS operated by other organizations.
Improve the reliability of ESS. A number of maintenance employees identified reliability as a concern and significant limitation on RWIS benefits. It is recommended that the ITD continue to work to improve the reliability of stations, including adding their own ESS in key locations where other organizations’ ESS are not sufficiently reliable.
“Level the playing field” statewide with respect to personal computers and Internet access. Even more fundamental to the use and usefulness of RWIDS than ESS coverage is that maintenance sheds have personal computers and Internet access; preferably high-speed access. It is strongly recommended that ongoing ITD efforts in this area continue and be accelerated if possible.
Continue, and accelerate if possible, implementation of personal computers and high-speed Internet access at all maintenance sheds.
Provide additional, focused training to ITD maintenance personnel. Most maintenance personnel desired more training, especially hands-on training, with computers at each training station. It is recommended that the follow-up training provide instruction on overall information gathering strategies and use of various types of information, including the various RWIDS data, in support of specific types of winter maintenance decisions as well as the use and benefits of road weather information to make proactive decisions about treatments (such as anti-icing).
Market RWIDS to the public.
Consider more fully integrating RWIDS and the remainder of the Road Report. The general public and commercial vehicle operators are interested in the “traditional” Road Report closure and conditions information as well as information on the RWIDS webpage. It is recommended that ITD consider integrating these information types, including possibly using RWIDS data to enhance the traditional roadway-specific Road Report advisories. Such integration could occur as part of the planned ITD implementation of a comprehensive traveler information system.
Develop system performance monitoring capabilities and monitor the utilization and effectiveness of RWIDS. Currently, ITD does not have effective mechanisms for monitoring the utilization of the RWIDS webpage by either the public or ITD maintenance personnel. In fact, limitations in this regard, such as the inability to clearly distinguish all ITD RWIDS user sessions, impacted this evaluation. In order to support a continuous quality improvement process it is recommended that the ITD implement mechanisms that will allow for tracking of RWIDS utilization and that they routinely analyze that data.
Monitor broader national developments in road weather information sharing and position ITD to capitalize on advances. The ITD should be aware of FHWA activities to promote technology transfer of the Federal MDSS prototype, which integrated road weather and resource information to provide route-specific treatment recommendations. In addition, the USDOT Clarus Initiative is one of the most important activities that ITD should be aware of, and if possible, participate in. That initiative is focusing on integration of all weather and pavement condition observations from transportation agencies and shares the same philosophy in data integration with this ITD earmark project.
Continue to consider opportunities to further synthesize road weather information for winter maintenance personnel, including a Maintenance Decision Support System.

Home | Table of Contents | 1.0 Introduction and Background