Measuring Driver Satisfaction with an Urban Arterial
Before and After Deployment of an Adaptive Timing
Signal System

PDF Version

Prepared for:

Dr. Joseph Peters, Manager of ITS Program Assessment
Intelligent Transportation Systems Joint Program Office
Federal Highway Administration
Washington, DC

Prepared by:

Margaret Petrella, Social Scientist
Jane Lappin, Program Manager
Volpe National Transportation Systems Center
Cambridge, Massachusetts

Acknowledgements

This report was funded by the ITS Joint Program Office, as part of its ongoing effort to evaluate the benefits of ITS. The Volpe Center gratefully acknowledges the guidance and support of Dr. Joseph Peters, who manages the evaluation program within the ITS Joint Program Office. The Volpe Center would also like to thank Stacey Bricka, of NuStats, and her staff, including Ramon Dickerson and Ana Arce, for their outstanding work in administering the survey. In addition, the Volpe Center would like to thank Mshadoni Smith, of Federal Highway Administration, Georgia Division, Joe Fletcher and Brook Martin, of Cobb County Department of Transportation, and Dr. Michael Patrick Hunter, of Georgia Institute of Technology, for their assistance and cooperation.

---

Table of Contents

Executive Summary
I	Introduction Background Site Selection Characteristics of the Treatment and Control Routes
II	Survey Methods Approach Target Population Sample Design    Incidence    Sampling Strategy Survey Design Pilot Test Data Collection Procedures Coordination and Scheduling of the Driver Surveys Response Rates
III	Survey Findings Driver Summary Travel Patterns Comparison of Questions on Both Driver and Background Surveys Driver Ratings Factors Related to Driver Satisfaction Assessing the Performance of the Methodology
IV	Lessons Learned Regarding Study Approach Controlling For External Explanatory Factors Target Population and Eligibility Requirements Sample Design Sample Size Survey Design Data Collection Procedures Coordination and Scheduling Supplementing Subjective Measures with Objective Measures
V	Conclusions Appendix A: The Study Brochure Appendix B: Wave 1 Recruitment Screener Appendix C: Wave 1 Survey Packet Appendix D: Panel Maintenance Letter Appendix E: Wave 2 Recruitment Screener Appendix F: Wave 2 Survey Packet

List of Tables

Table 1: Pilot Test Data Collection Targets
Table 2: Distribution of Recruited Wave 1 Drivers by Time of Day and Day of Week
Table 3: Distribution of Recruited Wave 2 Drivers by Time of Day and Day of Week
Table 4: Travel Patterns — Paces Ferry Usage
Table 5: Travel Patterns — Spring Road Usage
Table 6: Wave 1 Driver Satisfaction Ratings
Table 7: Change in Driver Satisfaction — Paces Ferry
Table 8: Change in Driver Satisfaction — Spring Road
Table 9: Change in Driver Satisfaction On Paces Ferry
(Among Drivers who said their Drive was typical)
Table 10: Factors Related to Driver Satisfaction
Table 11: Relationship Between Driver Satisfaction and Crossing the Railroad Tracks

List of Figures

Figure 1: Paces Ferry Road
Figure 2: Spring Road
Figure 3: PUMA Area Under Study
Figure 4: Selected Census Tracts
Figure 5: Trip Purpose on Paces Ferry
Figure 6: Trip Purpose on Spring Road
Figure 7: Usual vs. Actual Flexibility in Trip Making Time
Figure 8: Usual vs. Actual Concern about On-Time Arrival
Figure 9: Usual vs. Actual Companions
Figure 10: Usual vs. Actual Activities
Figure 11: Importance Ratings
Figure 12: Gap Between Importance and Wave 1 Satisfaction Ratings
Figure 13: Change in Satisfaction Among Off-Peak Drivers

---

Executive Summary

This report presents findings from a customer satisfaction study conducted in Cobb County, Georgia. The primary hypothesis of this study is that it is possible to develop customer satisfaction measures that are a reliable determinant of roadway quality. A signal system upgrade in Cobb County, Georgia offered the opportunity to test this hypothesis. The Cobb County Department of Transportation planned to instrument 15 signalized intersections on Paces Ferry Road with the Sydney Coordinated Adaptive System (SCATS). In order to test its proposed methodology for measuring customer satisfaction with roadway quality, the Volpe National Transportation Systems Center (Volpe Center) conducted a before-and-after study of drivers on the Cobb County urban arterial treated with the adaptive traffic signal system.

In addition to providing a test of the methodology, this study would also measure whether there were changes in driver satisfaction with roadway quality as a result of the ITS deployment. Cobb County had recently completed a retiming of the corridor, so the current study would provide insights on whether there is value added — from the customer's perspective — when one moves from an optimally-timed traditional signal system to an adaptive traffic signal system. In order to isolate external effects that might impact driver satisfaction ratings but which are unrelated to the traffic signal improvements, a "control" panel was also used, whereby a panel of drivers were surveyed on a comparable corridor where there was no ITS deployment. The expectation was that drivers on Paces Ferry Road (the treatment route) would be more satisfied with the roadway quality after the system was deployed compared to pre-deployment, whereas there would be no change in satisfaction on the control route.

Cobb County also assessed the performance of the new signal system through an independent evaluation conducted by the Georgia Institute of Technology. Using floating cars studies, the Georgia Institute of Technology collected objective measures for travel time, speed, and delay both before and after the signal upgrade.

Study Approach

The evaluation team determined that a pre-post study approach was the best method for measuring changes in driver satisfaction, and that the most reliable means of capturing the drivers' true experience was to have them assess roadway quality immediately following an actual driving experience on the road (both before and after the ITS deployment). The target population for this study was "regular" drivers of both the treatment and control routes. In particular, familiarity with the route was a key criterion for participation, as study participants must have some established expectations about how the road operates in order to notice a difference resulting from the ITS signal system deployment.

Three forms were developed and administered to each respondent on both the treatment and control routes: a telephone recruitment screener, background survey form and driver survey form (see Appendices B and C). The primary data collection tool was the driver survey form, which was designed to measure driver satisfaction with a variety of roadway attributes. It was also used to document drive conditions during the scheduled drive as well as other factors that may have influenced the driver ratings (such as schedule flexibility for that day).

Study Findings

On the treatment route, this study found that satisfaction ratings were similar across the two waves (pre-versus post deployment). The only statistically significant differences were increased satisfaction with Lane Width and Roadside Landscaping. The latter can easily be explained by the seasonal variation in when the interviews were conducted; whereas wave 1 was administered in the late fall, wave 2 was administered in the spring, when the landscaping was more attractive. On the control route, drivers also registered increased satisfaction with Roadside Landscaping, but all other roadway attributes were rated similarly across the two waves (as originally hypothesized).

For wave 1, Road Pavement Quality, Pavement Marking Quality, and Lane Width received the highest ratings among Paces Ferry drivers. The Number of Times Stopped by Red Lights, Amount of Time at Red Lights, and Driving Behavior of Others received the lowest ratings. While there were some differences between the Paces Ferry and Spring Road drivers, by and large their ratings were similar.

For the wave 2 drive, Paces Ferry drivers rated the attributes similarly: Road Pavement Quality, Lane Width, and Pavement Marking Quality received the highest ratings and Number of Times Stopped By Red Lights, Amount of Time at Red Lights, and Driving Behavior of Others received the lowest ratings. When the ratings for the Paces Ferry and Spring Road drivers were compared, the differences between the two samples in wave 2 mirrored those found in wave 1.

In addition to satisfaction, drivers also were asked to rate the importance to them of the roadway attributes. Interestingly, on the both the treatment and control route, drivers were least satisfied with those roadway attributes that were most important to them. Driving Behavior of Others is important to these drivers, but they are relatively less satisfied with it. Number of Times Stopped at a Red Light, Amount of Time at Red Lights, Overall Travel Speed, and Traffic Signal Coordination fall into this category as well.

Lessons Learned

This study offers a number of valuable "lessons learned" that future evaluators will want to consider in conducting future, similar evaluations. The following are some key lessons learned, grouped according to topic area.

1. Controlling For External Explanatory Factors
   A key challenge in pre-post studies is controlling for outside factors that may provide alternative explanations for why there was a change in satisfaction. To the extent that it is possible, alternative explanations that would impact the study findings must be identified and controlled for. Researchers need to consider the following issues:
   • Seasonal variation
   • Traffic incidents/severe weather
   • Characteristics of the individual trip (i.e., trip purpose, time of trip)
   • Infrastructure changes along the route
   • Traffic counts
2. Sample Design
   An important study design question involves the design of the sample. In order to draw reliable conclusions about driver satisfaction with roadway performance, a representative sample should be drawn. By employing representative sampling techniques, the sample that is collected will reflect the larger population of drivers on the route, thus making it possible to generalize from the sample findings.
   
   A key question that needs to be resolved at the outset is how will the sample be collected? In Cobb County, it made sense to sample by geographic area, since it was possible to obtain a residential telephone sample for census tracts near the study route(s). There are additional factors that evaluators will want to consider in developing the sampling strategy. These include:
   • Will the sample be distributed evenly across all days of the week?
   • How will the sample be distributed by time of day?
3. Sample Size
   Careful consideration needs to be given to what sample size is necessary to meet the data requirements of the study. If random sampling techniques are being used, decisions on sample size will depend on how large a shift (from pre to post) you want your test to be able to detect, as well as how powerful a test is required. With larger samples, the power of the test increases.
4. Survey Design
   The driver survey needs to be carefully designed in order to balance two oftentimes competing aims: collecting the required data and maintaining a reasonable number of questions. If the survey is too long, drivers may choose not to complete it. Careful consideration needs to be given to the specific list of roadway factors that will be evaluated. The set of roadway factors used in the Cobb County study provides a good starting point; however, depending on the characteristics of the specific roadway being tested, as well as the specific ITS enhancement that is being evaluated, items may be added (or deleted) as necessary.
5. Data Collection Procedures
   Rigorous data collection procedures were used in the Cobb County driver satisfaction study in order to achieve the highest possible response rates. The following measures were utilized to insure the collection of reliable data:
   • Pilot test
   • Advance letter and brochure
   • Incentives
   • Reminder calls/emails
   • Panel Maintenance letter
   • Minimal time lag between the two waves
   • Use of multiple data retrieval channels
   • Careful monitoring of each respondent's progress, with follow-up as necessary
   • Careful monitoring of respondents' survey comments during the study period

Conclusions

The findings from the Volpe driver satisfaction study and the Georgia Institute of Technology converge, indicating that in fact there was no observable improvement in roadway performance due to the adaptive timing signal system. A likely reason for the null findings is that the corridor was already performing at an optimal level with respect to traffic signal coordination under the initial signal timings. Overall, these results suggest that an adaptive signal system may not increase drivers' day-to-day satisfaction with their roadway experience on a corridor that is already optimally timed. In other words, day-to-day customer satisfaction is an insufficient justification for investing in SCATS versus having an optimally timed traditional system.

From a methodological standpoint, the findings from the Volpe study suggest that it is indeed possible to reliably measure driver satisfaction with roadway quality. First of all, response rates were good and similar to those obtained in other transportation studies. Secondly, the driver ratings were consistent with observable roadway conditions.

To guide future, similar evaluations, this report presents a detailed description of the methodology, a set of "lessons learned" and appendices that include all survey materials used in the study. Evaluators may find it necessary to modify the methodology, depending on the specific research question being addressed (or the specific characteristics of the test site). They will need to assess which components of the methodology can be adopted "off the shelf," and which need to be tailored.

back to table of contents

---

I. Introduction

Despite growing recognition that driver satisfaction with roadway quality is a useful and necessary measure, the absence of an established, validated methodology and tools for measuring customer satisfaction has resulted in a continued reliance on objective measures for assessing roadway quality. The primary hypothesis of this study is that it is possible to develop customer satisfaction measures that are a reliable determinant of roadway quality. A signal system upgrade in Cobb County, Georgia offered the opportunity to test this hypothesis. The Cobb County Department of Transportation planned to instrument 15 signalized intersections on Paces Ferry Road with the Sydney Coordinated Adaptive System (SCATS). In order to test its proposed methodology for measuring customer satisfaction with roadway quality, the Volpe National Transportation Systems Center (Volpe Center) conducted a before-and-after study of drivers on the Cobb County urban arterial treated with the adaptive traffic signal system.

In addition to providing a test of the methodology, this study would also measure whether there were changes in driver satisfaction with roadway quality as a result of the ITS deployment. Cobb County had recently completed a retiming of the corridor, so the current study would provide insights on whether there is value added — from the customer's perspective — when one moves from an optimally-timed traditional signal system to an adaptive traffic signal system. In order to isolate external effects that might impact driver satisfaction ratings but which are unrelated to the traffic signal improvements, a "control" panel was also used, whereby a panel of drivers were surveyed on a comparable corridor where there was no ITS deployment. The expectation was that drivers on Paces Ferry Road would be more satisfied with the roadway quality after the system was deployed compared to pre-deployment, whereas there would be no change in satisfaction on the control route.

The first section of this paper presents background information on the study, as well as information on the study site. This is followed by a detailed description of the survey methods. The survey results are then presented, with the primary focus being the substantive findings on changes in customer satisfaction with roadway quality (from pre- to post- deployment). The following section on "lessons learned" highlights some of the key issues and concerns that evaluators need to consider in planning future driver satisfaction studies.

Background

This study is an extension of earlier qualitative research conducted by Pecheux, Flannery, and Lappin investigating driver satisfaction on urban arterials.¹. For their study, drivers in four different cities — Atlanta, Tallahassee, Chicago, and Sacramento — were asked to drive a pre-determined route and to talk aloud about the factors that most affected their level of satisfaction during the drive. A key factor identified by drivers across all four cities was the "efficient flow of traffic." That is, drivers were more satisfied when there was a smooth progression to traffic, with minimal waiting at signalized intersections. Across all four cities drivers complained about traffic signals that were not efficiently timed and spoke about the need to coordinate the timing of multiple traffic signals in order to improve the flow of traffic. This study illustrated that drivers do notice roadway and driving conditions that are mediated by ITS-related service elements, and these conditions clearly influence their level of satisfaction with their driving experience.

Based on this qualitative work, the Volpe evaluation team proposed the development of a standard methodology for measuring customer satisfaction with roadway quality. The objective was to obtain quantitative measures of customer satisfaction through the collection of representative data, so that reliable conclusions could be drawn about customer satisfaction. The approach to evaluating a planned ITS enhancement had the following key components:

1. Conduct a qualitative pilot study to better understand the contextual variation at the selected site and to test the survey instrument for local relevance;
2. Conduct a pre-and post-survey with a panel of the same drivers on the route being treated with an ITS enhancement;
3. Conduct a control panel on a comparable route that has no planned ITS enhancement.

The details of this methodology are described in the next chapter.

Site Selection

In the first phase of the evaluation, a critical task was the selection of a site for testing the proposed methodology. Several criteria were developed for assessing the appropriateness of a potential site. First, the planned ITS enhancement had to be of sufficient magnitude to be noticed by drivers, and second, plans for the ITS enhancement had to be well underway, with implementation scheduled in the near future.

The evaluation team learned that in Cobb County, Georgia, there were plans to implement SCATS along a limited stretch of an urban arterial. The route to be treated included 11 signalized intersections along a two-mile stretch of Paces Ferry Road, as well as an additional three intersections on Cumberland Parkway, a major intersecting arterial, and one intersection on Atlanta Road, on the west end of Paces Ferry Road. Based on the performance of this 15-intersection system, a decision would be made on whether or not to instrument 55 additional intersections in Cobb County with the adaptive timing signal system. After several conversations with Cobb County Department of Transportation (DOT), the Volpe evaluation team made a site visit to Cobb County to determine whether this was an appropriate test site.

According to the Cobb County DOT, this corridor was chosen for the deployment in part because of the variable traffic volumes that result from the mixed land use development. The benefits of ITS adaptive signal systems tend to be realized when fluctuating volumes characterize traffic conditions. Cobb County expected that SCATS would improve roadway performance, particularly during off-peak hours, when traffic is more variable. In addition, because the corridor is somewhat isolated, it should be easier for engineers to measure the benefits of the deployment. Cobb County DOT also anticipated improved performance during non-recurring traffic fluctuations (i.e., incident, construction, etc.) and holidays. Due to the difficulty in gathering sufficient before and after data for these conditions, the Volpe evaluation is limited to day-to-day traffic, avoiding non-recurring events. However, it was hoped that the railroad crossing, with typically 30+ trains a day would allow for at least a sense of performance under non-recurring conditions.

In summary, the Cobb County site met the two key criteria for selection. Based on conversations with the Cobb County DOT, the Volpe evaluation team expected that the effects of the adaptive signal system on roadway performance should be large enough to be noticeable to drivers. Moreover, the system would be deployed relatively soon, in the fall of 2004.

There were several other reasons that Cobb County was an appealing test site for the current study. First, the corridor was already functioning at an optimal level using a traditionally-timed signal system. Approximately two years ago, major capacity improvements were made to the Paces Ferry diamond interchange with I-285, and in January 2004 updated time-of-day (TOD) signal timing and coordination plans were implemented. Given these improvements, it was the impression of Cobb County DOT that the corridor was working as well as could be expected given traditional signal control methods. Consequently, this site would provide a good test of whether the adaptive signal system (more specifically SCATS) increased driver satisfaction beyond an optimally timed traditional system.

Secondly, the site was appealing because Cobb County planned to collect data on the performance of the new signal system through an independent evaluation conducted by the Georgia Institute of Technology. Using floating cars studies, the Georgia Institute of Technology would collect objective measures for travel time, speed, and delay both before and after the signal upgrade. This evaluation would complement the effort by the Volpe evaluation team and would provide a useful context for interpreting the driver survey results.

In selecting Cobb County as the test site, consideration also was given to the fact that customer satisfaction is an important component of Cobb County's program. The county engineers were highly receptive to the project and welcomed the opportunity to work with the U.S. DOT to obtain direct measures of customer response to the new adaptive signal timing system.

Another component of the site selection process was the selection of the control route. During the on-site visit to Cobb County, the Volpe evaluation team considered several potential control routes. Ultimately, the decision to use Spring Road as the control route was based on the following considerations:

• Proximity to the treatment route. The control route should be close to the treatment route, so that weather patterns are similar during the evaluation, and major incidents on the freeway would have a similar impact on both routes.
• Similar mix of residential, retail and office development. This helps to ensure comparability across the two routes, in that similar types of drivers, with similar trip purposes will be driving both the treatment and control routes at similar times of day.
• Both treatment and control routes are in the same county. While this is a less important consideration, the administration of the survey will be much easier if dealing with one county.

Characteristics of the Treatment and Control Routes

There are a variety of land uses along the treatment route, Paces Ferry Road. The eastern end of the route (at the intersection of Paces Mill Road) begins in historic Vinings as a two-lane facility (one lane in each direction). This section of the route is residential, with smaller retail shops as well. The route crosses over a railroad crossing (which often causes traffic back-ups) and then increases to four lanes, then six lanes at the juncture of Interstate 285, which is approximately the halfway point of the corridor. In the vicinity of the interstate exchange are several office parks (including the Home Depot national headquarters), a large shopping center with a Publix food store and a Home Depot, and restaurants. After the interstate exchange, the roadway reduces to four lanes, and becomes primarily residential. The western end of the study route terminates at the signalized intersection where Paces Ferry Road intersects Atlanta Road.

The control route is a two-mile stretch along Cobb Parkway and Spring Road that contains approximately 12 signalized intersections. Similar to the treatment route, the control route crosses Interstate 285 (one exit south of Paces Ferry Road) and also runs through retail, office park and residential developments. The roadway design is somewhat different in that there is no stretch of the control route that has a single lane in each direction. Rather, the control route consists primarily of two lanes in each direction, with more lanes where the route crosses the highway. However, this route was selected because it was comparable to Paces Ferry Road on other, more important dimensions. First, it had a similar mix of residential, retail and office development, ensuring that similar types of drivers, with similar trip purposes will be driving both the treatment and control routes at similar times of day. Second, the control route is close to the treatment route, so that weather patterns are similar during the evaluation, and major incidents on the freeway would have a similar impact on both routes.

back to table of contents

---

II. Survey Methods

This section of the report describes the survey methodology, including the overall approach, the sample design, the survey design, and the data collection procedures.

Approach

At the outset, the key research question to be addressed was the following:

Can we reliably measure customer satisfaction, such that it is possible to determine whether an ITS enhancement significantly increases drivers' level of satisfaction with their roadway experience?

A pre-post or panel study approach was deemed the most appropriate method for addressing this research question. In order to determine whether there has been a change in satisfaction, there has to be a comparison of driver satisfaction measured before and after the ITS deployment. An alternative approach would have been to interview drivers at only one point in time — after the deployment of the ITS enhancement — and ask them whether or not they were more satisfied with different aspects of roadway quality, compared to previous experience on the roadway. However, such an approach requires drivers to: 1) assess their current driving experience, 2) recall their previous driving experiences, 3) compare those experiences, and 4) calculate whether their experience has improved. The more demanding nature of this task and the potential problem with accurate recall necessarily produces less reliable data. With the current study design, drivers simply rate roadway quality at two points in time based on their immediate driving experience. The drivers are not specifically asked whether their roadway experience has improved; rather, this question is addressed by comparing the independent measurements of driver satisfaction (pre vs. post deployment of the ITS).

The evaluation team also wanted to move beyond the qualitative methods employed in earlier work, to collect quantitative data. Surveys are a reliable, cost efficient mechanism for collecting such data. Moreover, surveys provide the most appropriate tool for meeting two key data requirements of this study, including:

• The study must provide representative results. The advantage of collecting representative data is that it is then possible to generalize from a sample of drivers to the larger population of drivers who use the route. The evaluation team wanted to insure that the opinions and perceptions of all roadway users were captured (as opposed to any one segment of users).
• The methodology is replicable. A goal of this study was to develop a standardized approach to measuring driver satisfaction, so that evaluators have a template for conducting their own studies. A standardized approach has two benefits. First, evaluators will save funds that they otherwise would have spent on the study design, and second, if evaluators are using the same overall methodology, it may be possible to compare findings across studies. ²

The evaluation team determined that the most reliable means of capturing the drivers' true experience is to have them assess roadway quality immediately following an actual driving experience on the road. While a lab experiment could have been devised whereby users evaluate roadway conditions based on videotapes (with and without adaptive signal systems), it was determined that the artificiality of the lab setting was a significant weakness of such a study design. Even if drivers in a lab setting registered increased levels of satisfaction (due to the adaptive signal system), how could we be sure that drivers on the roadway, in a real world setting, would also express increased satisfaction? During an actual driving experience, drivers have numerous demands on their attention. While this makes it more difficult to capture the effects of an adaptive signal system on driver satisfaction, it also provides a more realistic test of the hypothesis.

To strengthen the study design, a control panel was added. At the same time that the driver surveys were administered on the treatment route, the same survey would also be administered to a different set of drivers on the control route, using the exact same sampling and data collection procedures. Since no infrastructure or other roadway improvements were scheduled for the control route, the hypothesis was that there would be no increase in satisfaction on that route. The purpose of the control panel was to bolster our confidence that any increases in satisfaction on the treatment route could be attributed to the adaptive timing signal system. In other words, if there were increased levels of satisfaction on the treatment route, but not the control route, then the evaluation would feel more confident that the changes were due to the adaptive signal system. However, if there were increases in satisfaction on the treatment and control routes, then the results would be less conclusive, and one could not rule out the possibility that the increased satisfaction was due to random measurement effects.

In summary, the evaluation team concluded that the most appropriate method for collecting reliable data on driver satisfaction was to conduct a panel study whereby drivers complete a survey immediately following a typical drive on the study route, both before and after the ITS deployment. Moreover, the evaluation team determined that rigorous survey methods must be used to increase response rates and to insure the collection of reliable data. With higher response rates, there is greater confidence that the sample findings are indeed representative of the population. As discussed later in this chapter, every effort was made to successfully recruit and maintain the panel of drivers.

Target Population

The target population for this study was "regular" drivers of both the treatment and control routes. In particular, familiarity with the route was a key criterion for participation, as study participants must have some established expectations about how the road operates in order to notice a difference resulting from the ITS signal system deployment. This criterion was based on the findings from the qualitative work conducted by Flannery, Pecheux and Lappin. In their study, drivers who were familiar with a particular route had developed their own personal metrics by which they judged the performance of the roadway. Some drivers, for example, knew how many light cycles it usually takes them to get through an intersection, and judged the performance of the roadway against that standard. Drivers who were less familiar with a route generally had more difficulty providing assessments of their roadway experience.

For measurement purposes, a "regular" user of the roadway was defined as a driver who drove on either the treatment or control route at least three times per week or three times per month (if their driving typically occurred on weekends). Other important criteria were used to determine eligibility. Each of these is detailed below, as well as the reasons for including the particular criteria.

Eligibility Criteria	Reasons for Criteria
Respondent had to have a valid driver's license	Respondents must be drivers; this criteria screens out non-drivers (as well as illegal drivers).
Respondent's household had to own at least one vehicle	If the respondent's household has at least one vehicle, there is a greater likelihood that the respondent will be able to complete the scheduled driving task.
Respondent could not be employed by Cobb County, the Georgia Department of Transportation or the US Department of Transportation.	If employed by one of these organizations, the respondent may learn about the ITS deployment and/or the evaluation.
Respondent has to be between 21 and 75	Younger drivers and older drivers may not be capable of adequately performing the required task.

Travel days for the study included Tuesdays, Wednesdays, Thursdays, Saturdays, and Sundays. Mondays and Fridays were omitted because the traffic patterns on these two days often vary (due to their proximity to the weekend). In order to participate, drivers had to regularly drive the route on one of the study days. In addition drivers qualified for the study according to the time of day at which they drove the route. Drivers were eligible for either a peak or an off-peak drive time (depending on their normal usage). Based on input from the Cobb County DOT regarding local traffic conditions, parameters were set for peak versus off- peak hours. Off-peak was defined as:

• Weekdays between 9 am and 4 pm, and
• Weekends from 7 am to 6 pm.

Weekday peak hours included:

• Morning commute (7 am to 9 am), and
• Evening commute (4 pm to 6 pm).³

Consequently, the study findings are representative of drivers who regularly drive the study route during the designated peak and off-peak hours.

Sample Design

Once the target population is defined, a key consideration in the study design is how will the sample be collected? The most efficient method, and the one most likely to provide representative data, included the random sampling of residential telephone numbers in the geographic areas surrounding the treatment and control routes. These households would be most likely to regularly drive on the study routes, and a random sampling of such households would provide data on drivers who use the road for a variety of trip purposes, including commute trips, shopping, personal appointments etc. One weakness of the sample design is that drivers who reside outside the census tracts in which the study routes are located were excluded from the study. While it would have been possible to expand the geographic area from which the sample was drawn, this would have been extremely inefficient and costly, due to the significant number of calls that would be necessary to find eligible drivers (i.e., those who regularly drive on the study route). Moreover, the evaluation team determined that the exclusion of these drivers who live farther away should not bias the results.

Consideration was also given to identifying businesses along the route and trying to obtain a list of employees from which a random sample could be drawn. However, it would be impossible to obtain a complete list of employees, so such a sampling frame would be biased from the outset. Moreover, such a sample would be excluding key groups of drivers, such as those who are retired, those who work inside their home, and homemakers, among others.

Incidence

An important aspect of the sampling design was to estimate how many roadway users lived in the areas immediately adjacent to the roadways, and whether those residential areas could be geographically defined in order to serve as a foundation for a random sampling of telephone numbers. This was done by focusing on the two demographic characteristics measurable through the census (vehicle ownership and driver age) as well as geography surrounding the roadways under study. The geographic analysis began at the larger PUMAs (Public Use Microdata Areas) in the Atlanta area, and then focused more intently on the census tracts within the most closely aligned PUMA. Figure 3 indicates the PUMA that contained both the control and treatment segments.

The percentage of households in this area with at least one vehicle owned was 65 percent; the percentage of households with at least one individual between the age of 21 and 75 was 97 percent. Taking both criteria into consideration, the overall eligibility rate was determined to be 63 percent. The actual eligibility rate was anticipated to be something less than this given the employment constraint (study participants could not be employed by Cobb County, the Georgia Department of Transportation or the US Department of Transportation) and the fact that respondents needed to travel on either the treatment or control segment at a regular frequency.

Sampling Strategy

The initial sampling strategy focused on drawing samples according to the census tracts that contained both the treatment and control corridors. The map below provides an outline of the PUMA area under study (corresponding to the highlighted portion in Figure 3) and the census tracts that comprise that PUMA. The highlighted tracts are those that were the focus of the sampling effort for this study.

In addition, the sampling strategy called for oversampling off-peak drivers. Since Cobb County DOT expected that the greatest benefits of the adaptive signal timing system would be experienced during off-peak hours, the evaluation team wanted to oversample this group of drivers in order to have a sufficient sample size for capturing a change in satisfaction. The decision was made that two-thirds of the study drives should be comprised of off-peak drives.

Survey Design

The design of the survey instrument was based on the overall objectives of the study and an assessment of what specific data were required in order to meet the study objectives. The following three forms were developed and administered to each respondent on both the treatment and control routes: a telephone recruitment screener, background survey form and driver survey form (see Appendices B and C).

The recruitment screener was used to determine driver eligibility, document roadway usage, obtain demographic information, and schedule the drive date and time. Drivers who were found to be eligible and who agreed to participate were then asked a short series of questions about their "typical" usage on the roadway in order to assign the driver the most appropriate day and time slot. The successful administration of the recruitment screener was largely dependent on appropriate training of the interviewing staff. The staff was shown video of the treatment route and was required to familiarize themselves with the names of the intersecting roads as well as landmarks (i.e., Home Depot Headquarters, the Publix, etc.). In this way, the interviewer could more easily collect the data and could earn the trust of the participants.

The background information form was used to obtain data about the recruited driver's typical trip on Paces Ferry Road (or Spring Road). Key characteristics of the "typical" trip are documented, including trip origin and destination, trip purpose, trip distance (miles), trip time (minutes), number of stops typically made on trip, level of concern about on-time arrival, and degree of flexibility regarding when the trip can be made. The evaluation team wanted to be able to measure the relationship between these trip characteristics and the driver satisfaction items. Drivers were also asked to rate the importance (to them) of the roadway attributes when they make the trip. Information about the drivers' vehicle and general driving habits was also collected.

The primary data collection tool was the driver survey form, which was designed to measure driver satisfaction with a variety of roadway attributes. It was also used to document drive conditions during the scheduled drive as well as other factors that may have influenced the driver ratings (such as schedule flexibility for that day).

For the driver satisfaction measures, a seven-point scale was used, with the endpoints of the scale defined for the driver (1 being not satisfied and 7 being very satisfied). In designing the question on driver satisfaction, a key question at the outset concerned the appropriate scaling for the satisfaction items. Would a 5-point scale suffice, or would a 7-point or a 10-point scale be more appropriate? According to the literature, the type of scale that is used should be guided by the nature of the research question. Given that the ultimate objective of the Cobb County study was to measure change in satisfaction, the scale had to consist of enough points to reliably capture a change in satisfaction, but at the same time, the scale had to be meaningful to drivers. For this reason, a seven-point scale was chosen. This would provide sufficient discrimination to measure changes in satisfaction, but would not prove too unwieldy for the driver.

The development of the list of roadway attributes was based largely on the qualitative research conducted by Flannery, Pecheux and Lappin. Attributes consistently mentioned as most important by drivers of that study were included in the Cobb County driver survey (as long as they were relevant to the particular study site). The roadway attributes that were evaluated include:

Lane Width Quality of Road Pavement Quality of Pavement Markings Roadside Landscaping Driving Behavior of Other Drivers Overall Level of Traffic Congestion

Number of Times Stopped by a Red Light Amount of Time Spent at Red Lights, Amount of Green Time for Side Streets Coordination of Traffic Signals Along the Route Your Overall Travel Speed Availability of Turn Lanes

Four of the measures pertained to different aspects of traffic signal coordination, the key focus of the study. The qualitative interviews in the study conducted by Pecheux et al. revealed that drivers used different metrics for evaluating their satisfaction with traffic signal coordination. While some drivers spoke specifically about "traffic signal coordination," others assessed their experience (and their satisfaction) according to the number of times they had to stop at a red light, or alternatively the amount of time they had to wait at a red light. Others mentioned the amount of green time to side streets (i.e., too much green time to side streets decreased driver satisfaction). Each of these measures was included in the driver survey in order to capture the full range of drivers' experience on the roadway.

Additional measures, unrelated to traffic signal coordination (for example lane width and quality of road pavement) were also included in the survey. There were several reasons for this. First, the evaluation team did not want to tip off drivers that the main focus of the study was the evaluation of traffic signal coordination along the route. By including a variety of roadway attributes in the survey, attention would not be drawn to the issue of traffic signal coordination. Second, it would be useful to compare wave 1 and wave 2 driver ratings for roadway attributes that did not change, such as Lane Width, Quality of Road Pavement, and Quality of Pavement Markings⁴. The consistency of the ratings from wave 1 to wave 2 on these measures provides a test of the robustness of the method. Finally, by including a full range of roadway attributes, it is possible to conduct analyses regarding the relative importance of different roadway attributes in determining overall driver satisfaction, and so provide a richer understanding of the factors related to driver satisfaction in Cobb County.

The driver survey administered in wave 2 was identical to the driver survey administered in wave 1, with one exception. At the end of the wave 2 survey, drivers were asked to comment (in an open-end question) on whether or not their driving experience had changed over the last few weeks. This question was added to provide respondents with another means for reporting on any improvements with traffic signal coordination (see Appendix F for wave 2 driver survey).

Pilot Test

Prior to wave 1, a small pilot test was conducted with ten drivers. The purpose of the pilot test was threefold: (1) to conduct a "dress rehearsal" of the planned approach designed for use in the full study, (2) to test the questionnaire wording and understanding, and (3) to debrief respondents about their participation experience and reactions to materials. The related objectives included:

• Examine how well the data collection approach planned for the full study works
• Evaluate how well the questionnaire works and confirm that the data items collected through this process will enable us to estimate changes in driver satisfaction.
• Evaluate respondent reaction to the survey process

A 5-stage approach planned for the full study was tested, in whole or in part, as part of the pilot test. The stages to be tested included: advance notification, recruitment, provision of materials, driver reminders, and survey completion and debrief. Each stage was evaluated using questions agreed upon prior to the start of the pilot. The following section of the report provides detailed findings regarding each stage.

Advance Notification

The advance mailing was not tested in the pilot, given the desire to use the debrief interviews to identify key issues that would form the content of the advance letter and study brochure. Because the timing of the receipt of the advance mailing vis-à-vis the recruitment call is important, the length of delivery time for first-class letters between the research facility (from which all materials would be mailed) and Atlanta was tested. Specifically, three letters were mailed using first-class postage from the research facility on Thursday, September 9, 2004 to different locations in Atlanta. All three letters were reported received on Monday, September 13th. The mailing took a total of 4 "mail" days to reach the Atlanta residents (given that no mail activity takes place on Sundays). Thus, for the full-study, advance letters should be mailed 5 days prior to the planned recruitment call.

Recruitment

The purpose of the recruitment effort for the pilot was to secure the participation of 10 local drivers. These calls were made by research staff, focusing on telephone numbers known to belong to residences along the target route of Paces Ferry Road between Atlanta Road and Paces Mill Road. Ten drivers were recruited to participate and were assigned to the following drive times:

TABLE 1: PILOT TEST DATA COLLECTION TARGETS

Day of Week	AM Peak (7-9 am)	Noon Peak (noon - 1 pm)	PM Peak (4 - 6:30 pm)	Weekday Off Peak (all other hours)	Weekend Off Peak (all hours)
Tuesday	1	0	1	1
Wednesday	2	0	1	2
Saturday					1
Sunday					1

One interesting finding was that none of the respondents reported traveling on Paces Ferry Road during the peak noon hour. Given the low incidence of drivers in this time period, it was decided to schedule drivers for only one weekday "off peak" time period of 9 am to 4 pm.

The pilot test provided insight on a number of different aspects of the recruitment effort, including the overall performance of the recruitment screener, respondent questions about the survey process, average number of calls required to reach an eligible driver, reasons for refusal to participate in the study, and the length of the recruitment interview. Overall, the recruitment screener worked very well; respondents did not have any problems with the recruitment questionnaire, nor did they have any questions about the survey process. Familiarity with the target route was definitely a key to discussing and qualifying respondents to participate in the survey. It was noted that interviewer training needed to focus on the landmarks and locations along both the control and target routes, as well as the screening questions, in order to ensure that respondents were correctly qualified and assigned to the "best" time slot given their usual driving habits.

The recruitment interview length was shorter than anticipated, averaging eight minutes instead of ten minutes. This suggested that there was room to add a few additional questions. From a textual perspective, the interviewing team felt that they needed to enforce the notion of driver satisfaction along the target and control routes, making the questionnaire more conversation oriented. Specific wording in the introduction and recruitment text were targeted for improvement.

Of the 40 pieces of sample dialed, 10 resulted in recruits, 19 were disconnects (no new number), 4 additional were "new numbers" (in which case respondents were no longer qualified as they didn't drive Paces Ferry Road after their move), and the remainder were "non-contacts." An average of 2.5 call attempts were made for each eligible sample piece.

None of the respondents refused to participate. Two households were found to be ineligible because they moved and no longer drove on the target route, five households reported no drivers traveling along that route, and two drivers were too old to participate. In addition, one household reported making 2 round trips per week on Paces Ferry Road. Although this translated to 4 trips and could have been eligible for the survey, that household was not recruited given the goal of including drivers with more typical travel on specific days of the week.

Provision of Materials

Nine of the ten drivers were mailed a packet ⁵ that included a cover letter (thanking them for their participation, confirming the assigned drive time, conveying the importance of their participation, and reminding them of the general "driver survey" to be completed after their scheduled drive. The packets were mailed via priority or overnight services, depending on the scheduled drive time. Volpe letterhead was used for the cover letters. All of the respondents received their packet of materials, with the exception of one, whose drive had to be rescheduled because the packet was left at her neighbor's door.

Driver Reminders

The day prior to the assigned drive time, the survey process called for the research facility to send an email reminder (where available) AND make a telephone reminder call to the driver. The purpose of the reminder call was fourfold: (1) to confirm receipt of the packet, (2) to reconfirm driver participation at the scheduled day/time, (3) to answer any questions about the survey process, and (4) to schedule the best date/time for the post-survey call and debrief (note — in the full study, the retrieval would be passive. The retrieval call was for purposes of the pilot only.)

Very few respondents were reached during the reminder calls. Those that were reached acknowledged and appreciated the call. One respondent indicated she did remember her scheduled drive time, but appreciated the reminder. Reminder emails were sent to two respondents, both of whom acknowledged receipt. Of the respondents who were reached during the reminder call, none of them had questions about the survey process.

Survey Completion and Debrief

At the appointed time or the day following the scheduled drive time, each driver was re-contacted to obtain their survey responses and to debrief them on their participation experience. In the full study, an Internet-based "web" survey, fax, and mail-back options were offered, with telephone follow-up to clarify any inconsistent responses. For the pilot, however, the survey research firm wanted the in-person interaction to talk about the drivers' experiences and probe for details that will be important in conducting the full study. Four respondents ultimately answered the survey questions and were debriefed on their experiences.

Reaching the respondents after the scheduled drive time proved difficult. The pilot schedule bumped up against the Labor Day weekend and was accompanied by extremely bad weather, which made weekend follow-up very ineffective. One respondent (whose wife was also recruited) was reached 4 times and each time told us to talk to his wife ("she had the forms"). Three other respondents indicated conflicts that precluded their participation in the study (after the fact). One respondent (who preferred electronic communications) never responded to either email or telephone message requests for his information.

The interview and debrief averaged fifteen minutes. The reporting of the background and driver information went quickly and smoothly. The debrief questions took longer, given the qualitative nature of the questions. Overall, the debrief revealed that the survey instruments worked well. The only question that had to be revised was G2 on the Background Survey, as the original phrasing of the question was confusing to respondents. Otherwise, no problems were reported with the background or driver survey forms. However, all four respondents asked about the purpose of the study. Most were concerned that "they" would try to widen Paces Ferry Road in the historic Vinings section, which would "ruin" the character of that area. They also felt that the main problem was the railroad crossing, but felt there was no specific design solution. Finally, two drivers noted that their driving experience was vastly improved since the I-285 interchange had been redone.

Based on this feedback, the evaluation team felt it was important that the survey materials should re-iterate the purpose of the study and that the results would not be used to make physical changes to the roadways. In addition, the pilot study revealed that there should be a convenient way to reschedule and not lose any respondents due to unexpected occurrences precluding the scheduled drive.

Pilot Test Conclusions

Initial respondent reaction to the survey was positive. Of all respondents contacted for potential participation in the study, there were no refusals. Interested respondents were screened and terminated as ineligible (they did not drive on Paces Ferry Road at all or on a regular-enough basis) or because they were too old. Those respondents that completed the survey indicated that the survey task was explained well, they understood what was being requested of them, and how to complete the background and driver survey forms.

It was disappointing that only four respondents completed the pilot. Of the six respondents that were recruited but did not complete the survey, three had unplanned events that prevented them from making a "normal" drive, the other three could not be re-contacted after the reminder call/email. Recommendations to boost response rates included changes to the timing of the reminder call, pre-payment of the incentive (rather then payment after the task), and adding a reminder postcard to arrive the day prior to the scheduled drive. Other details or recommended changes included:

• All mailings should be sent five days prior to the planned contact to allow sufficient time for delivery.
• Plans to offer a website for entry of results should proceed as a means of increasing response.
• The recruitment text needed to be strengthened with regard to study objectives and how the results will be used.

Data Collection Procedures

The data collection procedures incorporated features designed to elicit the highest possible response rate from eligible drivers. During the first stage of the study, a study brochure was mailed to all households in the sample for which a name and address were known prior to the recruitment call. This brochure served as advance notification to the household that it had been randomly selected for the study and that it would be receiving a call shortly asking members of the household to participate (see Appendix A).

The brochure provided information about the purpose of the study. Participants were not told about the specific objectives of the study, as this might bias their perceptions. The brochure stated that: "The results will be used to develop satisfaction ratings that can then be used when identifying and prioritizing roadway improvement projects." The pilot test revealed that residents were concerned that the findings from the study would be used to make physical changes to the roadways, particularly near the historic Vinings area. To address this concern, the brochure tried to reassure potential respondents that: "The results of the study will be used to help transportation planners across the country make the most of limited funds by focusing them on improvements that do not involve new road construction but are directly related to driver satisfaction."

The brochure also served as a tool for legitimizing the study and for conveying the overall importance of the project. To this end, the brochure listed the United States Department of Transportation as the sponsor of the study, and explained that the survey was being conducted in cooperation with the Cobb County Department of Transportation. The link to local government was used to emphasize the local relevance of the study to the participant (i.e. you can help shape future transportation projects in your local area). In addition, the brochure provided contact information for the U.S. DOT study manager should participants have any questions.

The recruitment interview was administered using Computer Assisted Telephone Interviewing (CATI). Each sampled household was telephoned by an interviewer who administered the recruitment screener and scheduled a drive with the respondent. As previously mentioned, travel days for the study included Tuesdays, Wednesdays, Thursdays, Saturdays, and Sundays, and respondents were assigned to a day of the week according to when they typically drove the route.

After agreeing to participate, eligible respondents were mailed the study materials, including a cover letter, a five-dollar incentive, the background information form, the driver survey, and a postage-paid envelope for returning the survey. The cover letter, printed on Volpe letterhead, was used to re-iterate the purpose and importance of the study, to provide general information and instructions on the survey process, and to thank the respondent for their participation. Drivers were asked to complete the background information form prior to their scheduled drive, whereas the driver survey was to be completed immediately following their drive. Respondents could return the background form and the driver survey either by mail, internet, or fax.

The night prior to the assigned travel day, reminder calls were made to each driver to confirm that they had received the materials, to answer any questions that the respondent might have, and to re-iterate the importance of the study (thus increasing the likelihood that they would complete the task). In addition, a reminder postcard was mailed to respondents so that it would arrive the day before the scheduled drive.

Table 3 shows the distribution of the 1,470 recruited drivers by route, time of day (peak/off peak) and day of week (weekday or weekend). The assignment of drivers to peak and off-peak was intentionally set at 33% peak, 67% off-peak.

TABLE 2: DISTRIBUTION OF RECRUITED WAVE 1 DRIVERS BY TIME OF DAY AND DAY OF WEEK

	Paces Ferry Road			Spring Road			Grand Total
	Peak	Off Peak	Total	Peak	Off Peak	Total
Weekday	294	240	534	199	112	311	845
Weekend		347	347		278	278	625
Total	294	587	881	199	390	589	1,470

In between the two waves, the survey research firm mailed respondents a letter thanking them for their participation and indicating the upcoming and final phase of the study. Enclosed in the letter was a $2 incentive, used to increase the likelihood that the respondent would complete wave 2 of the survey.

In early April, after the adaptive timing signal system had been deployed, respondents were re-contacted and the recruitment screener was re-administered in order to assess eligibility. In order to re-qualify, respondents had to continue to be regular users of the route, and the timing of their "typical" drive had to be consistent with wave 1 (peak vs. off-peak). Based on a review of the transportation literature, the evaluation team determined that a change in trip purpose could be permitted as long as the wave 2 trip still fell into the same general trip category as wave 1, namely: 1) subsistence/mandatory (commute, work related trips), 2) maintenance (shopping, personal business, medical etc), or 3) discretionary/leisure. The driver survey was mailed to eligible respondents, along with a $10 incentive, and the same data collection procedures used in wave 1 were repeated for wave 2.

Table 4 shows the distribution of the 724 recruited wave 2 drivers by route, time of day (peak/off peak) and day of week (weekday or weekend).

TABLE 3: DISTRIBUTION OF RECRUITED WAVE 2 DRIVERS BY TIME OF DAY AND DAY OF WEEK

	Paces Ferry Road			Spring Road			Grand Total
	Peak	Off Peak	Total	Peak	Off Peak	Total
Weekday	153	129	282	94	52	146	428
Weekend		180	180		116	116	296
Total	153	309	462	94	168	262	724

Data Processing

Data processing took place throughout the study, beginning with the creation of the advance brochure mailing, continuing with the release of sample for recruitment, processing recruitment data for the respondent mailout, appending the background and driver survey data to the master tables, and performing initial quality control measures on the data. The same steps were repeated for wave 2. A master control file tracked the progress of each driver through the various survey stages, with codes to allow immediate identification of problem cases that were not progressing according to schedule as well as confirmation that cleared cases moved along as appropriate. All cases were manually checked to confirm that the driver used the selected route according to project criteria. When driver surveys were not returned, multiple attempts were made to contact the driver to determine the reason why the survey had not been returned (didn't make the scheduled drive, forgot to put the survey in the mail, etc.) and whether the drive needed to be rescheduled.

Coordination and Scheduling of the Driver Surveys

In addition to developing the sampling methodology and data collection procedures, another important aspect of this study was the scheduling of the survey administration period. With panel studies, where a survey is administered before and after a particular treatment, the timing of the surveys is necessarily more complicated, as the administration of the surveys is contingent on the timing of the treatment (in this case, the adaptive timing signal system). Any delay in the deployment of the ITS results in a delay to the survey schedule.

For the Cobb County study, the installation of the adaptive signal system was scheduled for fall, 2004. The tentative evaluation schedule was to conduct wave 1 in the late summer/early fall and then conduct wave 2 in the early spring, once the new system was functioning optimally and drivers had several weeks to experience the new system. One schedule constraint was that the second wave of the study had to be completed before the end of the school year (the third week of May). The evaluation team did not want to conduct the second wave during the summer months, when typical driving (and traffic) patterns tend to change as result of schools being closed, and thus might have an effect on driver ratings.

During the summer of 2004 it became evident that there would be a delay in the installation of the adaptive timing signal system, as the Cobb County Department of Transportation had to re-bid the project. Based on assurances from the Cobb County Department of Transportation that the delays would not be significant, the evaluation team decided to proceed with the administration of wave 1 during the fall.

The survey research firm allotted approximately 6 weeks to complete the first wave of driver surveys, including the initial mailing, the recruitment of respondents and the conduct of the actual drives. Drives were scheduled beginning on October 26, 2004. Wave 1 drives were to be completed approximately 4 weeks later, prior to the Thanksgiving holiday. The evaluation team wanted to avoid conducting any drives during the Thanksgiving holiday weekend or during the December holiday season.

The number of completed surveys collected during the four-week period prior to Thanksgiving fell somewhat short of the desired targets. Based on wave 1 retrieval rates, the survey research firm was concerned about reaching its final target: 400 interviews at the end of wave 2. Consequently, in January and early February, additional respondents were recruited on both the treatment route (N=150) and the control routes (N=50), with the goal of obtaining an additional 130 completed interviews. For the control group, the number of recruits was further increased by 450 in order to obtain approximately 300 additional completed surveys. This large increase in sample size for Spring Road would improve the statistical reliability of the data.

During the winter months, the evaluation team maintained close communication with the Cobb County DOT in order to track the progress of the installation of the adaptive timing signal system. Based on the schedule for the ITS deployment, the evaluation team had to determine the appropriate timing of the panel maintenance letter, as well as when to begin recruitment for wave 2. Before the administration of the second wave, the evaluation team wanted to insure that the system was working optimally and that drivers had at least a couple of weeks to experience the new system.

The adaptive signal system was installed at the end of February, and significant tweaking of the system occurred in March. With the concurrence of the Cobb County Department of Transportation, the Volpe evaluation team began data collection for wave 2 of the study in early April. At the same time that the Volpe survey was fielded, the Georgia Institute of Technology collected its "after" data on travel time, speed, and delay. The schedule was very tight for wave 2 of the study, and so aggressive targets needed to be set to achieve the desired number of interviews. Both the Volpe team and the Georgia Institute of Technology were working under similar schedule constraints; they needed to complete data collection no later than the third week of May when schools would close (due to changes in traffic patterns during the summer months).

Response Rates

As detailed in previous sections of this chapter, significant efforts were made to achieve a high response rate. With higher response rates, there is greater confidence that the sample findings are indeed representative of the population of drivers who regularly drive on the route.

The overall response rate for wave 1, calculated according to standards established by the Council of American Survey Research Organizations, was 32% (this included a 50% recruitment rate and a 63% completion rate). This means that of all eligible households contacted, half agreed to help with the survey but 32% actually completed all wave 1 activities.

The corresponding wave 2 survey response rate is a straightforward calculation that involves the total number of drivers that agreed to participate in wave 2 divided by the number of panel members eligible for wave 2. Of the 924 drivers that completed the wave 1 survey:

• 724 agreed to complete the wave 2 survey
• 84 were no longer eligible (either due to a change in travel patterns or because they moved) for the wave 2 survey
• 42 drivers adamantly refused to participate in wave 2 activities
• Contact could not be made with 74 drivers, despite attempts varied by day of week and time of day.

Thus, the wave 2 response rate is 71% and is calculated by dividing the number of drivers who completed the wave 2 survey by the total number of drivers still eligible for the wave 2 survey (594/840).

back to table of contents

---

III. Survey Findings

The final data set for the Cobb County Driver Satisfaction Study contains data for the 1,470 drivers initially recruited to participate in the survey, the 924 drivers who completed the wave 1 survey, and the 594 drivers who completed both wave 1 and wave 2 surveys. These drivers provided information about their travel patterns on Paces Ferry and Spring Roads, as well as general travel characteristics such as annual miles traveled and on what type of roadways. Each driver was assigned a specific day and time to drive the selected route in the course of typical travel and then asked to complete a survey evaluating that drive. The data obtained through this study will be used to evaluate driver satisfaction with specific roadway characteristics.

The purpose of this chapter is to summarize the study findings. It is organized about the following topics: first the drivers — their demographic characteristics. This is followed by a summary of travel patterns, both in general as well as specifically on Paces Ferry and Spring Road and associated driver importance and satisfaction ratings. In the final section, correlations among the characteristics and ratings are presented.

Driver Summary

Drivers were randomly recruited from the residential areas surrounding Paces Ferry and Spring Road. This section provides a demographic description of the drivers, which were obtained during the wave 1 recruitment call. Descriptors include gender, age, educational attainment, employment status, and income.

Overall, the Cobb County driver study obtained a good distribution of drivers by key socio-demographic characteristics. The drivers were generally well represented in terms of gender (48% male, 52% female on Paces Ferry Road), with no statistical differences between the two routes. Based on eligibility requirements, drivers had to be between the ages of 21 and 75, and overall the final sample of Paces Ferry drivers had a good distribution of age ranges. There were fewer drivers in the youngest age cohort; only 8% were between the ages 21 and 29. An additional 25% were between the ages of 30 to 39, 24% were between the ages of 40 to 49 and 26% were 50 to 59 years of age. Close to one-fifth of the sample fell in the oldest age category (18% were 60 to 75 years of age).

For several demographic measures, there were significant differences between drivers of the two routes. Overall, a majority of the drivers reported having at least some college education; however, the Paces Ferry drivers were better educated, with significantly higher numbers having a graduate degree. With regard to income, the drivers tended to be drawn from higher income groups, and this was especially true for the Paces Ferry drivers. Forty-six percent of Paces Ferry drivers reported annual household incomes of $100,000 or more, compared to 18% of Spring Road drivers. These differences in the demographic composition of the two samples were not deemed problematic, however, since the key focus of the study was to measure changes in satisfaction within each sample (differences in ratings across the two samples were of secondary interest).

Most drivers were employed either full-time or part-time, with no significant differences between drivers of the two routes. In terms of household size, the samples tended toward smaller households in general, though the Paces Ferry drivers reported larger household sizes, on average than the Spring Road drivers (65% of Paces Ferry drivers reported one or two person households, compared to 75% of Spring Road drivers).

Travel Patterns

In addition to understanding demographic differences between drivers, it is also important to note any differences in travel patterns that might impact satisfaction ratings. In this section, information about general usage of the target and control routes is presented. Information about roadway usage was obtained during the recruitment interview. At that time, drivers were asked whether they used the roadway at specific times of day and days of week. If they indicated affirmatively, trip purpose was also obtained.

Table 5 shows when the Paces Ferry drivers are using that roadway during the travel times of interest to the study. As shown in that table, two-thirds of drivers are using Paces Ferry in the PM peak, with about half each reporting usage during the AM Peak or mid-day time period (the responses for each day/time cell are unique, so one driver could conceivably drive on Paces Ferry during all three time periods) for a given weekday. In terms of trip purpose, most of the trips were either to go to work or for personal business/shopping.

TABLE 4: TRAVEL PATTERNS — PACES FERRY USAGE

Day of Week	AM Peak (7-9 am)	PM Peak (4 - 6 pm)	Weekday Off Peak (9 am to 4 pm)	Weekend Off Peak (7 am to 6 pm)
Tuesday	194 (51%)	252 (66%)	190 (50%)
Wednesday	198 (52%)	255 (67%)	199 (52%)
Thursday	196 (51%)	246 (64%)	187 (49%)
Saturday				327 (85%)
Sunday				301 (79%)

Roadway usage was similar on Spring Road. There, almost three-fourths of the drivers were on the roadway during the PM peak period, with about half also reporting using the roadway in the AM peak and off-peak as well. Most drivers (81%) reported Saturday travel on Spring Road, with 75% also reporting Sunday usage. In terms of trip purpose, Work was 30% of the trips, Personal Business was 23%, and Returning Home was 20%.

TABLE 5: TRAVEL PATTERNS — SPRING ROAD USAGE

Day of Week	AM Peak (7-9 am)	PM Peak (4 - 6 pm)	Weekday Off Peak (9 am to 4 pm)	Weekend Off Peak (7 am to 6 pm)
Tuesday	110 (52%)	153 (73%)	92 (44%)
Wednesday	113 (54%)	150 (71%)	91 (43%)
Thursday	111 (53%)	152 (72%)	96 (46%)
Saturday				170 (81%)
Sunday				158 (75%)

Comparison of Questions on Both Driver and Background Surveys

The background survey was designed to help set the trip in the mind of the driver, as well as obtain general information about the constraints faced by the driver and activities normally undertaken when making the scheduled trip. The driver survey followed up to determine how "typical" this trip was by asking similar questions. The differences between responses to these parallel questions are presented in this section. This includes questions about flexibility in when to make the trip, concerns about on-time arrival, others riding in the vehicle, and typical activities while making the drive. In terms of trip flexibility, the drivers reported having more flexibility on the day of their scheduled drive than in general. This was consistent across both wave 1 and wave 2 reports.

Drivers were also asked how concerned they were about getting to their destination on time. As shown in Figure 8, drivers again report lower levels of concern on the actual day of the drive, compared to "in general" (as reported in the Background survey). One explanation for this may be that many drivers scheduled for a weekend drive also drove during the week. So it may be possible that they filled out the background form thinking about their most typical usage, which is a work trip, while due to the desire to have off-peak observations, they were assigned to a weekend time slot. During wave 2, drivers were somewhat less concerned about on-time arrival than they were for wave 1.

Many in-vehicle factors are known to influence driver satisfaction ratings. This includes whether there is anyone else in the vehicle with them and what their activities are during the actual drive. The differences between "typical" and "actual" with regard to companions, cell phone usage, listening to music, listening to talk shows, eating, and other in-vehicle activities are shown in Figures 9 and 10.

As shown in Figure 9, there was little difference in general vs. actual companions, suggesting that any analysis of the wave 1 and wave 2 results does not need to adjust for having different people in the vehicle with the driver than those who normally accompany.

In terms of activities, the reported activities on the actual drive (for both wave 1 and wave 2) were consistent and lower than the general background information provided. This may be an artifact of people recording on the background form all activities they usually do (but not specific to their evaluation drive) but only recording what was done on the actual drivers on the driver surveys. (Note: the "other" activity category was most often referring to the driver talking with others in the vehicle.).

Driver Ratings

The focus of the study was to obtain documentation on importance and satisfaction ratings by drivers for specific roadway characteristics, and to document the change in those ratings after the installation of the adaptive traffic signal system. The survey instruments were designed to obtain importance ratings as well as satisfaction ratings. Both are important in interpreting the driver's perspective (for example, most drivers reported a high satisfaction rating for landscaping, but it is not really important to them). In this section of the report, the initial importance and satisfaction ratings are presented overall and for drivers on each roadway segment.

Figure 11 shows the overall importance ratings, as well as ratings for Paces Ferry and Spring Road drivers separately. As indicated, the driver importance ratings were very similar and except for two attributes, there was no difference in their ratings. For Paces Ferry drivers, as well as Spring Road drivers, the most important roadway attributes include Traffic Congestion, Driving Behavior of Others and Traffic Signal Coordination, while the least important aspects of their drive are Roadside Landscaping, Lane Width, and Amount of Green Time to Side Streets. There was a statistically significant difference in the ratings for only two attributes; Paces Ferry drivers found Roadside Landscaping and Driving Behavior of others to be less important than the Spring Road drivers did. The difference on Roadside Landscaping may be due to the fact that Paces Ferry Road is a more visually pleasing road to drive on, and so roadside landscaping may not be as important to Paces Ferry drivers (they take it for granted).

In addition, it was interesting to note that drivers rated Traffic Signal Coordination and Amount of Time at Red Lights as more important than Overall Travel Speed, suggesting that drivers on urban arterials prefer continuous movement to higher travel speeds. This confirms the findings of the qualitative study conducted by Flannery, Pecheux and Lappin.

Table 6 shows the satisfaction ratings, recorded after each driver made the wave 1 scheduled drive. Again, the Spring Road and Paces Ferry drivers rated the attributes similarly. Road Pavement Quality, Lane Width, and Pavement Marking Quality received the highest ratings. The Number of Times Stopped by Red Lights, Amount of Time at Red Lights, and Driving Behavior of Others received the lowest ratings. Despite overall similarities, there were some statistically significant differences between the two driver groups. Paces Ferry drivers gave higher ratings to Road Pavement Quality, Driving Behavior of Others, Overall Level of Traffic Congestion, and Overall Travel Speed. These findings are perhaps related to differences in actual conditions. In particular, the recent repaving of the section of Paces Ferry Road near the interstate exchange would explain why Paces Ferry drivers are more satisfied with Road Pavement Quality. Moreover, Spring Road drivers' lower satisfaction rating for Driving Behavior of Others is not surprising given their greater concern for this aspect of their driving experience (as reflected in the greater importance Spring Road drivers assigned to this factor relative to Paces Ferry drivers).

TABLE 6: WAVE 1 DRIVER SATISFACTION RATINGS

(Mean scores)

Attribute	Wave 1
	Paces Ferry	Spring
Road Pavement Quality	5.60	5.25
Pavement Marking Quality	5.31	5.19
Lane Width	5.30	5.30
Availability of Turn Lanes	5.21	5.23
Traffic Congestion	5.12	4.89
Overall Travel Speed	5.11	4.92
Roadside Landscaping	4.76	4.89
Traffic Signal Coordination	4.72	4.66
Green Time for Side Streets	4.71	4.61
Driving Behavior of Others	4.63	4.44
Time at Red Lights	4.49	4.53
# Times Stopped by Red Light	4.40	4.25

Figure 12 shows the "gap" between the importance and wave 1 satisfaction ratings. The value shown is the difference between the importance rating and the satisfaction rating. A value of zero means that the driver's satisfaction with the attribute is equal to its importance to them. A positive value means that the drivers felt that the attribute was important, but were not as satisfied with it (i.e., there is room for improvement in satisfaction). A negative value means that the drivers were very satisfied with the attribute and it received a higher rating than its importance to the driver.

Based on these differences, the survey showed that Roadside Landscaping is not very important to the drivers, but they are very satisfied with what they see. Lane Width is also not important, but drivers are satisfied with it. Driving Behavior of Others is important to these drivers, but they are not very satisfied with it. Similarly, the Number of Times Stopped at a Red Light is important, but they are not very satisfied with it. Amount of Time at Red Lights, Overall Travel Speed and Traffic Signal Coordination fall into this category as well.

For the wave 2 drive, again Paces Ferry and Spring Road drivers rated the attributes similarly. As with the wave 1 survey, Road Pavement Quality, Lane Width, and Pavement Marking Quality received the highest ratings. The Number of Times Stopped By Red Lights, Amount of Time at Red Lights, and Driving Behavior of Others received the lowest ratings (which were the same attributes receiving the lowest ratings in wave 1). The only differences between the two samples mirrored those found in wave 1, where Paces Ferry drivers were more satisfied with Road Pavement Quality, Driving Behavior of Others, and Overall Level of Traffic Congestion. In wave 2, Spring Road drivers were more satisfied with Roadside Landscaping. Given the greater importance of Roadside Landscaping to Spring Road drivers, their satisfaction ratings may be more sensitive to the seasonal changes in the landscaping.

One of the objectives of the study was to measure the change in satisfaction levels after an adaptive traffic signal system is installed. Table 7 shows the mean satisfaction rating with each attribute for Paces Ferry drivers, as well as the net difference in ratings (wave 2 minus wave 1). Overall, satisfaction ratings were similar across the two waves; the only statistically significant differences were increased satisfaction with Lane Width and Roadside Landscaping. The latter can easily be explained by the seasonal variation in when the interviews were conducted; whereas wave 1 was administered in the late fall, wave 2 was administered in the spring, when the landscaping was more attractive.

For many of the roadway attributes, the difference in ratings between wave 1 and wave 2 were relatively small (less than one percent difference). Though not statistically significant, it is worth noting that the decreased satisfaction found for Number of Times Stopped at Red Light and Traffic Signal Coordination was relatively larger than that found for other roadway attributes.

TABLE 7: CHANGE IN DRIVER SATISFACTION — PACES FERRY

Attribute	Wave 1		Wave 2		Difference	% Difference
	Mean	SE Mean	Mean	SE Mean
Lane Width	5.30	0.07	5.48	0.07	0.18	3.4%
Road Pavement Quality	5.60	0.06	5.59	0.06	-0.01	-0.2%
Pavement Marking Quality	5.31	0.07	5.30	0.07	-0.01	-0.2%
Roadside Landscaping	4.76	0.07	5.04	0.07	0.28	5.9%
Driving Behavior of Others	4.63	0.08	4.64	0.07	0.01	0.2%
Traffic Congestion	5.12	0.08	5.02	0.07	-0.10	-2.0%
# Times Stopped by Red Light	4.40	0.09	4.26	0.08	-0.14	-3.2%
Time at Red Lights	4.49	0.09	4.38	0.08	-0.11	-0.4%
Green Time for Side Streets	4.71	0.08	4.68	0.07	-0.03	-0.6%
Traffic Signal Coordination	4.72	0.09	4.57	0.08	-0.15	-3.2%
Overall Travel Speed	5.11	0.07	5.07	0.07	-0.03	-0.8%
Availability of Turn Lanes	5.21	0.08	5.20	0.08	-0.01	-0.2%
Overall	5.11	0.07	5.14	0.06	0.03	0.6%

Table 8 presents wave 1 and wave 2 ratings for Spring Road drivers. Similar to findings for Paces Ferry Road, there was a significant increase in satisfaction for Roadside Landscaping. This was the only attribute whose overall satisfaction level changed statistically from wave 1 to wave 2. For Spring Road drivers, net satisfaction levels for Roadside Landscaping increased by 12%, a finding easily explained by the seasonal difference in survey administration. As originally hypothesized, then, drivers on the control route rated the roadway attributes similarly across both waves.

TABLE 8: CHANGE IN DRIVER SATISFACTION — SPRING ROAD

Attribute	Wave 1		Wave 2		Difference	% Difference
	Mean	SE Mean	Mean	SE Mean
Lane Width	5.30	0.09	5.41	0.09	0.11	2.1%
Road Pavement Quality	5.25	0.09	5.22	0.09	-0.03	-0.6%
Pavement Marking Quality	5.19	0.09	5.30	0.09	0.10	2.1%
Roadside Landscaping	4.89	0.10	5.48	0.09	0.59	12.1%
Driving Behavior of Others	4.44	0.11	4.35	0.10	-0.09	-2.0%
Traffic Congestion	4.89	0.11	4.80	0.10	-0.10	-1.8%
# Times Stopped by Red Light	4.25	0.12	4.30	0.11	0.06	1.2%
Time at Red Lights	4.53	0.11	4.56	0.10	0.04	0.7%
Green Time for Side Streets	4.61	0.10	4.62	0.10	0.01	0.2%
Traffic Signal Coordination	4.66	0.11	4.76	0.11	0.09	2.1%
Overall Travel Speed	4.92	0.09	5.01	0.09	0.09	1.8%
Availability of Turn Lanes	5.23	0.09	5.32	0.09	0.09	1.7%
Overall	5.01	0.08	5.10	0.08	0.08	1.8%

As detailed in the introductory section to this report, the study was structured to measure changes in driver satisfaction based on the installation of an adaptive traffic signal system. The null hypothesis was that there would be no change in satisfaction and the results suggest that for Paces Ferry and Spring drivers as a whole, there was no statistical change in driver satisfaction, resulting in the acceptance of the null hypothesis.

Despite the null findings, analysis was undertaken to investigate changes in satisfaction by time of drive, as it was originally anticipated that off-peak drivers would be more likely to experience an improvement in their drive. In particular, this analysis focused on the key roadway factors related to adaptive traffic signal systems (including Number of Times Stopped by a Red Light, Amount of Time at a Red Light, Amount of Green Time to Side Streets, and Traffic Signal Coordination). Contrary to expectations, off-peak drivers were less satisfied with Traffic Signal Coordination in wave 2 (compared to wave 1), though there was no significant change on the other three measures of interest (see Figure 13).

For peak drivers there was no statistically significant change in satisfaction for Number of Times Stopped at a Red Light, Time at Red Lights, Green Time for Side Streets, or Traffic Signal Coordination.

Why Was There No Change in Driver Satisfaction?

In order to better understand the survey findings, the evaluation team considered potential factors that might explain why there was no change in driver satisfaction on Paces Ferry Road. One possibility is that the lack of change in driver satisfaction might be due to a change in the drive conditions or driver constraints. Several characteristics of the drive were measured in both the wave 1 and wave 2 driver surveys, including level of traffic congestion (relative to normal), concern about on-time arrival, and degree of schedule flexibility. So, for example, it is possible to determine if drivers were experiencing greater traffic congestion in wave 2 versus wave 1, or if they were more concerned about on-time arrival. Changes in these measures might affect their satisfaction ratings. However, these measures were fairly consistent across both waves of the study, and there were no significant changes in schedule constraints or reported levels of traffic congestion. In other words, these factors should not explain why drivers were not more satisfied.

In addition, in each wave of the study, drivers were also asked if their scheduled drive on Paces Ferry Road had been "typical" or not. In order to rule out the possibility that respondents with "atypical" drives had skewed the overall satisfaction ratings, the wave 1 and wave 2 satisfaction ratings were compared only among those drivers who said their drive was typical ⁶. However, the findings from this analysis replicated the overall study findings. That is, among those who reported having typical drives, the only significant change in satisfaction across the two waves was for "Lane Width" and "Roadside Landscaping."

TABLE 9: CHANGE IN DRIVER SATISFACTION ON PACES FERRY

(AMONG DRIVERS WHO SAID THEIR DRIVE WAS TYPICAL)

Attribute	Wave 1		Wave 2		Difference	% Difference
	Mean	SE Mean	Mean	SE Mean
Lane Width	5.30	0.08	5.49	0.08	0.19	3.6%
Road Pavement Quality	5.64	0.07	5.61	0.07	-0.03	-0.6%
Pavement Marking Quality	5.34	0.08	5.30	0.08	-0.04	-0.7%
Roadside Landscaping	4.70	0.09	5.06	0.08	0.36	7.61%
Driving Behavior of Others	4.58	0.09	4.55	0.08	-0.03	-0.6%
Traffic Congestion	4.97	0.09	4.98	0.08	0.01	0.2%
# Times Stopped by Red Light	4.35	0.10	4.24	0.09	-0.11	-2.5%
Time at Red Lights	4.44	0.10	4.37	0.09	-0.07	-1.6%
Green Time for Side Streets	4.68	0.09	4.66	0.08	-0.02	-0.4%
Traffic Signal Coordination	4.69	0.10	4.58	0.09	-0.11	-2.3%
Overall Travel Speed	5.07	0.08	5.06	0.07	-0.01	-0.2%
Availability of Turn Lanes	5.19