Oregon Statewide Transit Trip Planning

Evaluation Plan

PDF Version 1.19MB

Prepared for: U.S. Department of Transportation Dr. Joseph Peters, Joint Program Office Brian Cronin, Federal Transit Administration

Prepared by: SAIC 8301 Greensboro Drive McLean, Virginia 22102

NOTICE This document is disseminated under the sponsorship of the Department of Transportation in the interest of information exchange. The United States Government assumes no liability for its contents or use thereof. This report does not constitute a standard, specification, or regulation. The United States Government does not endorse products or manufacturers. Trade and manufacturers’ names appear in this report only because they are considered essential to the object of the document.

TABLE OF CONTENTS
LIST OF TABLES
LIST OF FIGURES
ABBREVIATIONS

1. Introduction

• 1.1  Background
• 1.2  Project Overview
• 1.3  Evaluation Objectives
• 1.4  Organization of the Report

2. Background

• 2.1  Project Context
• 2.2  Existing Information Systems
• 2.3  Supporting Programs
  • 2.3.1    State Agency Transportation Coordination Project
  • 2.3.2    InterCity Passenger Transportation Program
  • 2.3.3    Special Transportation Fund Program
  • 2.3.4    Transportation Demand Management Program
  • 2.3.5    Small Cities and Rural Areas Transit Assistance Program
  • 2.3.6    Department of Human Services Transportation Brokerages Program
• 2.4  Requirements Workshops

3. Trip Planner Project Description

• 3.1  Overview
• 3.2  Functional Requirements
• 3.3  Technical Scope
• 3.4  Expected Benefits
  • 3.4.1    Improved Customer Satisfaction
  • 3.4.2    Statewide Coordination of ODOT and Oregon DHS Transportation Resources
  • 3.4.3    Increased Efficiency and Utilization
  • 3.4.4    Enhanced Coordination of Connecting Public Transportation Services
• 3.5  Project Management
• 3.6  Project Stakeholders
  • 3.6.1    Data Providers
  • 3.6.2    System Users
• 3.7  Deployment Schedule

4. Evaluation Approach

• 4.1  Overview
• 4.2  System Impact Study
  • 4.2.1    Availability and Use of Web-Based Public Transportation Information
  • 4.2.2    Customer Satisfaction with Public Transportation Information
  • 4.2.3    Coordinated Use of ODOT and DHS Transportation Resources
  • 4.2.4    Use of Public Transportation Services
  • 4.2.5    Coordination of Connecting Services
• 4.3  Detailed Test Plans
• 4.4  Institutional Challenges Review

5. Risk Assessment

• 5.1  Overview
• 5.2  Scope Risk
• 5.3  Schedule Risk

6. Evaluation Management Plan

• 6.1  Overview
• 6.2  Level of Effort
• 6.3  Deliverables
• 6.4  Evaluation Schedule

Appendix A – TripCheck Website Usage Statistics
Appendix B – TripCheck Public Transportation Web-Based Information Inventory  
References
Internet Resources

LIST OF TABLES

Table 1-1.  Trip Planner Development Phases
Table 2-1.  National Transit Database Statistics for Oregon Transit Agencies
Table 2-2.  Oregon Transportation Brokerages
Table 2-3.  Stakeholder Workshops
Table 3-1.  Trip Planner Release 1.0 Functional Requirements
Table 3-2.  Steering Committee and Technical Advisory Committee
Table 3-3.  Trip Planner Data Providers and Required Data Elements
Table 4-1. Trip Planner Project Evaluation Goals, Hypotheses, MOEs, and Data Sources
Table 4-2.  System Impact Study Detailed Test Plans Overview
Table 4-3.  Detailed Test Plan Activities
Table 6-1.  Evaluation Organizational Overview
Table 6-2.  Level of Effort
Table 6-3.  Evaluation Schedule
Table A-1.  TripCheck Website Usage for busrail.htm 6/01 – 2/03
Table B-1.  TripCheck Public Transportation Web-Based Information Inventory

LIST OF FIGURES

Figure 2-1.  Urban and Rural Areas of Oregon
Figure 2-2.  Oregon Intercity Connections
Figure 2-3.  Sample Tri-Met Trip Planner Request Web Page
Figure 2-4.  Sample Tri-Met Trip Plan Web Page
Figure 2-5.  TripCheck Bus and Rail Directory of Public Transportation Providers
Figure 2-6.  Transportation Brokerage Areas in Oregon
Figure 2-7.  Requirements Workshop in Salem, Oregon
Figure 3-1.  Concept of Operations – Referral Agent as User
Figure 3-2.  Concept of Operations – General Public as User
Figure 4-1.  TripCheck Usage Trend for Public Transportation Information
Figure 4-2.  Aspects Tracked for the Web-Based Information Inventory

ABBREVIATIONS

1.      Introduction

1.1       Background

The Oregon Statewide Transit Trip Planning Project[1] is funded in part by the FY2001 Intelligent Transportation Systems (ITS) Integration Program, a program designed to accelerate the integration and interoperability of ITS solutions in metropolitan and rural areas. Projects funded through the ITS program have been assessed as supporting the improvement of transportation efficiency, promoting safety, increasing traffic flow, reducing emissions of air pollutants, improving traveler information, enhancing alternative transportation modes, building on existing ITS projects, and/or promoting tourism.

Oregon’s Trip Planner was one of several projects selected for national evaluation. Science Applications International Corporation (SAIC) was selected as the independent evaluator (Evaluation Team) for this project. The Evaluation Team is responsible for preparing this Evaluation Plan document, which outlines the approach to be used for project evaluation, and to provide program managers at the United States Department of Transportation (USDOT) with information needed to assess project progress.

1.2       Project Overview

According to the Oregon Department of Transportation (ODOT), “nearly one third of all Oregonians – primarily the elderly, people with disabilities, and lower income families – have trouble getting around.”[2] A lack of information about available public transportation services is one barrier to both the effective use of those services and the mobility of Oregon’s citizens. ODOT summarizes the purpose of the Trip Planner project as follows:

“The Trip Planner is intended to make public transportation of all kinds easier to use by increasing the availability, quality, and amount of information about these services, and improving the ability of travelers to plan a trip, especially using multiple transportation providers.”[3]

In 2000 and 2001, ODOT assessed the opportunities, issues, risks, and costs involved in implementing a statewide transit trip planning system. Three documents were generated from this effort: a review of existing trip planning systems; a survey of public transportation agencies in Oregon regarding existing plans and capabilities for trip planning; and a recommended approach to developing a statewide trip planning system.

After completing this assessment, ODOT adopted an implementation strategy that incorporated two key elements: first, to leverage the existing exposure of the TripCheck traveler information Website (www.tripcheck.com); and second, to use an incremental build strategy beginning with an enhanced Web portal (Release 1.0) and evolving to an itinerary trip planner (Release 2.0). The incremental build strategy reflects ODOT’s funding constraints. Table 1-1 identifies the development phases and the status of each.

Table 1-1.  Trip Planner Development Phases

In 2002, ODOT released a request for proposals for the design and deployment of Release 1.0. A vendor was selected and work began in November 2002. The design phase is scheduled for completion in June 2003 and the project plan currently calls for Release 1.0 of the Trip Planner to be deployed in October 2004. Release 2.0 of the Statewide Trip Planner currently is not funded. This Evaluation Plan addresses only the design and development of Release 1.0.

Release 1.0 will offer various map and text-based tools for directing transit users to the most appropriate services to meet their travel needs. The system will be accessible through the Oregon TripCheck traveler information Website (www.tripcheck.com), and serve as a single source of current transit information for the more than 200 public transportation service providers in Oregon. Release 1.0 is referred to as a “zone-to-zone” trip planner that will allow users to select an origin area (or “zone”) and a destination zone. The trip planner application will identify the connecting public transportation service providers that can meet a user’s trip needs. Not until Release 2.0 (currently not funded) will the trip planner application have the capability of generating detailed point-to-point trip plans.[4] However, Release 1.0 work includes creating the core data infrastructure, institutional agreements, and data maintenance processes and tools that will support Release 2.0 functionality.

The Washington State Department of Transportation (WSDOT) is also participating in the design phase of the Trip Planner project; however, that aspect of the project is not being addressed by this evaluation effort. WSDOT is currently working to identify funding for development of trip planning capability in Washington with the intent of eventually integrating the Oregon and Washington trip planners. This integration will be particularly useful in serving border communities such as the Portland, Oregon – Vancouver, Washington metropolitan area.

1.3       Evaluation Objectives

As mentioned above, this Evaluation Plan addresses only Release 1.0 of the Trip Planner – the Web Clearinghouse/Zone-to-Zone Trip Planner. There are several objectives for conducting an evaluation of this project. First, an evaluation of the system impacts following deployment will contribute to a better understanding of the value and effectiveness of this type of ITS solution, thereby aiding other communities and stakeholders in making future investment decisions. Ridership impacts are of particular interest to USDOT. In addition, identifying and reviewing the associated institutional challenges will provide valuable “lessons learned” for other communities planning to implement this type of system. USDOT has special interest in the lessons learned from coordination with health and human services delivery and from the use of the Transit Communication Interface Profiles (TCIP). Ideally, the results of the evaluation will make it possible to optimize this and future deployments of statewide trip planning systems.

Although there is increasing interest nationwide in implementing Web-based trip planners, there is almost no research supporting the expected benefits of implementing such a system. In a report about the state of the practice in trip planning published in 2002 by the Federal Transit Administration (FTA), it was noted that by implementing Web-based trip planning systems, surveyed agencies expected to save money, provide better service, increase ridership, and make geo-coded data available to other applications. However, the study authors reported that few evaluations were available to support these expectations, and that “there were no reports, either quantitative or qualitative, of ridership increases resulting from trip planners.”[5]

However, a computerized trip planning system in London, designed for use by transit agency staff (not a Web-based application and not available to the general public) is credited with influencing just over 10%[6] of those who contacted London Transport for trip planning assistance to make a trip that they would not otherwise have made by public transport.[7]

A second reason for interest in this particular application of trip planning is the statewide geographic scope of the project. At the time of the FTA report, there were 30 Web-based transit trip planners operational in the United States. Only eight of these were multi-agency, and only two of the eight included more than 10 agencies and the multi-agency applications were for metropolitan regions – not for entire states. The Oregon Trip Planner is unique in that it is a statewide effort including nearly 200 agencies in both urban and rural environments.

The third reason, and perhaps the most significant, for interest in this trip planner deployment is that it is being accomplished in cooperation with Oregon’s Department of Human Services (DHS). In fact, Release 1.0 will be targeted to users in agencies that assist beneficiaries of the various social services programs in the state. The intent is to try to reduce the cost of providing transportation to these clients by coordinating the state’s transportation resources – an effort that will be supported by the Trip Planner’s functionality.

1.4         Organization of the Report

The remaining sections of this Evaluation Plan document are organized as follows:

• Section 2.  Background. This section provides additional background about the context into which the project is being implemented including an overview of the existing transportation service providers, existing information systems and supporting programs.
• Section 3.  Trip Planner Project Description. This section describes the Trip Planner project in terms of functional requirements, technical scope, expected benefits, project management, stakeholders, and deployment schedule.
• Section 4.  Evaluation Approach. This section presents the evaluation activities to be performed as part of the System Impact Study and the Institutional Challenges Review for the Trip Planner project.
• Section 5.  Risk Assessment. This section presents an assessment of the risks that may affect the meaningfulness and timeliness of the evaluation results.
• Section 6.  Evaluation Management Plan. This section provides an overview of the Evaluation Team’s organization, the level of effort, deliverables and schedule.

Following Section 6 are two appendices. Appendix A includes TripCheck usage statistics for the public transportation information pages and Appendix B is an inventory of the Web-based public transportation information currently available through TripCheck. Reference lists of project documents and Internet resources used in developing this Evaluation Plan document follow the appendices.

2.      Background

2.1       Project Context

The vast majority of land in Oregon is considered rural. Nearly 2.5 million of the state’s 3.5 million people live in the urban counties along the I-5 corridor in the western portion of the state.[8] Figure 2-1 shows the location of Oregon’s cities.

Figure 2-1.  Urban and Rural Areas of Oregon

A significant challenge for the Trip Planner project is the to integrate transportation service information from rural and urban agencies with very different service areas and customer needs. According to ODOT, approximately 200 separate agencies, varying in size from very small demand-responsive services to large urban transit systems and multi-state intercity carriers, are providing public transportation services in Oregon[9]. The bulk of these are providing demand response service for the general public or special needs riders in rural areas.

Four of the public transportation service providers in Oregon are large enough to report annual operating statistics to the National Transit Database. As shown in Table 2-1, the size of these agencies in terms of passenger miles operated ranges from a high of over 355 million to a low of just over 1 million.[10]

Table 2-1.  National Transit Database Statistics for Oregon Transit Agencies

In addition, ODOT has made supporting intercity mobility a priority through the InterCity Passenger Transportation Program, a program through which the availability of intercity services is closely monitored. Figure 2-2 shows the network of intercity connections that are currently being served by either long distance bus or train.[11]

 

Figure 2-2.  Oregon Intercity Connections

2.2       Existing Information Systems

Within Oregon there are already three point-to-point Web-based trip planners in operation. Greyhound (www.greyhound.com) and Amtrak (http://www.amtrak.com) have systems that allow station-to-station trip planning for anywhere in the United States, including Amtrak’s thruway bus connections and Greyhound’s rural interlined services. These Websites also allow users to make reservations and users can purchase tickets online from Amtrak. In the Portland metropolitan area, Tri‑Met, the bus and light rail transit provider, operates a sophisticated Web-based point-to-point trip planning system (www.trimet.org) that generates trips to and from any address, intersection, or landmark in the Tri-Met service area. The information provided by the system includes walking directions to transit stops. Transit riders can also purchase ticket books and passes on the Tri-Met Website.

Figure 2-3 shows a sample Tri-Met trip planner request screen for a user requesting a trip to a local mall. The user has selected parameters regarding the starting point, destination, desired date and time of arrival, and a preference for arriving in the “quickest time” with less than a half-mile walk from the point of exiting the selected transit service. Figure 2-4 shows the trip plan automatically generated for the user.

Figure 2-3.  Sample Tri-Met Trip Planner Request Web Page

Figure 2-4.  Sample Tri-Met Trip Plan Web Page

ODOT currently maintains two Web-based lists of public transportation service providers and their contact information. One service provider list is maintained as part of the TripCheck Website (http://www.tripcheck.com/About/aboutindex.htm). ODOT’s Public Transit Division (PTD) maintains the other list (http://www.odot.state.or.us/pubtrans/directory.html).

Figure 2-5 shows a portion of the list on the TripCheck Website relating to bus and rail information. In some cases, only a phone number is provided; in others, a link to the service provider’s Website is included; and for still others, there is a link to schedule information provided by TripCheck. As of the end of January 2003, 18 out of the 40 public transportation service providers listed on TripCheck had a Website available. (See Appendix B for a detailed list of the Web-based information available for transit service providers in Oregon.). For January and February 2003, visits to the TripCheck list of transit providers averaged just over 720 visits per month. (See Appendix A.)

Figure 2-5.  TripCheck Bus and Rail Directory of Public Transportation Providers

2.3       Supporting Programs

The Trip Planner project has the advantage of being an integral part of several state‑level programs and high-profile initiatives. Consequently, its progress is being monitored closely and a broad range of stakeholders is committed to its success. The Trip Planner is envisioned as an important tool for the implementation of the programs described in Sections 2.3.1 through 2.3.6.

2.3.1        State Agency Transportation Coordination Project

This project is the result of an initiative by the Governor to address unmet special transportation service needs via better coordination of existing transportation resources. The Governor requested that representatives from state agencies serving as primary investors in public, student, and client transportation, develop a strategy for coordinating transportation resources. ODOT along with other key state agencies including DHS; Education; Veterans; Corrections; Employment, Housing, and Community Services; and Administrative Services are all project participants.

2.3.2        InterCity Passenger Transportation Program

This program is the city-to-city piece of the Oregon Transportation Network. This program coordinates public and private transportation providers to offer service between cities. The Intercity program focuses on three main program areas supporting statewide connectivity and seamless customer experiences: information dissemination; coordination of service connections; and service-level standards.

2.3.3        Special Transportation Fund Program

This is the Public Transportation Division’s largest program. It provides grants to improve the quantity and quality of transportation services available to Oregon's elderly and disabled residents. Grants are awarded to counties, transit districts, and transportation service providers.

2.3.4        Transportation Demand Management Program

This program assists communities with the development of services and facilities for alternative transportation methods, including rideshare programs, car share, park and ride lots; telecommuting programs; and incentive programs to encourage the use of alternatives to driving alone.

2.3.5        Small Cities and Rural Areas Transit Assistance Program

This program provides grant funds for operating and capital purchases for general public transportation services in rural areas and cities with a population of less than 50,000 people. The program goals are two-fold: to ensure that individuals living in rural and small urban areas have access to transit to meet basic mobility needs; and to enhance the access of individuals living in rural areas to health care, shopping, education, employment, public services and recreation.

2.3.6        Department of Human Services Transportation Brokerages Program

DHS reports that clients identify transportation as one of the biggest challenges to getting to social services. Because of this finding, DHS is aggressively moving to set up regional transportation brokerages to coordinate transportation resources and manage eligibility and payment requirements. The goal of the transportation brokerage program is to coordinate existing transportation resources as a means of increasing efficiency, accountability, and responsiveness.

DHS programs requiring transportation include the following:

• Oregon Medical Assistance Program
• Seniors and Disability Services Waivered Program, a senior program that provides non‑medical rides to allow users to remain in their own homes rather than be institutionalized.
• Developmental Disabilities Waivered Program, a program for individuals with developmental disabilities that provides non-medical rides to allow users to remain in their own homes rather than being institutionalized.
• Jobs Programs, including welfare-to-work programs for adults and Job Access Reverse Commute.

Table 2-2 lists the status of the transportation brokerages in Oregon and Figure 2-6 shows the Oregon counties that are currently or will soon be served by brokerages.

Table 2-2.  Oregon Transportation Brokerages

Figure 2-6.  Transportation Brokerage Areas in Oregon

2.4       Requirements Workshops

In January 2003, ODOT hosted a total of eight stakeholder workshops, four in Oregon and four in Washington.[12] The purpose of the workshops was to obtain stakeholder feedback on proposed Trip Planner system objectives, functionality, data elements, data maintenance procedures, and implementation priorities. Each workshop focused on the needs of specific groups as shown in Table 2-3. The workshop results are reflected in the functional requirements described in Section 3. The results were also useful in the development of this Evaluation Plan.

Figure 2-7.  Requirements Workshop in Salem, Oregon

Table 2-3.  Stakeholder Workshops

3.      Trip Planner Project Description

3.1       Overview

Release 1.0 of the Statewide Trip Planner is envisioned to have two basic functions. The first is to serve as a “Web-based clearinghouse” to provide users with a comprehensive view of Oregon’s public transportation service providers, along with phone numbers or links to Websites through which users may contact those providers directly. The other function is to serve as a “zone‑to-zone” trip planner through which users will be able to specify an origin area, a destination area and other trip requirements, such as time of day and accessibility needs. The user will then receive information about the service providers for the origin area, the destination area, and any required connecting carriers between the two. Using route and schedule information available from the Trip Planner Website, the user then determines which route and trip or service to use. To support this functionality, a central repository for schedule and route information for all service providers will be created along with tools for data providers to enter and maintain the data.

This section of the report describes the functional requirements, technical scope, expected benefits, project management structure, stakeholder roles, and deployment schedule for Release 1.0 of the Statewide Trip Planner project.

3.2       Functional Requirements

The functional requirements for Release 1.0 are presented in Table 3-1.

Table 3-1.  Trip Planner Release 1.0 Functional Requirements

Figures 3-1 and 3-2 illustrate the two primary scenarios for the concept of operations for Release 1.0. The first is with the referral agent as the user. Referral agents are those who make transportation arrangements for others. Examples include staff at transit call centers, transportation brokerages, social service caseworkers, and community organizations. The second scenario is with a member of the general public as the user.

In Scenario 1 (Figure 3-1), a social services client (a beneficiary of various social services agencies) requests travel planning assistance from a referral agent who then uses the Trip Planner. If the trip will use a service requiring reservations or scheduling in advance (demand response service, intercity bus or train for example) the referral agent contacts the service provider directly to schedule or reserve a trip for the client. If the trip will use services that do not require that the rider made arrangements in advance (standard local fixed-route services for example), the referral agent gathers the necessary schedule information from the service provider’s Website (linked to the Trip Planner) or telephone information line. Once the referral agent has the travel information, he or she passes it along to the client who then makes the trip.

Figure 3-1.  Concept of Operations – Referral Agent as User

In Scenario 2 (Figure 3-2), a member of the general public accesses the Trip Planner directly to find out which service providers meet the requirements of his or her trip. If the trip will use services that require reservations or scheduling in advance, the general public person contacts the service provider directly to reserve or schedule the trip. If the trip will use services that don’t require reservations or scheduling in advance, the general public person gathers the necessary schedule information from the service provider’s Website (linked to the Trip Planner) or telephone information line. The general public person is then prepared to make the trip.

Figure 3-2.  Concept of Operations – General Public as User

3.3       Technical Scope

ODOT has identified the following six technical work elements required to complete Release 1.0:

1. Enhance the TripCheck Website to provide comprehensive statewide transportation service information. Since TripCheck is a popular traveler information Website, ODOT desires to leverage this established Internet infrastructure, current user base, and existing publicity mechanisms.
2. Conduct a package evaluation to determine if existing trip planner software solutions would be a better investment than developing a new system.
3. Develop a zone-to-zone trip planner based on a geographic information system (GIS) architecture, and integrate it with the TripCheck Website.
4. Develop an open system architecture and design consistent with National and Oregon ITS standards. This includes the design of the network and communication architecture.
5. Develop a database and supporting tools necessary to accommodate the system data.
6. Develop data collection interfaces to support data capture from the transportation service providers. Large service providers have indicated a preference for putting their data into an ODOT- prescribed format. Smaller agencies will be able to update their data in the Trip Planner system using a specially developed Web-based tool.

3.4       Expected Benefits

Release 1.0 is not intended to be the more traditional type of fully functional point-to-point trip planner. The target audiences and expected benefits vary from those that might be expected for a more traditional application. ODOT has identified the following four primary expected benefits. These form the basis of the evaluation hypotheses identified in Section 4.

3.4.1        Improved Customer Satisfaction

ODOT expects that implementation of the Trip Planner will result in improved customer satisfaction with the quality and availability of public transportation system information. “Availability” includes convenience, quantity of information available, usability, and understandability. A key objective of this system is to provide users with enough information to easily plan and execute their desired travel.

It is expected that the users receiving the most benefit from the new system in this area will be the referral agents, those who coordinate trips for others. As such, the primary target audiences for Release 1.0 are transit agency call center staff  (fixed-route and demand-response services), social services caseworkers, community service agencies, and personnel at the growing number of transportation brokerages in Oregon.

3.4.2        Statewide Coordination of ODOT and Oregon DHS Transportation Resources

It is also anticipated that Release 1.0 can help reduce the cost of providing transportation services for beneficiaries of various social services programs. Oregon currently spends $15 million for transportation benefiting seniors and the disabled, and another $40 million for non-emergency medical transportation of human services clients.[13] It is anticipated that these costs may be reduced once the brokerage staff and caseworkers who schedule the transportation services become more aware of all the public transportation services available. This is particularly true for intercity trips where volunteer drivers are often reimbursed to take people to a neighboring city or regional center, rather than scheduling the trip on an existing, but perhaps unfamiliar, intercity service provider.

A 2002 assessment of special transportation needs among persons 60 years of age and older and persons with disabilities in three Oregon counties provides some insight into needs that can be addressed by the Trip Planner.[14] The study involved mail-back surveys, telephone surveys, public meetings, and focus groups for transportation providers, social service administrators, and residents in the tri-county area. Although lack of transportation services emerged as the most often referenced unmet need, there were some useful results relevant to providing increased access to information about public transportation services as reported in the following results:

• In a survey of Department of Health caseworkers, adult foster home operators, and medical clinic patients, 72 percent indicated “offering printed schedules that are easier to understand” was a very important improvement.
• Of those residents surveyed who do not use dial-a-ride services regularly, 75 percent indicated it was because they had no need of those services; however 3 percent indicated that they didn’t use the services because they didn’t know about the system or how it operates.
• Several participants at all three public meetings mentioned lack of information about available transportation services as a shortcoming of current programs.

Six priorities for improvement emerged from the study, one of which was stated as follows:

“Implement measures to increase public awareness of transportation systems and how to use them for it is possible that additional needs could be served with the existing system if more of those in need knew exactly what services are available and how to take advantage of them. New or enhanced services will probably lack ridership unless appropriate public awareness efforts are undertaken.”[15]

3.4.3        Increased Efficiency and Utilization

As a result of improved access to schedule, route, and service area information, public transportation service providers are expected to experience increased effectiveness in terms of the use of the services they offer. In particular, intercity service providers are expected to benefit from the Trip Planner system implementation due to increased awareness of the availability of these services.

3.4.4        Enhanced Coordination of Connecting Public Transportation Services

The Trip Planner is expected to provide transportation service providers with the comprehensive information necessary to better coordinate their services with connecting services. It is expected that this coordinated approach will lead to a more integrated statewide system of public transportation services.

3.5       Project Management

In a somewhat unique arrangement for ODOT, representatives of three separate departments within ODOT have combined to manage the Trip Planner project. The technical project manager is from the Information Systems Division; the project owner and system manager are from the ITS Group, and the subject matter expert leads are from the Public Transportation Division. Each of these organizational units brings a unique perspective to the project. An outsourced systems developer/integrator is contracted to ODOT to perform the technical tasks. Roles and responsibilities for all participants in the project have been clearly defined in the project documentation.[16]

The Project Management Team is relying on two multi-agency committees to support their efforts: a Steering Committee and a Technical Advisory Committee.[17] The Steering Committee meets as needed to approve system deliverables, build consensus on system design features, assist in outreach efforts, and to help resolve project issues.

The Technical Advisory Committee was established to review systems engineering documents; provide recommendations on key architecture, design, and implementation issues; address cross-platform technology issues; and identify legal, institutional, and resource constraints to the use of proposed technology strategies.[18] Table 3-2 provides an overview of the multiple agency representatives serving on the two committees.

Table 3-2.  Steering Committee and Technical Advisory Committee

Steering Committee Members	Technical Advisory Committee Members
Amtrak	Amtrak
Broadway Cab	Contractor
C-Tran	Greyhound
City of Bend	King County Metro
City of Hood River	Lane Transit District
City of Portland	Link Transit
Tillamook County Transportation District	ODOT
DHS	Rural Resources Community Action
Disabilities Commission	Salem Area Mass Transit
Federal Highway Administration (FHWA)	Tri-Met
Greyhound	Washington Assistive Technology Alliance
Lane Transit District	WSDOT
ODOT
Rogue Valley Transit
Salem Area Mass Transit
SMART
Tri-Met
Wheels
WSDOT

3.6       Project Stakeholders

Aside from those stakeholders represented on the Project Management Team (ITS and PTD) and the Steering and Technical Advisory Committees, there are two other main groups of stakeholders. The first is the set of transportation service providers that will provide data to the system; the second is the system users. Both are described in the sections below.

3.6.1        Data Providers

The data supporting the Trip Planner will come from the following transportation service providers:

• Local fixed-route transit providers such as Tri-Met in Portland and Lane Transit in Eugene, Oregon.
• Local demand-response transit providers, such as Wheels and Tri-Met LIFT.
• Long-distance bus services, such as Greyhound and Valley Retriever.
• Long-distance passenger rail services, such as Amtrak.
• Taxi and shuttle services, such as Broadway Cab in Portland and HUT Shuttle.

In addition to general fare, hours of operation and contact information, the data providers will be asked to provide and maintain geographic, temporal, and descriptive information for the required data elements shown in Table 3-3.

Table 3-3.  Trip Planner Data Providers and Required Data Elements

3.6.2        System Users

The system users are expected to fall into two categories: the general public and referral agents. Referral agents are those who help others make travel arrangements. This evaluation effort will put emphasis on referral agent users of the system because these are the users likely to benefit most from the functionality of Release 1.0. The referral agents of interest include the following agency representatives:

• Transit Call Center Staff – These individuals answer the phone at transit agencies and provide route and schedule information or schedule demand-response trips. Although well versed in the services their organizations offer, they may have very little access to information about connecting services. Release 1.0 is expected to provide transit call center staff with reliable service area, eligibility requirement, route, and schedule information for service providers throughout the state.
• Social Services Caseworkers – These individuals work directly with the clients of many social service programs to arrange necessary services including transportation. Caseworkers tend to be familiar with many of the local public transportation services available, but they may not be aware of all the services that could meet their clients’ transportation needs.
• Community Service Organization Staff – Many community service organizations such as senior housing, assisted-living facilities, and community centers are asked to assist with making travel arrangements. Release 1.0 is expected to be very helpful in identifying those public transportation service providers that can meet trip requirements.
• Transportation Brokerage Staff – Transportation brokerages are being established throughout Oregon to coordinate non-emergency medical transportation for clients of various social service agencies. Brokerages are centralized call centers that confirm client eligibility, inquire about transportation resources, and organize the most appropriate and lowest cost rides for clients. Release 1.0 is expected to help brokerage staff plan cost-effective trips to take advantage of the many publicly supported transportation service providers.

It is because of these last three groups of system users that DHS administrators are taking an active role in the design of Release 1.0. DHS sees the Trip Planner as a means to improve mobility by improving awareness of transportation services and access to information about those services.

3.7       Deployment Schedule

The preliminary design phase began in July 2001 and will continue through the end of June 2003. System development is expected to begin in October 2003. Implementation is scheduled for a three-month period starting July 2004 and completing in October 2004.

4.      Evaluation Approach

4.1       Overview

The evaluation of the Trip Planner project will involve two components: a study of system impacts and a review of institutional challenges. The starting point for the Systems Impact Study is the set of expected benefits and target stakeholders identified by ODOT. Associated hypotheses have been formulated and the measures of effectiveness (MOEs) and data sources to be used have been proposed and are described in Section 4.2. Details of the individual tests and test activities are provided in Section 4.3. Section 4.4 includes a plan for review of the anticipated institutional challenges pertaining to the Trip Planner project.

4.2       System Impact Study

The System Impact Study will address project goals in the following five areas:

• Availability and use of Web-based public transportation information.
• Customer satisfaction with public transportation information.
• Efficient use of public transportation resources.
• Use of public transportation services.
•  Coordination of connecting services.

Table 4-1 summarizes the goals, target users, hypotheses, measures of effectiveness (MOEs), and data sources for the System Impact Study.

4.2.1        Availability and Use of Web-Based Public Transportation Information

ODOT expects that as a result of the Trip Planner project more transportation service providers will have more information available in a Web-based format. In addition, it is anticipated that the project will result in a greater number of visits to the TripCheck public transportation information pages. The following two hypotheses will be tested regarding this goal area:

• Because of the Trip Planner project, more transportation service information will be available on the Web.
• Because of the Trip Planner project there will be more visitors to the public transportation information pages on the TripCheck Website.

Inventories will be created for the TripCheck public transportation pages before and after the new system is implemented. A review of these inventories will provide data regarding the number of transportation service providers with Web-based information and the level of detail of that information. In addition, the Evaluation Team will use statistics reported through ODOT’s existing WebTrends automated usage reporting tool to determine if there is a statistically significant increase in usage of TripCheck’s public transportation pages. (See Section 4.3 Detailed Test Plans for Tests #1 and #2.)

4.2.2        Customer Satisfaction with Public Transportation Information

Through this project, ODOT hopes to improve customer satisfaction with the quality and availability of public transportation information. The specific customers of interest are the referral agents at transportation brokerages, social service agency caseworkers, and transportation service provider call centers.[19] The two hypotheses to test in this case are:

• The Trip Planner will improve customer satisfaction with the ability to effectively and efficiently plan trips.
• Customers will perceive the information provided as easy to access, accurate, reliable, and useful.

A Web-based or mail-back survey of a subset of the previously cited referral agents will be used to gather information about perceptions of the change in availability, accuracy, reliability, and usefulness of trip planning information after implementation of the system; and customer interest in continuing to use the Trip Planner. (See Section 4.3 Detailed Test Plans for Test #3.)

4.2.3        Coordinated Use of ODOT and DHS Transportation Resources

ODOT and DHS are both heavily involved in the State Agency Transportation Coordination Project aimed at addressing the lack of adequate special needs transportation by coordinating the use of existing transportation resources. The Trip Planner is envisioned as a tool for transportation brokerages and social service agencies to support this effort. Specifically, ODOT and DHS believe that the Trip Planner will increase awareness of available intercity transportation services that are less costly than using volunteer driver programs. The hypothesis is as follows:

• Improved access to information about transportation service options will lower the cost of intercity trips planned by transportation brokerages and social service agencies.

Transportation brokerage records of the costs of providing intercity trips before and after implementation of the system will be analyzed to determine if this system goal is being met. (See Section 4.3 Detailed Test Plans for Test #4.)

4.2.4        Use of Public Transportation Services

In the FTA’s Trip Planning State of the Practice report,[20] a research approach was proposed to answer the following question: “Does an Internet-based itinerary planner increase transit ridership?” According to the report, a trip planner could increase ridership in four ways. Although these scenarios apply to trip planners with point-to-point trip planning capability, it is useful to think about how ridership could be influenced by a trip planner application. The FTA report identifies the following scenarios:

1. Reducing the information costs associated with planning public transit trips (making it easier) might create diversions from other modes. For instance, the more quickly a potential user can find out the desired information, the greater the likelihood that the user might choose to ride transit; or, a visitor could easily get information in advance of his or her visit and plan to use public transportation and avoid using a taxi or renting a car.
2. Lower information costs can generate extra trips that otherwise may not have been made at all.
3. Higher levels of customer satisfaction could lead to improved rider retention so that ridership grows over time as the attrition rate falls and attraction of new customers holds steady.
4. Cost savings associated with reduced call center volume could be used to expand service or marketing efforts with the goal of increasing ridership.

ODOT expects increased ridership because of increased awareness of available services, particularly for intercity transportation service providers. Although this goal applies to all forms of public transportation services, it is expected that the benefits realized by the intercity services will be the most noticeable. The hypothesis is as follows:

• Improved access to transportation service information will result in an increased use of these services, particularly for intercity services.

Ridership and level of service provided (revenue miles or hours) will be analyzed to determine any statistically significant change in the efficiency of these services as a result of the new system. (See Section 4.3 Detailed Test Plans for Test #5.)

4.2.5        Coordination of Connecting Services

The project stakeholders expect that a single, easily accessible source of comprehensive public transportation service information will make it easier for service planners, at public agencies and private companies, to coordinate schedules with connecting services. Currently reliable information about connecting services is not always readily available. The following hypothesis applies to this benefit area:

• Developing a statewide trip planning system will make it easier for public and private service providers to coordinate services with each other.

To determine if this expectation is fulfilled, the Evaluation Team will interview planners at transportation service providers regarding the new system’s usefulness in supporting coordination of services. (See Section 4.3 Detailed Test Plans for Test #6.)

Table 4-1. Trip Planner Project Evaluation Goals, Hypotheses, MOEs, and Data Sources

4.3       Detailed Test Plans

The detailed test plans for the six System Impact Study tests are described below and summarized on Tables 4-2 and 4-3. To avoid compensating for seasonal variations, “before” and “after” data will be gathered for the same months. For example, assuming implementation of the Trip Planner in October 2004, “after” data would be gathered for February, March, and April 2004 and compared to the same data for February, March and April 2005.

• Test #1: TripCheck Usage Assessment – The objective of this test is to determine if implementation of the Trip Planner results in a statistically significant increase in the use of the public transportation pages on the TripCheck Website. ODOT has been monitoring the use of the bus/rail page for over 21months. (See Figure 4-1and Appendix A.) The number of visits is automatically calculated by WebTrends, a tool for reporting Website use. ODOT will continue to monitor the number of monthly visits. The average of 3 months in 2003 and also in 2004 (before implementation) will be compared to the same 3 months in 2005 (after implementation).

Figure 4-1.  TripCheck Usage Trend for Public Transportation Information

• Test #2: TripCheck Public Transportation Inventory – The objective of this test is to determine if the project results in a change in the number of public transportation service providers with Web access to information about their services, and if there is a change in the level of sophistication of the available information. For instance, a Website with schedule information would be considered more sophisticated that one without. An inventory of the status of Web-based access to public transportation provider information has been completed as the “before implementation” inventory (see Appendix B). The aspects tracked are shown in Figure 4-2.

TripCheck Public Transportation Web-Based Information Inventory For each public transportation service provider listed on TripCheck: q Is there a link to a Web site? q     Is there a contact phone number? 1.         For fixed route service providers with their own Websites: q     Is there a contact phone number? q     Is the service area defined? q     Is there a system map? q     Are the hours of operation indicated? q     Are fares shown? q     Are timetables (schedules) available? q     Are individual route maps available? q     Is automated trip planning capability offered? 2.         For Demand Response service providers: q     Is there a contact phone number? q     Is the service area defined? q    Are the hours of operation indicated? q    Are eligibility criteria defined? q   Are fares shown?

 

Figure 4-2.  Aspects Tracked for the Web-Based Information Inventory

• Test #3: Customer Satisfaction Survey – The objective of this test is to determine to what extent target users (referral agents at transportation brokerages, transportation service provider call centers, and social services caseworkers) perceive the Trip Planner system as reliable and useful, especially compared to information available before implementation of the Trip Planner. After the deployment strategy has been identified, the Evaluation Team will work with DHS and ODOT to determine the individual referral agencies to be surveyed. This decision will be based on the quantity and variety of public transportation service information available in various regions in the State that have existing transportation brokerages. A questionnaire will be designed (and reviewed by ODOT and DHS) and made available to study participants in an on-line format unless mail-back questionnaires are required. It is expected that all referral agents in the geographic area of interest will be invited to participate in the survey. The results will be used to describe customer satisfaction with various aspects of the Trip Planner system.
• Test #4: Trip Cost Assessment – The objective of this test is to determine if use of the Trip Planner is resulting in any cost reductions due to increased awareness of transportation options. Transportation cost data currently collected and maintained by DHS will be requested for specific months prior to implementation of the system, and again for the same months after system implementation. The data will be normalized and compared for evidence of statistically significant differences. Any trip level detail available will be analyzed to determine if specific reasons for any cost reductions can be identified.
• Test #5: Ridership Assessment – The objective of this test is to determine if implementation of the Trip Planner resulted in any changes in ridership relative to service provided. Ridership and revenue miles (or revenue hours) information will be collected from intercity service providers in the geographic area of interest. Riders per revenue mile (or riders per revenue hour) will be calculated and compared for a specified number of months before the Trip Planner system implementation, and again for the same specified number of months after the implementation.
• Test #6: Transportation Service Provider Interviews – The objective of this test is to determine if the comprehensive information available from the Trip Planner is reliable and useful enough to assist in planning coordinated connecting services. Service planners at various agencies will be interviewed to ascertain their perceptions of the reliability and usefulness of the information available through the Trip Planner. Depending on the number of service providers participating, a focus group environment may be more useful than individual interviews.

Table 4-2.  System Impact Study Detailed Test Plans Overview

Table 4-3.  Detailed Test Plan Activities

4.4       Institutional Challenges Review

Several institutional challenges are likely to influence the development and deployment of the Trip Planner system. ODOT prepared themselves for many of these issues by conducting interviews with other agencies with trip planner experience. After deployment of Release 1.0, the Project Management Team and major stakeholders will be interviewed regarding their experiences with, and perceptions of, the following anticipated challenges:

1. Completing institutional agreements between ODOT and transportation service providers for providing and maintaining data.
2. Achieving consensus on a standardized digital base map.
3. Using Transit Communications Interface Profiles (TCIP) for transit data definitions in an environment of existing proprietary scheduling systems.
4. Acceptance of ongoing operating cost, especially for small agencies.
5. Lack of technology skills and equipment, especially for smaller agencies without dedicated information technology staff.
6. Managing expectations for the level of effort required and cost of data maintenance.
7. Keeping the Trip Planner data current.
8. Avoiding information gaps in transportation network due to non-participating service providers.
9. Accommodation of rural and urban services.
10. Multiple ODOT departments active in project management, all with different priorities and terminology.
11. Integrating existing trip planners without duplicating efforts.
12. Resolving data ownership and access issues.

5.      Risk Assessment

5.1       Overview

The probability that the Trip Planner project will produce meaningful system impact data in a reasonable timeframe is of concern to the USDOT. There are two significant areas of risk involved in a deployment such as the Trip Planner project: scope and schedule. These risks are discussed as follows.

5.2       Scope Risk

If the scope of the Trip Planner system changes significantly, the hypotheses regarding the expected benefits and associated test activities may no longer be valid. Based on current progress regarding system development activities, the Evaluation Team assesses this risk as low. For this project the following scope issues apply:

• Data Provider Scope – Although planned as a statewide system, the actual data contained in the system database will be dependent upon the participation level of the transportation service providers. The Evaluation Plan requires that the Trip Planner database include nearly all of the major providers, especially the intercity service providers, within at least one region of the State. As of March 2003, there were no formal commitments on the part of service providers to participate (interagency agreements will be included as part of the design phase of the deployment); however, the broad range of service providers involved in the user workshops and participating as members of the Steering and Technical Advisory Committees does indicate a high level of interest. In addition, ODOT has been very responsive to the needs of the Evaluation Team up to this point. For these reasons, it is expected that at least a portion of the State will have the data in place to support a meaningful evaluation.
• Functional Scope – From a technical point of view, the functional scope of Release 1.0 is far less complex than implementation of a more traditional point-to-point trip planner. In addition, the vendor developing and implementing the system has solid experience with this type of system. The most significant technical challenge will be implementing interactive map-based tools for users because of the amount of data that must be geo-coded. However, due to the vendor’s experience level it is expected that the system will include the anticipated capabilities.

5.3       Schedule Risk

For this evaluation, a schedule slip will not negatively affect any aspect of the planned tests. However, significant delays in the deployment schedule may decrease the relevancy of the results. Already the project has been delayed by nine months as a result of revised estimates of the time required to execute interagency agreements between ODOT and the data providers. The Deployment Team is using a very structured project management approach, including formal quality assurance oversight, which has kept the project close to schedule for the past year. Additional delays are possible as a result of the interagency agreements, however, there are highly visible programs such as the State Agency Transportation Coordination Project that rely to some extent on the Trip Planner and, as previously mentioned, the Trip Planner project is not overly technically complex for the participating vendor.

6.      Evaluation Management Plan

6.1       Overview

Leadership for the SAIC Evaluation Team will come from the SAIC Project Manager, Ms. Catherine Bradshaw Boon, who will report to the SAIC Program Manager, Mr. Mark Carter. The SAIC Project Manager will directly manage all data collection and analysis activities. Mr. Brian Cronin, FTA Task Manager, and Dr. Joseph Peters, the ITS JPO Contracting Officer’s Technical Representative (COTR) will provide oversight for the evaluation. Data collection, analysis, and documentation support activities will be provided by Ms. Jennifer Rephlo, an SAIC Transportation Analyst, and Ms. Carol Mitchell, an SAIC Data Analyst and Technical Writer. An overview of the organizational structure is presented in Table 6-1.

Table 6-1.  Evaluation Organizational Overview

6.2       Level of Effort

Table 6-2 presents the anticipated level of effort for the evaluation activities described in this Evaluation Plan.

Table 6-2.  Level of Effort

6.3       Deliverables

The evaluation activities described in Table 6-2 will result in the delivery of a draft and final comprehensive Evaluation Report documenting the methodology and the detailed results of the System Impact Study and the Institutional Challenges Review. The report will consist of the following six major sections:

Executive Summary

1. Introduction
2. Methodology
3. Results
4. Conclusions
5. Recommendations

6.4       Evaluation Schedule

The schedule for the evaluation activities is detailed in Table 6-3.

Table 6-3.  Evaluation Schedule

Appendix A – TripCheck Website Usage Statistics

Table A-1 shows the trend in usage of the TripCheck bus/rail information page as reported in “visits” by ODOT’s WebTrends tool before the implementation of the new Trip Planner.[21]

Table A-1.  TripCheck Website Usage for busrail.htm 6/01 – 2/03

	Jan	Feb	Mar	Apr	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec
2001						608	754	819	735	712	806	868
2002	819	630	890	748	904	717	907	829	664	804	715	910
2003	778	665

Appendix B – TripCheck Public Transportation Web-Based Information Inventory

This following spreadsheet is a record of the status of Web-based public transportation information available through the TripCheck Website before implementation of the Trip Planner. The information was collected from links on the TripCheck bus/rail page (www.tripcheck.com/About/busrail.htm) on January 24, 2003.

Table B-1.  TripCheck Public Transportation Web-Based Information Inventory

References

1.      Oregon Department of Transportation Travel Trip Planning System Study: National and International Transportation Provider Survey Results (Final Technical Working Report 1), January 2000. Prepared for the Oregon Department of Transportation by GIS Trans, Ltd. January 2000.

2.      Oregon Department of Transportation Travel Trip Planning System Study: State Transportation Provider and Related Agency Survey Results (Final Technical Working Report 2). Prepared for the Oregon Department of Transportation by GIS Trans, Ltd., March 2001.

3.      Oregon Department of Transportation Travel Trip Planning System Study: Oregon Transit Trip Planning: System Recommendations (Final Report 3). Prepared for the Oregon Department of Transportation by GIS Trans, Ltd., April 2002.

4.      Transit Trip Planner Release 1.0: Project Statement. Oregon Department of Transportation Intelligent Transportation Systems, March 2002.

5.      Transit Trip Planner Release 1.0: Written Summary of Workshop Findings. Oregon Department of Transportation Intelligent Transportation Systems, February 2003.

6.      Transit Trip Planner Release 1.0: Survey of the Existing System. Oregon Department of Transportation Intelligent Transportation Systems, February 2003.

7.      Trip Planning State of the Practice. Prepared for the United States Department of Transportation Federal Transit Administration by Volpe National Transportation Systems Center, July 2002.

8.      Special Transportation Network Planning Project: Narrative Report of Needs Assessment Research. Prepared for the Linn, Benton, and Lincoln Counties by The Gilmore Research Group and Cogan Owens Cogan, November 2002.

9.      Transit Trip Planner Release 1.0: Project Management Team Structure. Oregon Department of Transportation Intelligent Transportation Systems, July 2001.

10.  Regional Trip Planner: Technical Advisory Committee Overview – Phase 1 Preliminary Analysis. Oregon Department of Transportation and Washington Department of Transportation, March 2003.

Internet Resources

American Public Transit Association, Oregon Transit Links, http://www.apta.com/sites/transus/or.htm

Amtrak Trip Planner, www.amtrak.com

National Transit Database, http://www.fta.dot.gov/ntl/database.html

ITS Benefits and Unit Cost Database, www.benefitcost.its.dot.gov

Oregon Department of Transportation, Intelligent Transportation Systems, Transit Trip Planner Project http://www.odot.state.or.us/its/transit.htm

Oregon Department of Transportation, Public Transit Division http://www.odot.state.or.us/pubtrans/

Portland University Population Research Center, http://www.upa.pdx.edu/CPRC/

Tri-Met Trip Planner, www.trimet.org

TripCheck, http://www.tripcheck.com