Hazardous Material Transportation Safety and Security
Field Operational Test Beta Test and Baseline Data
Report
EXECUTIVE SUMMARY
Contract No.
DTFH61-96-C-00098
Task
9851
Submitted to:
ITS Joint
Program Office
Submitted by:
Science Applications International Corporation (SAIC)
Arroyo
September 29, 2003
QUALITY ASSURANCE
STATEMENT
The Federal Highway Administration
provides high-quality
information to serve Government,
industry, and the public in a
manner that promotes public understanding.
Standards and policies
are used to ensure and maximize the
quality, objectivity, utility, and
integrity of its information. FHWA
periodically reviews quality
issues and adjusts its programs and
processes to ensure continuous
quality improvement.
Beta Test and Baseline Data Collection Results Summary
The Beta Test and Baseline Data Collection efforts ensured that the test technologies would successfully operate during the field operational test (FOT) in the designed scenario configurations. These efforts also ensured that FOT systems would successfully capture and deliver the test data streams to the Evaluation Team for subsequent analysis.
The initial technology demonstration conducted during May and June 2003 provided a preview of the selected test technologies to be deployed during the beta and subsequently during the FOT.
In early June 2003, on-site meetings occurred at Qualcomm
facilities in
The beta test ran for a 5-day period from July 14 – 18,
2003, at Qualcomm facilities in the
All of the test technologies performed successfully during
the week-long test in
· Test technologies demonstrated the necessary functionality in simulated scenario conditions comprising the FOT.
· Generated test data streams witnessed during the week-long test proved the ability to deliver data to enable thorough Evaluation Team analysis throughout the duration of the field operational test. (It should be noted that the Evaluation Team and Operational Team members worked in close conjunction with Qualcomm in formatting the test data during the beta test week to determine which data elements were necessary to be included in actual test data collection.)
The beta testing was conducted primarily over a 4-day period, with a fifth day being devoted to preliminary data analysis and data formatting from July 14 – 18, 2003. Each of the four testing days was based on a specifically designated scenario or sub-scenario configuration to capture a segment of the 16 total technologies to be deployed during the full scale FOT.
FOT participating motor carriers will provide the opportunity to collect comparative data to assess the individual test technology and technology suites. This data will lead to ROI and performance metric calculations to allow for various comparisons pre- and post-test. The Evaluation Team is collecting comprehensive qualitative information from FOT participants to identify their current processes and procedures. This action will help establish the baseline safety and security elements for this FOT and identify precise vulnerabilities applicable to each FOT participant’s operational situation.
During the summer of 2003, eight of the nine FOT participant motor carriers were interviewed at either their operations terminal or corporate headquarters during the summer of 2003 to confirm their “day in the life” operational descriptions for FOT participation. Baseline data from the ninth FOT participant is being obtained from phone interviews and early visits during the FOT deployment. The interviews were also used to collect all available operations data to be used in comparison with the FOT automated systems-generated operations data. The on-site interviews quantified existing operations, including key operating metrics, and collected qualitative information in addition to identifying participant perspectives regarding security and safety concerns.
The most critical outcome to the on site baseline visits is that all FOT participants are working with the Evaluation Team to provide comprehensive short-term historical data to be compared against system generated test data. In addition, FOT participants expressed enthusiasm for the FOT and many stated an extreme interest in the outcome of this test, especially if any government mandates result from the FOT results.
Evaluation Process Lessons Learned
The Evaluation Team has learned some valuable lessons from the close interaction with the Operational Team in conducting the Beta Test and on site baseline data collection interviews. Some of these important lessons learned include:
· Nothing replaces consistent, frequent communication. The Evaluation Team and the Operational Team have maintained constant communication throughout this FOT. This has simplified the process of information exchange between the two sides and allows all parties in this FOT with a clear picture of where the FOT is moving both in regards to the deployment and evaluation effort.
·
Early review of test technology system data
streams is beneficial to all involved. The Evaluation Team was provided an
preview of all technology test data streams and test scripts far in advance of
the Beta Test in July. This enabled the Evaluation Team and Operational Team to
agree on preliminary formatting and monthly data delivery of test data at the
early June meeting in
· Collaborative working arrangements are beneficial to all involved. Throughout this FOT, both the Evaluation and the Operational Teams have worked very closely to ensure technical success in terms of needed data generation and analysis, development of deliverable documents, and performing the FOT on time. Frequent exchange of project notes, coordinated site visits involving both the Evaluation and Operational Teams, advance input and review of major deliverables, and scheduled in-person and teleconference meetings kept all parties intimately involved in this FOT.
· Breaking up the data delivery process. Although it is early in this FOT, it is assumed that the monthly delivery of data test streams to the Evaluation Team in reasonably sized files will allow for more time to be spent on relevant analysis against historical data and should produce interim FOT results faster than if data was delivered to the Evaluation Team in larger amounts less frequently.