Electronic Intermodal Supply Chain Manifest
Freight ITS Operational Test Evaluation
Final Report
PDF Version 1.26MB
Notice This document is disseminated under the sponsorship Mark Jensen |
Field Operational Test Evaluation
Final Report
FREIGHT & ITS WEB RESOURCES
USDOT ITS Joint Program Office:
http://www.its.dot.gov
USDOT Office of Intermodalism
http://www.dot.gov/intermodal/freight.html
FHWA Office of Freight Management
http://ops.fhwa.dot.gov/freight/
ITS Cooperative Deployment Network (ICDN):
http://www.nawgits.com/jpo/icdn.html
ITS Electronic Document Library (EDL):
http://www.its.fhwa.dot.gov/cyberdocs/welcome.htm
USDOT ITS Joint Program Office
USDOT Office of Intermodalism (OST)
FHWA Office of Freight Management and Operations
U.S. Department of Transportation
Federal Highway Administration Operations Unit
400 7th Street, S.W., HOP
Washington, DC 20590
Toll-Free "Help Line" (866) 367-7487
1. Report No. FHWA-OP-02-XXX |
2. Government Accession No. |
3. Recipient’s Catalog No. |
||
4. Title and Subtitle Electronic Intermodal Supply Chain Manifest Field |
5. Report Date December 2002 |
|||
6. Performing Organization Code |
||||
7. Authors Mark Jensen (SAIC), Mike Williamson (Cambridge Systematics), Robert Sanchez (SAIC), Carol Mitchell (SAIC) |
8. Performing Organization Report No. |
|||
9. Performing Organization Name and Address Science Applications International Corporation 8301 Greensboro Drive M/S E-7-6 McLean, VA 22102 |
10. Work Unit No. (TRAIS) 11. Contract or Grant No. DTFH61-96-C-00098; Task 9811 |
|||
12. Sponsoring Agency Name and Address United States Department of Transportation ITS Joint Program Office, HVH-1 400 7th Street SW Washington, DC 20590 |
13. Type of Report and Period Covered 14. Sponsoring Agency Code HOIT-1 |
|||
15. Supplementary Notes Dr. Joseph I. Peters (COTR) |
||||
16. Abstract This report presents the results of a 2.5 year freight ITS evaluation of an air cargo security and logistics system which was deployed at O’Hare and JFK international airports. In September 1999, the Federal Highway Administration (FHWA) and the Federal Aviation Administration (FAA) jointly funded a field operational test to develop an electronic supply chain manifest system for air cargo. The primary objectives of this test were to increase the security of air cargo operations, while providing shippers and carriers with improved efficiencies in their operations. Several key conclusions of this evaluation report are summarized as follows:
|
||||
Key Words Intelligent Transportation Systems, ITS, Freight, Intermodal, Electronic Chain Supply Manifest, Biometrics, Smart Cards, Field Operational Test, Evaluation |
18. Distribution Statement No restrictions. This document is available to the public from: The National Technical Information Service, Springfield, VA 22161. |
|||
19. Security Classif. (of this report) Unclassified |
20. Security Classif. (of this page) Unclassified |
21.No of Pages 82 |
22. Price N/A |
|
Form DOT F 1700.7 (8-72) Reproduction of completed page authorized.
TABLE OF CONTENTS
LIST OF TABLES
LIST OF FIGURES
ABBREVIATIONS
EXECUTIVE SUMMARY
1. INTRODUCTION
2. ESCM SYSTEM Deployment Overview
3. Technical Approach
4. Operational Impact of The ESCM System
4.1 Description of Participants’ Use of ESCM System
4.2 Comparison of Manual and Automated Systems
4.3 Customer Satisfaction and Institutional Challenges
5. Overview of the Technical Effectiveness of the ESCM System
5.1 Integrated air cargo system OPERATIONAL PERFORMANCE
5.1.1 Positive Association of Driver with Electronic Manifest
5.2 ESCM Computer System TECHNICAL Performance
5.3 Electronic Manifest Technical Performance
5.4 Biometric Fingerprint Identification technical Performance
6. Conclusions and Recommendations
REFERENCES
APPENDIX
LIST OF TABLES
Table ES-1. ESCM Operational Test Participants
Table ES-2. Efficiency Improvements for Trucking Companies
Table 4-1. ESCM System vs. Manual Process Times per Shipment for Manufacturers
Table 4-2. ESCM System vs. Manual Process Times per Shipment for Trucking Companies
Table 4-3. ESCM System vs. Manual Process Times per Shipment for Airlines
Table 5-1. Total Manifest/Cargo Cycle Time
Table 5-2. Physical Cargo Cycle Times
LIST OF FIGURES
Figure ES-1. Cargo Operations at O’Hare Experienced Rapid Growth in 1990’s
Figure ES-2. ESCM System Design Concept
Figure ES-3. ESCM System Processes Overview
Figure 2-1. Biometric Fingerprint Identification Technology.
Figure 2-2. ESCM System Smart Card Reader and Smart Cards.
Figure 2-3. Data Entry of ESCM System Manifest at Manufacturer Location.
Figure 2-4. Centralized ESCM System Server Enabling Internet Access.
Figure 2-5. Shipment Acceptance and Processing at Participating Airline.
Figure 2-6. ESCM System Login Screen for All Authorized Participants.
Figure 2-7. ESCM System Manifest Search Screen after Manufacturer Login.
Figure 2-8. ESCM System Manifest Maintenance Screen Before a Manufacturer Creates an Air Waybill.
Figure 2-9. ESCM System Manifest Maintenance Screen After a Manufacturer Creates an Air Waybill.
Figure 2-10. Truck Driver’s Screen After Login (Air Waybill #14 Created).
Figure 2-11. Air Cargo Clerk’s Screen View After Login.
Figure 2-12. Air Cargo Clerk’s Screen View After Air Waybill Number Selection.
Figure 2-13. ESCM System Logout Screen for All Participants.
Figure 3-1. Phase II Pre-Deployment Data Collection.
Figure 3-2. Phase II Deployment Data Collection.
Figure 5-1. Frequency of ESCM Transactions
Figure 5-2. Frequency of Elapsed Times for Manifest Creation to Release
Figure 5-3. Frequency of ESCM System Errors
Figure 5-5. Second-Try Outcome
Figure 5-6. False Acceptance Rate as a Function of Threshold
AMS |
Automated Manifest System |
ATIS |
Advanced Traveler Information Systems |
COTR |
Contracting Officers Technical Representative |
CVO |
Commercial Vehicle Operations |
DOT |
US Department of Transportation |
ESCM |
Electronic Supply Chain Manifest |
FAA |
Federal Aviation Administration |
FHWA |
Federal Highway Administration |
FOT |
Field Operational Test |
ITS |
Intelligent Transportation Systems |
JPO |
Joint Program Office |
MOE |
Measures of Effectiveness |
SAIC |
Science Applications International Corporation |
EXECUTIVE SUMMARY
Introduction
Air transport is the fastest growing freight transportation mode today with both volume and revenue projected to double by 2006. Although airfreight comprises only 1% of total freight moved worldwide by weight, it accounts fort 38% of total freight by value. Airfreight can therefore be classified as high value/low density products with a heightened requirement for timely delivery.
Figure ES-1. Cargo Operations at O’Hare Experienced Rapid Growth in the 1990’s
Security concerns and time pressures to deliver air cargo more quickly than ever are focusing attention on the ground-to-air intermodal link. Truck-to-air cargo movements grew at a rapid rate in the 1990’s, yet this logistics link today is still largely maintained by industry and regulated by government using paper- and telephone-based information exchanges. This situation is exacerbated by the fact that much of the air cargo is transported on passenger planes, a major safety concern that is now even more critical following the events of September 11, 2001. To support the needs of the marketplace and to ensure the security of air passengers and cargo shipments, new tools and processes are being encouraged by the U.S. Department of Transportation.
The ESCM Field Operational Test
In support of the Office of the Secretary of Transportation, the Federal Aviation Administration (FAA), the FHWA Office of Freight Management and Operations, and others, over the past three years the ATA Foundation led the development of a pubic-private partnership to develop and test the first operational electronic air cargo manifest and security system in the United States. The goal of this test was to demonstrate the improvements in efficiency and security of an Internet-based electronic air cargo security system compared to traditional processes and paper-based manifest systems. The operational test was conducted in conjunction with manufacturing, trucking, and airline participants in the Chicago-O’Hare International Airport and New York City-JFK International Airport service areas. A summary of test participants and their roles is presented in Table ES-1.
Table ES-1. ESCM Operational Test Participants
|
ROLE |
||||||||
Project Management |
System Development |
System Deployment |
Participant Recruitment/ |
System Participant |
Evaluation |
Project Oversight |
|||
Public Sector Partners: |
|||||||||
USDOT & FHWA |
● |
||||||||
Federal Aviation Administration |
● |
||||||||
State of Illinois |
● |
||||||||
NY Department of Transportation |
● |
||||||||
Chicago Dept. of Aviation (O’Hare Airport) |
● |
● |
|||||||
Port of NY-NJ (JFK Airport) |
● |
● |
|||||||
Private Sector Partners: |
ATA Foundation |
● |
● |
● |
● |
SecurCom (System Engineer) |
● |
● |
● |
Identix (Biometrics) |
● |
● |
Manufacturers (2 @ O’Hare) |
● |
MotorCarriers (4 @ O’Hare, 4 @ JFK ) |
● |
Air Cargo Carriers (4 @ O'Hare, 5 @ JFK) |
● |
SAIC & Cambridge Systematics |
● |
This test builds on an earlier FAA test at Chicago’s O’Hare Airport. That test used biometric “smart cards” to confirm the identity of the driver using a stored thumbprint and to provide information about the seal on the cargo the driver was transporting. The purpose was to improve security of freight movement into and out of the airport. Building on this experience, this operational test deployed and tested a secure electronic manifest system -- utilizing leading edge security technologies including encrypted internet transactions, 8K smart cards and biometric fingerprint readers. An overview of the system design concept is presented in Figure ES-2.
Figure ES-2. ESCM System Design Concept
This system is designed to allow only authorized users to enter and monitor cargo movement and access valuable shipment information at specific points in time and in the logistics process. However, this system also provides for substantially more freight management functions, all of which differ depending on whether the user is a manufacturer, a motor carrier, and an air freight consolidator or airline. For example this system his could allow an airline advance notice of incoming freight and reduce consolidation time in planning specific flight loads. In the unfortunate case of an air transport incident, the cataloging of electronic manifests can provide immediate access cargo content records by public sector agencies to aid in incident reconstruction.
This system allows manufacturers to send cargo information real-time along the distribution channel in advance of pick-ups and deliveries. And the electronic manifest offers a secured identity process through biometric imprints (fingerprint recognition) in addition to the reduction in information errors due to electronic processing at all times. An overview of the system processes utilized in this test is presented in Figure ES-3.
Figure ES-3. ESCM System Processes Overview
Evaluation Overview
An Evaluation Team led by SAIC was selected in January 2000 by USDOT to develop and implement an independent evaluation of the ESCM Operational Test. The objective of this evaluation was to identify goals and “lessons learned” with respect to implementing the technologies. The evaluation focused on the four areas:
- Intermodal Freight System Operations
- Technology Effectiveness
- Institutional Issues
- Participant Satisfaction
A key component of the evaluation approach was the teamwork and coordination that existed between the ATA Foundation (the deployers) and the Evaluation Team. This was a critical component to the evaluation as repeated on-site collections had the potential for unnecessary disruption of participants’ operating routines.
There were two groups of objectives developed for the evaluation: operational and technical. The operational objectives addressed the participants' experiences with the ESCM system, and the technical objectives addressed the ESCM system functionality. The objectives of the evaluation consisted of the following:
- To measure the impact of the ESCM system on overall freight system operations (including shipment security) of the participants, to document the participants’ level of satisfaction with the performance of the system, and identify key institutional challenges associated with the deployment.
- To measure the technical performance of the ESCM technologies in their operating environment, in terms of both the integrated air cargo system operational performance and the individual technology technical performance for electronic manifests, smart cards and biometric technologies.
Participant Expected Benefits
During pre-ESCM system deployment interviews, participants were specifically asked by the Evaluation Team about the type of results they hoped the test would provide. Their responses are aggregated into the following three categories of operational impacts:
- Efficiency. One major opportunity for achieving this was the ability of the ESCM system to automate the shipment transfer procedures (biometric/Smart Card authorization vs. manual duplication/photocopying of all paperwork).
- Security. A secure, closed system that ensures an individual entity is responsible for a load at all times, coupled with the ability to track the ownership of shipments was viewed as an enormously valuable tool by all participants
- Regulatory Compliance. The Participants were hopeful that a system of this type would be a step in the right direction for streamlining and improving their ability to meet what they currently see as cumbersome paper-based FAA requirements.
System Usage
In most cases, although the system was used, it was not used as fully as it could have been – most participants did not successfully integrate the ESCM system into their daily operations for all transactions. This was primarily the result of the duplicate processing (they had to still use the FAA-mandated paper-based system), limited staff resources, and working with intermodal partners who were not part of the test..
Efficiency Analysis
The Evaluation Team conducted a comparison of existing manual security functions versus the operational test systems automated processes for manufacturers, trucking companies and airlines. This comparison showed significant time savings in every category measured. For example, Table ES-2 shows the time savings calculated for trucking companies. These savings are substantial when multiplied by total number of shipments per year for a typical trucking company.
Table ES-2. Efficiency Improvements for Trucking Companies
Trucking Company Activity |
Percent Reduction in Time |
Order acceptance over the phone and data input |
100% |
Load acceptance at manufacturer |
93% |
Input to create master manifest |
66% |
Reproduction of manifests |
100% |
Paperwork error correction |
100% |
Contact airline and arrange shipping |
100% |
Delivery to airlines |
94% |
System Technology Effectiveness
The evaluation of the system operational performance showed that the integrated air cargo system (consisting of the electronic manifest, Smart Cards, and biometric identification technologies) completed the required tasks expeditiously. ESCM system transactions were recorded by the server and errors were captured in a timely manner. Although the limited number of project participants resulted in only eight shipments completing the entire electronic manifest cycle (create, release, pick up, delivery), the ESCM server did capture the transactions for subsequent analyses.
Over a 6 month test period, the ESCM computer system and network was found to be reliable. Reliability was measured in terms of system up time, unexpected errors, availability of system resources (and not working near capacity), and lack of unauthorized access attempts. Five metrics of system availability were examined: all indicated that the computer resources (computer processor, network activity, disk space, and physical memory) were working below capacity and were readily available.
Participant Satisfaction and Institutional Challenges
The evaluation found that heavy users of the system reported continued interest and commitment to the system throughout the test. These participants had operations staff that found the system easy to use and they remained committed in many cases in hopes that the system would be expanded and further deployed. They reported that once the ESCM became the primary system for their and their supply chain partners operations the true benefits could be measured. This being said, a major concern documented by the Evaluation Team with this FOT was the number of participants that dropped out of the test over its duration. This occurred because they could no longer afford to operate two duplicate systems. While the ESCM was designed to replace existing FAA regulatory paper-based processes, it did not replace these processes as part of this test -- it simply added a new set of procedures that duplicated an existing processes. Staffing constraints and profitability became the ultimate priority of these companies, as should be expected of for-profit organizations. Given the serious nature of air cargo security, and the fact that not all supply chain partners of these companies were involved in the test, it would have been difficult to eliminate the existing and regulated process. However, without doing so, the test was necessarily restricted to partial use of the system as resources allows.
Conclusions
The time savings estimates developed in this report show the potential for substantial industry time savings by the implementation of this system. Many of these savings were estimated to come from replacement of manual processes with automated ones like notification of load pickup or acceptance.
Few shipments were processed through the entire ESCM system. Over the defined test period, only eight shipments moved completely through the system from manufacturer to airline using the ESCM system. This was largely due to participant staff resource problems and interactions with non-test logistics partners.
Participants have reported overall satisfaction with the ESCM system. Some participants felt the ESCM system would be significantly more useful with wider deployment to more of their supply chain partners.
Significant outreach activities were conducted to build support for the ESCM system. Nearly 100 ESCM system demonstrations were provided for participants.
Drivers were very interested and supportive of a system that provides a single Smart Card that replaces their commercial driver’s license (CDL) and paper work. One Smart Card that eliminates the need for cumbersome paperwork and duplicate copies of their CDLs was welcomed.
Recruitment of participants was an ongoing challenge. Identifying complete supply chains was difficult; starting with motor carriers proved the best way to build supply chains; getting the managers to personally “buy in” to the test was critical.
Several participants used the system for 18 months as opposed to the originally planned 6-month test period. “When will this system become fully operational so that I can discontinue the manual processes,” was a common theme.
The test was successfully developed and deployed within the confines of a complicated organizational structure. This created the need for a high level of coordination by ATA Foundation and SecurCom staff to ensure that all stakeholders were provided with the appropriate level of service, while not negatively impacting the operational participants.
Recommendations
The following highlights some of the most important recommendations of this evaluation final report which have been developed for consideration by FHWA and FAA as these technologies are expanded to LAX and Toronto International airports:
Acquire a partial waiver from FAA to allow full testing of the ESCM system to document actual changes/improvements in operations based on full deployment. This would require participants to fully embrace the system and rely on it. This would take the test to the next step as lessons learned would be on actual system use as opposed to partial testing as time permits.
Expand the current follow-on phase of this operational test to include the second half of the freight movement. This FOT covered manufacturer to motor carrier to airline. This should be expanded to include final delivery of the shipment (airline to motor carrier to customer).
Identify an appropriate participant (high volumes and multiple partners) with a legacy system and develop an interface between it and the ESCM system to test the ease of customization. Of the participants with automated systems, they were frustrated the inability to integrate their electronic systems.
Develop an organizational structure that streamlines access to funding and facilitates decision making from an operations perspective. The Phase II test had multiple agencies playing lead roles and multiple funding sources that impacted ease of implementation early in the development and deployment.
Consider developing an incentives program that stimulates broader system testing. If regulatory agencies are unable to provide waivers during FOTs, then reimbursing participants for time spent on test activities to stimulate system testing should be considered – the could be done by tax incentives, staff incentives, etc.
1. INTRODUCTION
1.1 Background/Purpose of Field Operational Test
In September 1999, the Federal Highway Administration (FHWA) and the Federal Aviation Administration (FAA) jointly funded a field operational test to develop an electronic supply chain manifest system for air cargo. Improving the existing system is necessary as air transport is the fastest growing freight transportation mode today with both volume and revenue projected to double by 2006[1]. Although airfreight comprises only 1 percent of total freight moved worldwide by weight, it accounts for 38 percent of total freight by value[2].
The primary objectives of this test were to increase the security of air cargo operations, while providing shippers and carriers with improved efficiencies in their operations. The development of the electronic supply chain manifest (ESCM) system was the second phase of the project. Phase I specifically focused on establishing a Smart Card/ biometric-based driver security system to improve the efficiency of truck access to airports for the delivery of air cargo by automating the transfer process, which historically consisted of manually photocopying the driver’s license for each bill of lading. The Phase I system allowed the driver to communicate his/her identity via a personalized Smart Card and biometric reading. In addition, each truck trailer was sealed at the point of origin and the seal number was loaded onto the Smart Card.
Phase II consisted of the development of an Internet-based manifest system that was access-restricted/managed using the biometric and Smart Card technology developed in Phase I. The use of trailer seals was not continued in Phase II. Phase II focused on testing the manifest system and its access controls across three node supply chains (manufacturers, motor carriers, and airlines). This test had many participants and was led by the American Trucking Association Foundation (ATA Foundation). This test was deployed initially at Chicago’s O’Hare International Airport and was expanded to New York City’s J.F.K. International Airport. Initially, there were plans to acquire a waiver from the FAA to allow this system to be the official system for participants over the finite period of the test. However, this waiver was not enacted as part of the test, therefore, the test required dual systems maintenance for the old and new systems employed.
In support of the United States Department of Transportation’s (USDOT) intermodal Intelligent Transportation Systems (ITS) program, an Evaluation Team lead by Science Applications International Corporation (SAIC), under the direction of the USDOT Joint Program Office (JPO), was selected in January 2000 to develop and implement an evaluation of the Electronic Intermodal Supply Chain Manifest ITS Field Operational Test (FOT). The ultimate goal of this evaluation, as defined by the JPO, was to identify “lessons learned” with respect to implementing intermodal ITS technologies for four study areas: system operational processes, technology applications, institutional agreements, and user acceptance. The Evaluation Team has worked closely with the deployers over the course of this FOT to develop a comprehensive evaluation.
In addition to the independent evaluation described in this document, the ATA Foundation, as the system developer and deployer, conducted its own “self-evaluation” activities in support of FAA and State of Illinois requirements. It was agreed upon by all parties early in the process that all data collected in support of evaluation activities would be coordinated and shared. This has occurred and is described in more detail throughout this document.
1.2 Organization of the Final Report
This draft final report has been organized to describe the activities undertaken by the deployers and the Evaluation Team, and to document lessons learned and recommendations for future research. The remaining sections and content of this document is described as follows:
Section 2. ESCM System Deployment Overview. Section 2 provides a detailed description of the ESCM system FOT, including a summary of Phase I and an overview of the Phase III expansion activities already underway.
Section 3. Technical Approach. Section 3 provides a review of the methodology used to conduct the evaluation activities. This includes a review of key objectives and the data collection and analysis efforts.
Section 4. Operational Impact of System. Section 4 defines the participants' use of the ESCM system, compares the available manual and automated data, and identifies key customer satisfaction and institutional challenges.
Section 5. Overview of the Technical Effectiveness of the System. Section 5 describes the technical performance of the ESCM system at two levels – integrated system operational performance and major technology technical performance.
Section 6. Findings, Conclusions, and Recommendations. Section 6 highlights the findings and conclusions of the evaluation and provides recommendations for consideration by FHWA and FAA.
2. ESCM SYSTEM Deployment Overview
The ESCM system evaluated by the Evaluation Team is the second phase of a three-phase project. The first phase was completed prior to the Evaluation Team's involvement, and the third phase currently is ongoing and is not part of this evaluation. The following provides descriptions of each phase.
2.1 Highlights from Phase I
Phase I of this project was sponsored by the FAA and the State of Illinois, and consisted of establishing a Smart Card/ biometric-based driver security system to improve the efficiency of truck access to airports for the delivery of air cargo. This was accomplished by automating the transfer process, which historically consisted of manually photocopying the driver’s license for each bill of lading. The Phase I system allowed the driver to communicate his/her identity via a personalized Smart Card and biometric reading. In addition, each truck trailer was sealed at the point of origin and the seal number was loaded onto the Smart Card. This phase also allowed the FAA to review its “known shipper” regulations and protocols.
Phase I involved over 500 drivers and 11 airlines and/or freight forwarders. Results from this project indicated that biometric identification and Smart Card systems can provide tangible improvements in air cargo security and greater efficiencies for motor carrier operational processing. Further results determined that comprehensive training and communication programs must be developed to ensure user acceptance. Additionally, it was determined that technology upgrades must be performed regularly to ensure high system performance.
In Phase I, a computer equipped with a Smart Card reader and software was installed at each participating trucking company. A participating driver placed a numbered seal on the trailer door and entered the seal number and cargo information into the computer prior to delivery at the airport. The Smart Card was created with the seal and driver information on it. The driver then proceeded to the air cargo loading area, where the air cargo attendant scanned and read the card along with the driver’s thumbprint. Based on the card information, the attendant’s computer then retrieved a picture of the driver, driver information, and seal number. The computer then displayed an approval, denial, or request for additional information. Based on an approval, the attendant checked the seal on the container and allowed the driver to proceed to the unloading area.
2.2 Overview of Phase II
Phase II built upon the Phase I technologies. Using the truck driver/cargo access system developed in Phase I, Phase II integrated a newly developed biometrically secured electronic manifest. This system was accessed through the Internet, which provided all supply chain participants access to the load information in their respective supply chains.
The shipment was first originated in the system when the manufacturer entered the load information in the ESCM. This data entry process was secured by Smart Card and biometric technology. Once the manifest was complete, it was uploaded to the manifest database on the server. At this time, emails were automatically sent to the downstream supply chain partners. At each subsequent transfer, emails were sent to all three supply chain participants. This communicated to the motor carrier that the shipment was ready for pickup and to the airline that the shipment could be tracked. When the truck driver arrived to pick up the load, he/she accepted the load electronically via Smart Card and biometric confirmation. The shipment status was electronically transferred from the shipper to the motor carrier via this process. The shipment was then transported to the airline. When the driver entered the air cargo office, both the clerk’s and the driver’s identities were confirmed with Smart Cards and biometric scan. This completed the supply chain as the shipment was electronically transferred to the airline and the shipment was accepted.
Figures 2-1 through 2-13 illustrate the ESCM process. Figure 2-1shows the biometric and Smart Card technologies used to secure the system. At right, the fingerprint is scanned and converted to a “minutiae” template as shown below. The pattern displays a mathematical representation of the fingerprint, which can be easily stored and retrieved in identification information systems. This method of biometric fingerprint identification and storage is the most commonly used format.
Figure 2-1. Biometric Fingerprint Identification Technology.
Figures 2-2 through 2-5 show various components of the participant process and equipment installations, ranging from the Smart Card reader and Smart Cards that contain shipment information, to data entry of the manifest at the manufacturer’s location, to the centralized ESCM system main server that allows authorized users access for shipment tracking via the Internet, to the final shipment acceptance and processing with the destination airline.
Figure 2-2. ESCM System Smart Card Reader and Smart Cards.
Figure 2-3. Data Entry of ESCM System Manifest at Manufacturer Location.
Figure 2-4. Centralized ESCM System Server Enabling Internet Access.
Figure 2-5. Shipment Acceptance and Processing at Participating Airlines.
Figures 2-6 through 2-13 provide screen shots of the ESCM system as used by the authorized supply chain participants.
Figure 2-6. ESCM System Login Screen for All Authorized Participants.
Figure 2-7. ESCM System Manifest Search Screen after Manufacturer Login.
Figure 2-8. ESCM System Manifest Maintenance Screen Before a
Manufacturer Creates an Air Waybill.
Figure 2-9. ESCM System Manifest Maintenance Screen After a
Manufacturer Creates an Air Waybill.
Figure 2-10. Truck Driver’s Screen After Login (Air Waybill #14 Created).
Figure 2-11. Air Cargo Clerk’s Screen View After Login.
Figure 2-12. Air Cargo Clerk’s Screen View After Air Waybill Number Selection.
Figure 2-13. ESCM System Logout Screen for All Participants.
The ESCM system provided continuous tracking capabilities, provided one automated paperless shipment history, and accounted for shipment responsibility throughout the supply chain. The system required that all three participants use the ESCM system. There were two Phase I components that were not included in Phase II. First, the use of seals was eliminated in Phase II due to the limited success of using seals during Phase I in a multi-stop truck loading environment that is common in air-to-truck cargo operations. And second, the drivers’ photographs were not loaded onto the Smart Cards electronically due to concerns expressed by drivers during the Phase I test.
There were three main groups of participants involved in this FOT:
- The first group consisted of the participants responsible for developing and deploying the systems (i.e., ATA Foundation, SecurCom).
- The second group consisted of the regulatory agencies responsible for reviewing and approving the FOT components and facilitating the deployment process (i.e., FHWA, FAA).
- The third group consisted of the operational participants who were responsible for incorporating the FOT into their operations (manufacturers, motor carriers, airlines/air freight forwarders).
In addition, there were representatives from other entities that provided support and input over the course of this FOT (i.e., O’Hare International Airport, New York and New Jersey Port Authority, Chicago Area Transportation Study). The preceding list of participants/ representatives is meant to illustrate the diversity of the group – it does not accurately reflect all participants (public or private).
In addition to the operational test, it was anticipated that the data derived from this system could be used to provide real-time traffic information to regional Advanced Traveler Information Systems (ATIS) in the Chicago area, such as the Gary-Chicago-Milwaukee Priority Corridor project. This did not occur as part of this FOT; however, it could be an opportunity for development in future applications of these technologies.
There were expectations for a Phase III project that would expand this test to include an automatic cargo profiling function to assist regulatory agencies in identifying dangerous goods and thus “red-flag” specific loads for additional inspection. Although there was a Phase III in operation during the development of this report, this phase is focusing on geographic expansion, including expanding to an airport outside the United States. A description of Phase III is provided in the next section.
2.3 Description of Phase III
Phase III of the ESCM system was originally envisioned to be an expansion of the Phase II system to provide for additional cargo screening capabilities. Specifically, it was to provide cargo information to the regulatory agencies making decisions on which shipments should receive additional screening, such as hazardous materials or other shipments that were categorized as high risk. Although the system has been expanded beyond the parameters of the Phase II test, it was not expanded in this manner.
The current Phase III expansion effort consists of geographically expanding the system to cover two additional airports (four in total): the Los Angeles International Airport and Toronto International Airport. There were two primary objectives driving this expansion. The first was to continue testing the Phase II system in a broader market place. The second was to test the system in the international arena. As part of this second objective, discussions have taken place with the U.S. Customs Service to investigate integration of the ESCM system with the AMS system. Data are being collected currently for analysis by the deployer, the ATA Foundation.
3. Technical Approach
The technical approach to the evaluation focused on two elements: proof of concept that the technologies functioned as designed; and the impact the ESCM system had on the operations of the participants. The impact on operations evaluation addressed quantitative differences between the existing conditions and the automated conditions provided by the system. In addition, the impact evaluation qualitatively measured customer satisfaction and identified institutional challenges. From a systems perspective, the technical performance of the system also was evaluated.
A key component of the evaluation approach was the teamwork and coordination that existed between the ATA Foundation (the deployers) and the Evaluation Team. It was determined at the start of Phase II that there would be an enormous amount of duplication between these two teams regarding data collection. In an effort to mediate the potential negative impact this would have had on the industry participants, it was agreed that the two would work closely together to collect the data, and then each would conduct their own analyses. This was a critical component to the evaluation as repeated on-site collections had the potential for unnecessary disruption of participants’ operating routines. To have doubled this intrusion by conducting nonparallel duplicate data collection activities would have resulted in an extremely negative situation. In addition, this gave the deployers and the evaluators the opportunity to work together and benefit from each others' perspectives.
The following provides a summary of the activities undertaken to conduct the evaluation. For a more detailed description of these activities, readers are referred to the following two documents:
Electronic Intermodal Supply Chain Manifest ITS Field Operational Test Evaluation Plan, U.S. Department of Transportation and Federal Highway Administration by Science Applications International Corporation, July 12, 2000.
Electronic Intermodal Supply Chain Manifest ITS Field Operational Test Evaluation Detailed Test Plans, U.S. Department of Transportation and Federal Highway Administration by Science Applications International Corporation, June 8, 2001.
3.1 Objectives
- To measure the impact of the ESCM system on overall freight system operations (including shipment security) of the participants, to document the participants’ level of satisfaction with the performance of the system, and identify key institutional challenges associated with the deployment.
- To measure the technical performance of the ESCM technologies in their operating environment, in terms of both the integrated air cargo system operational performance and the individual technology technical performance for electronic manifests, smart cards and biometric technologies.
3.2 Approach
The overall freight system impacts were evaluated by identifying the changes in operations and information flow between the pre-test conditions and test conditions associated with the ESCM system deployment. This required the Evaluation Team to gain a more complete understanding regarding the operational characteristics of the participants, documenting their experiences with the system, identifying their level of satisfaction with the system, and documenting the deployment/institutional challenges encountered.
Interview guides and surveys were designed to collect data on operational impacts, levels of customer satisfaction, and identification of any institutional challenges. Data were collected multiple times over the course of the FOT to capture changes in perceptions and document any problems encountered. In addition, data were collected from the ESCM system deployers to document any challenges associated with the deployment activities, including any ongoing troubleshooting required over the test period. Figures 3-1 and 3-2 illustrate the changes in operations the Evaluation Team attempted to measure pre- and post-deployment of the ESCM system.
The technical performance of the system components in their operating environment were examined to assess the effectiveness of the technologies and to identify needed system improvements. This performance evaluation focused primarily on system data provided by the automated system administration reports. More qualitative user performance data were incorporated as available. The activities focused on both the operational performance of the integrated air cargo system itself (manifest transaction statistics, etc.), as well as specific functionality of system components (system downtime, etc.), which include electronic manifests, Smart Cards, and biometric fingerprint identification.
Figure 3-1. Phase II Pre-Deployment Data Collection.
Figure 3-2. Phase II Deployment Data Collection.
3.3 Work Steps Completed
The following work steps were implemented to support the technical approach:
- Data collection protocols were established with the ATA Foundation for interaction with the participating carriers, airlines, freight forwarders, and manufacturers.
- Data collection tools were developed to support interviews, surveys, and timing activities on-site at participants' places of business.
- Data collection tools were used to collect data from participants before and during the ESCM system test in coordination with the ATA Foundation.
- Participated in the ATA Foundation’s pre-deployment interview process.
- Participated in the ATA Foundation’s close-out interview process.
- The ATA Foundation's draft final report was reviewed and incorporated into this evaluation as appropriate.
- Qualitative and quantitative data were analyzed to identify key findings and conclusions.
- Data collection protocols were established with the ATA Foundation to schedule exports of archived data from the ESCM system.
- Phase I system analyses were reviewed and summarized as part of the ESCM system evaluation.
- Quantitative system data were summarized and analyzed.
Implications of these technologies were assessed in regard to the National ITS Architecture and Standards.
4. Operational Impact of The ESCM System
A key focus of this evaluation was to document, and measure where possible, the impact the ESCM system had on daily operations of manufacturers, motor carriers, and airlines. These three entities work together to move products from point of manufacture to customers using an intermodal system. It was difficult to identify and recruit representatives from each of these categories because they had to be based on existing supply chains. One supply chain “participant” consisted of a manufacturer, motor carrier, and an airline that already worked together. Without this relationship in place, there could be no test of the system.
In addition, there were many differences among the participants involved in this test. Manufacturers producing different products; motor carriers providing LTL service with fixed routes, variable routes, cross-dock operations, and consolidation operations; and airlines or air freight forwarders that served varying markets.
Within this group of participants, the existing level of automation also varied immensely. Some companies had completely manual paper-based systems with no Internet access, while others had well-established automated systems that were used for all business transactions. This diversity in manual vs. automated operational systems presented a range of challenges for the ESCM system developers and deployers. While some companies had no technological infrastructure (e.g., no Internet access), others had sophisticated systems that could not be integrated or accessed as part of this FOT based on incompatible hardware or software applications.
Regardless of these differences, the participants had similar expectations or hopes for what the system could provide their industry. During pre-ESCM system deployment interviews, participants were specifically asked about the type of results they hoped the test would provide. Their responses are aggregated into the following three categories:
Efficiency. The ability to do their jobs better and more efficiently was the universal theme identified. Although each company has its own operational system in place, whether manual or automated, they all continue to look for new ways to streamline and simplify their operations. One major opportunity for achieving this was the ability of the ESCM system to automate the shipment transfer procedures (biometric/Smart Card authorization vs. manual duplication/photocopying of all paperwork).
Security. A secure, closed system that ensures an individual entity is responsible for a load at all times, coupled with the ability to track the ownership of shipments was viewed as an enormously valuable tool by all participants. Manufacturers reported wanting to able to know the shipment was secure from the time of pick up at the factory to the moment of delivery to their customer. Airlines wanted to know that the load they accept for transport has not been tampered with and is in the same condition in which it left the manufacturer.
Regulatory Compliance. The FAA has established protocols and regulations that must be met for air cargo transport. These requirements have resulted in the cumbersome, paper-intensive system used today. Participants were hopeful that a system of this type would be a step in the right direction for streamlining and improving their ability to meet these requirements.
This section summarizes the data collected in support of this evaluation. This summary includes a description of how the system was used, quantifies comparisons of the manual and automated systems, presents the level of satisfaction experienced by the participants, and identifies the institutional challenges.
4.1 Description of Participants’ Use of ESCM System
The participants involved in this FOT were recruited to test the ESCM system over a finite period of time in coordination with their established supply chain partners. This was a voluntary commitment and their agreement to participate was the result of their interest in improving their operations.
It was hoped that the participants would be able to incorporate the system into their daily operations for regular daily use. In most cases, although the system was used, it was not used as fully as it could have been. Following is a summary describing the manner in which participants used the ESCM system. The subsequent customer satisfaction and institutional challenges section further explains their use patterns.
The ESCM system-generated manifests were used in parallel with the existing manual processes, as time allowed. Few manifests were actually entered into the system due to the limited number of customers on the system.
Manifests were processed simultaneously in both systems. For one clerk interviewed, nearly every manifest created during the test period was entered into the system.
Manifests were handled simultaneously in both systems; however; use of the ESCM system was very limited due to removal and reinstallation of the systemThe ESCM system processes were conducted concurrent with the current processes as staff and time allowed; relatively few were entered because creating manifests using the ESCM system was seen as time consuming for the limited staff (existing system was automated with imbedded customer data).
Manifests were primarily pass-throughs from shipper to airline. Emphasis was placed on driver pick up and delivery procedures; enrolled drivers found the system easy to use.
ESCM system use was limited because primary intermodal partners were difficult to recruit or maintain, or were not part of the FOT.
Current system uses EDI; ESCM is a standalone system, which requires separate data entry; few manifests were processed due to lack of time and staff for data entry.
Due to heavy layoffs and staffing limitations, shipment information was only entered a few times. However, the system was used some for shipment tracking.
The preceding interview data illustrates that most participants did not successfully integrate the ESCM system into their daily operations for all transactions. Most were able to test the system, and in some cases provide ongoing data entry, but not for the majority of their shipments. This was primarily the result of duplicate activities, limited staff resources, and working with established intermodal partners who were not part of the FOT. Interestingly, those participants with manual systems and those with automated systems both had the same issue with duplication.
4.2 Comparison of Manual and Automated Systems
The Evaluation Team, in coordination with the ATA Foundation and SecurCom, timed the manual activities at several participants’ operations. These data were collected and compared to ESCM system-generated reports, which provided the details for the same activities as conducted with the ESCM system. The following summary presents data from this comparison.
The manual “time on-task” data were collected at several participant sites. These data represent a variety of tasks identified to represent those activities present in the ESCM system. Both the manual timings and the ESCM system data represent aggregates of the data collected: the sum of all manual timings were compared to the sum of all ESCM system data. However, manual timings were not collected from all participants; the ESCM system data were collected from all participants that used the system. Comparison of these data, therefore, represents an estimate of time savings based on the manual timings of a subset of the participants.
The quantitative data presented here supports the findings of the qualitative analyses. A comparison of the manual and automated processes shows time savings in every category measured. Tables 4-1 through 4-3 show these time saving estimates.[3] In instances where the ESCM system replaces/eliminates human activity, a null time value of “zero” is assumed (e.g., e-mails automatically generated to communicate load transfer information). In addition, other major savings occur with key activities like load acceptance. For example, load acceptance at an air cargo facility was timed at 3 minutes, 3 seconds less than the timing for the manual process. This same activity using the biometric/Smart Card verification process took just 11 seconds, which represents a 94 percent reduction in time.
Table 4-1. ESCM System vs. Manual Process Times
per Shipment for Manufacturers
Activity |
Manual Time |
ESCM Time |
Time Savings |
Percent Reduction |
Filling out manifests |
2:35 |
1:07 |
1:28 |
54% |
Contacting motor carriers (from carrier order acceptance) |
0:51 |
- |
0:51 |
100% |
Search out documentation, load verification, driver sign-off |
4:12 |
0:18 |
3:54 |
93% |
Paperwork error correction |
0:12 |
- |
0:12 |
100% |
Table 4-2. ESCM System vs. Manual Process Times per Shipment
for Trucking Companies
Activity |
Manual Time |
ESCM Time |
Time Savings |
Percent Reduction |
Order acceptance over the phone and data input |
0:51 |
- |
0:51 |
100% |
Load acceptance at manufacturer |
4:12 |
0:18 |
3:54 |
93% |
Input to create master manifest |
2:08 |
0:43 |
1:25 |
66% |
Reproduction of manifests |
1:03 |
- |
1:03 |
100% |
Paperwork error correction |
1:03 |
- |
1:03 |
100% |
Contact airline and arrange shipping |
4:09 |
- |
4:09 |
100% |
Delivery to airlines |
3:03 |
0:11 |
2:52 |
94% |
Table 4-3. ESCM System vs. Manual Process Times per Shipment for Airlines
Activity |
Manual Time |
ESCM Time |
Time Savings |
Percent Reduction |
Order taking/contact motor carriers |
4:09 |
- |
4:09 |
100% |
Load acceptance |
3:03 |
0:11 |
2:52 |
94% |
Clerical time for creating airplane load documentation |
3:00 |
- |
3:00 |
100% |
Paperwork error correction |
0:41 |
- |
0:41 |
100% |
Copy and file for FAA audits |
1:10 |
- |
1:10 |
100% |
The time savings estimates presented in the preceding tables show significant reductions by percent for these tasks. These savings are substantial when multiplied by total number of shipments per year for a given company. The ATA Foundation’s final report has converted these time savings into dollar savings. As part of the independent evaluation, percent reduction has been used to focus more on direct operational metrics as opposed to financial feasibility.
Given the qualitative results, which suggest that the system was well received but underutilized, the time savings should be viewed as a conservative estimate of potential savings. However, conversely, the test parameters required that the existing manual and ESCM system processes be completed. Without fully testing the ESCM system as “the system”, it is difficult to predict with confidence that all these benefits would remain. In summary, these results show extreme potential for time and cost savings through ESCM system deployment.
4.3 Customer Satisfaction and Institutional Challenges
As with many FOTs, the experiences of the participants over the course of the deployment provide valuable lessons learned for future deployments of this type. In fact, with all of the challenges faced by this test with participant turnover and dropping out, the qualitative data provided by the participants became extremely important. Participants dropped out for a variety of reasons documented throughout this final report. The major factor was the events of September 11, 2001 combined with the overall slowdown in the economy. This led companies to pull back from their voluntary commitments to tests of this type to focus on their bottom lines.
As the following section documents, participants did not necessarily drop out because of system performance, but rather because they could no longer afford to operate two duplicate systems. Staffing constraints and profitability became their ultimate priority, as should be expected of for-profit organizations. Unfortunately, this limited the quantity of quantitative data, however, a lot was learned by discussing with the participants what they thought of the system and having them explain why they dropped out.
The following section summarizes the level of satisfaction experienced by the participants, and documents the challenges faced over the course of the test by the participants and the deployers.
4.3.1 Customer Satisfaction
Customer satisfaction was measured qualitatively via interviews. The interviews were conducted throughout the course of the operational test and focused on collecting perceptions on training, functionality offered by the system, ease of system use, and Internet accessibility in tracking shipment information.
Heavy users reported continued interest and commitment to the system throughout the FOT. These participants had operations staff that found the system easy to use and they remained committed in many cases in hopes that the system would be expanded and further deployed. They reported that once the ESCM became the primary system for their and their supply chain partners operations the true benefits could be measured. In addition, other less heavy users acknowledged that they expected the benefits would more easily manifest themselves with broader deployment of the system.
The ESCM system successfully duplicated existing systems, providing all the functionality required. The participants were very satisfied with the ability of the ESCM to duplicate all necessary business functions and in some instances reported finding the system easier to use than their traditional processes.
Training activities were “very thorough”; the system was simple and easy to use. Many reported the ESCM system would be easier for new employees to learn than the existing system. Most participants found that not much training was required; those that did were provided the necessary level of detail and found SecurCom staff easy to work with and responsive.
Access to shipment status was a major benefit; it could be integrated with carrier websites to provide customers with additional shipment tracking capabilities. Given the relatively small number of shipments that traversed the complete system from shipper to receiver, the support documented for this system capability indicates the overall importance of shipment tracking to the transportation industry.
Very few participants reported technical problems; when they did, it almost always consisted of Internet access issues. Some companies did not have Internet access prior to the test, were using dial-up modems, or were unable to install the ESCM into a secure environment, which is where their Internet access was provided. The participants were satisfied with the efforts put forth by SecurCom to provide them with the necessary connections.
Technical support was timely and effective; there was rarely a need to ask for help as the system was dependable and simple to use. The dependability of the system, combined with the professional support provided is likely what motivated many of the participants to remain involved for most or all of the operational test.
The system provides a more effective way to exchange information among supply chain partners; it provides a “one-stop shop” for shipment information. Many of the participants relied on traditional communications tools (telephone, fax, and e-mail) to access shipment status. Under those conditions there were times that it was unclear exactly what the status was. The ESCM provided the ability to provide real time information on who had possession of the shipment at any given time.
The system provides improved security because there is limited access and documentation of all individuals that had access to the load. This characteristic addresses the primary reason the FAA sponsored this operational test. The existing process relies heavily on a paper trail, which is labor intensive and cumbersome. The ESCM automatically accomplishes this. In addition, driver validation is especially useful for any clerks that are not familiar with regular drivers by face.
The ESCM system is a faster system to use than existing manual procedures and was reported to take less time than some in-house automated systems due to screen layout and format. Therefore, for some participants the ESCM did more than duplicate the existing processes – it improved upon them.
Drivers were very interested in and supportive of reducing pickup/drop off delays. For a driver, the time spent at a terminal is unproductive time. This is especially true if he/she is simply waiting for paperwork. The ability to walk in, log in to the system and have all the paperwork completed electronically was seen as a major strength of the system.
The ESCM system was well received by the participants. They felt it was easy to learn and use, it provided all the required functionality of the existing system, and it was believed to provide improved operational efficiency when more fully deployed. Their major frustrations focused primarily on the incompleteness of the system for two reasons. First, it did not replace their daily routine, it added to it. And second, not all of their customers (supply chain partners) were online.
4.3.2 Institutional Challenges
The institutional challenges were also addressed through qualitative interviews completed throughout the operational test. This process allowed the participants to comment on the factors they viewed as barriers to system acceptance. The following lists the challenges identified by this process:
- Duplicate systems created need for multiple data entry; in many instances this restricted the number of manifests entered into the system as participants reported this as too time-consuming given their staff resources. This situation created a bottleneck for testing the system, as it greatly restricted the number of manifests generated. In some instances, use of the system was limited to those clerks that were interested in the system. This factor was further exacerbated because for a manifest to travel completely through the entire system there had to be interested users at three or more locations.
- Staff turnover and layoffs restricted the resources available for a duplicate system. As described under the customer satisfaction findings, staffing played a major role in use of the system as well as in determining whether or not a company remained involved in the test.
- Lack of integration between the ESCM and legacy systems restricted use and acceptance of the system. Many of the participants had existing systems for processing shipments. The existing system had to be maintained as this was only a test. Therefore, the ESCM was only used when time allowed. System integration could not be provided on a test of this scale but was considered absolutely necessary if the system was to ever be fully deployed.
- Lack of participation by enough supply chain partners restricted use of system. Many participants reported that there simply were very few of their transportation partners enrolled in the system; without a larger number of participants the benefits are difficult to quantify.
- Several participants stated that they did not have enough air freight shipments to further test the system. This factor is the result of a decrease in shipments caused by current events, inappropriate participants included because they were very interested in the system, or the result of an excuse by the participant to explain their low volume of manifests.
- Slow connection time to access the Internet was frustrating; some participants felt this was an internal problem, while others simply said faster access was required. As discussed in the customer satisfaction findings, some participants had restricted Internet access for one reason or another. SecurCom ensured that all participants were in fact connected, but most participants felt high speed access was required for the system to be successful in the operating environment.
- Driver aversion to biometric fingerprinting remained a topic for discussion throughout the FOT; however, it diminished over time. As with any technology there was some apprehension with the technology. However, at specific times during the operational test, the biometric technology received serious scrutiny from the participating truck drivers.
The key institutional challenge that impacts most of the preceding specific issues is available resources. Although the ESCM was designed to replace existing processes, it did not replace any system as part of this operational test. It simply added a new set of procedures that duplicated an existing process. Given the serious nature of air cargo security, and the fact that not all supply chain partners were involved in the test, it is difficult to eliminate the existing and regulated process. However, without doing so, the test is restricted to partial use of the system as resources allows.
5. Overview of the Technical Effectiveness
of the ESCM System
The technical effectiveness evaluation was conducted to measure the performance of the ESCM system and its components. This evaluation was organized and focused on the following four areas:
- Integrated Air Cargo System Operational Performance
- ESCM Computer System Technical Performance
- Electronic Manifest Technical Performance
- Biometric Fingerprint Identification Technical Performance
In addition, a questionnaire was developed and administered to trucking companies, manufacturers and airlines to qualitatively assess the technical performance of hardware and software, and the participants’ satisfaction with the system performance and features. These findings are discussed in the Customer Satisfaction and Institutional Challenges evaluation provided in Section 4.
5.1 Integrated air cargo system OPERATIONAL PERFORMANCE
The integrated Air Cargo Information System (consisting of the electronic manifest, Smart Cards, and biometric identification technologies as shown in Section 2) was evaluated using ATA Foundation System administration reports and statistics, and Evaluation Team survey results. The ATA System administration reports and statistics provided ESCM website transactions and computer system/network performance data. These reports produced detailed logs of ESCM system performance data, which enabled the detailed assessments to evaluate the technical effectiveness of the technologies.
The ESCM website recorded predefined transactions to study the effectiveness and efficiency of the ESCM system. The ESCM system automatically recorded all transactions including the date, time, manifest access type, transaction events and measures, and user login types.
5.1.1 Positive Association of Driver with Electronic Manifest
Over the past year, from April 11, 2001 to May 2, 2002, the ESCM system server recorded 374 transactions. Of all the transactions, creating and releasing a manifest were the most commonly recorded types (129 and 126 occurrences, respectively). Figure 5-1 shows the frequency of ESCM transactions. Of the 129 manifests created, 126 corresponded with releases to the respective trucker and receiver (airline). Three manifests were created, but had not yet been released. There were 23 shipments that were picked up and recorded by the ESCM system. Of these 23 shipments, the ESCM system recorded eight of these as being delivered to the receiver (airline).
Although the ESCM system server data only had eight shipments during the defined time period that represented the complete electronic manifest cycle (create to release to pick up to delivery), this is not surprising considering the deployment parameters. There were a limited number of complete supply chains, and given the limited staff resources, not all shipments were entered into the system.
For example, manufacturers could be entering manifests that were subsequently being picked up by nonparticipating motor carriers, or the motor carrier may deliver the shipment to a nonparticipating airline. What the data does show, however, is that the ESCM system successfully recorded and tracked electronic manifests, the movement of cargo along the supply chain, and the overall transfer of responsibility from a given manufacturer to motor carrier to final recipient (airline).
Figure 5-1. Frequency of ESCM Transactions
5.1.2 Total Manifest/Cargo Cycle Time
The eight manifests/cargo shipments that completed the entire electronic manifest cycle were examined to measure the total manifest/cargo cycle time. The manifest and cargo cycle time is the time elapsing from manifest creation to physical delivery of the cargo to the final recipient (airline). The time begins when a manufacturing clerk completes filling out the ESCM web-based manifest form. Once the truck driver makes the delivery, the driver logs into the ESCM system and verifies that the cargo has been delivered. The cycle is complete and the ESCM system server records the date and time of delivery.
To compute the total time elapsing between manifest creation and physical delivery of the cargo to the final recipient, the date and time the manifest was created was subtracted from the recorded date/time for shipment delivery to the receiver (airline). Table 5-1 presents the date/time of creation and delivery and elapsed times for the eight manifests.
Table 5-1. Total Manifest/Cargo Cycle Time
Manifest |
Create |
Delivered |
Elapsed Time (hours:min:sec) |
77 |
5/31/2001 9:32 |
5/31/2001 9:59 |
00:27:00 |
84 |
7/2/2001 11:42 |
7/3/2001 10:09 |
22:26:41 |
85 |
7/2/2001 13:21 |
7/3/2001 10:09 |
20:47:28 |
86 |
7/2/2001 13:37 |
7/3/2001 10:07 |
20:29:57 |
89 |
7/18/2001 9:38 |
2/5/2002 11:34 |
7 months, 20 days, 1:56:01 |
143 |
2/5/2002 17:19 |
2/5/2002 17:21 |
0:01:44 |
144 |
2/6/2002 10:42 |
2/14/2002 14:46 |
8 days, 4:04:06 |
145 |
2/6/2002 11:42 |
2/6/2002 11:49 |
0:07:47 |
Excluding Manifests # 89 and 144, the average elapsed time was 10 hours, 43 minutes, and 26 seconds. However, elapsed times appear to fall into three groups: short (less than 30 minutes); medium (elapsed times of about one day); and long (elapsed times taking numerous days, even months). Those manifests in the less than 30 minutes group averaged just over 12 minutes with the shortest elapsed time being one minute and 44 seconds (manifest 143). For the manifests lasting about one day the average time was about 21 hours and 15 minutes. Finally, the manifest elapsed times that were long appear to be shipments that were either delayed or the ESCM transactions were not entered at the actual time of the event.
Irrespective of the underlying cause resulting in the elapsed times, the ESCM system did function correctly by capturing the transactions upon entry by the user.
5.1.3 Physical Cargo Cycle Time
The eight manifests/cargo shipments that completed the entire electronic manifest cycle were also examined to assess the physical cargo cycle time. The physical cargo cycle time is the time elapsing from pickup of the cargo (at the manufacturing site) by the trucker to physical delivery of the cargo to the final recipient. The time begins after the ESCM system verifies a truck driver as being in the system and registers the individual as the assigned truck driver. Once the truck driver makes the delivery, a truck driver logs into the ESCM system and verifies that the cargo has been delivered. The cycle is complete and the ESCM server records the date and time of delivery.
The physical cargo cycle time is calculated using the difference in date and time between when the trucker picked up the cargo at the manufacturing site and physically delivers the cargo to the final recipient. Eight entries were available. For each manifest, the date/time of pick up and delivery and elapsed times are shown in Table 5-2.
Table 5-2. Physical Cargo Cycle Times
Manifest ID # |
Picked Up Date/Time |
Delivered Date/Time |
Elapsed Time (hours:min:sec) |
77 |
5/31/2001 9:58 |
5/31/2001 9:59 |
0:00:44 |
84 |
7/3/2001 10:08 |
7/3/2001 10:09 |
0:01:10 |
85 |
7/3/2001 10:08 |
7/3/2001 10:09 |
0:00:27 |
86 |
7/2/2001 13:40 |
7/3/2001 10:07 |
20:26:48 |
89 |
7/18/2001 9:40 |
2/5/2002 11:34 |
7 months, 20 days, 1:54:01 |
143 |
2/5/2002 17:20 |
2/5/2002 17:21 |
0:01:02 |
144 |
2/14/2002 14:16 |
2/14/2002 14:46 |
0:29:43 |
145 |
2/6/2002 11:47 |
2/6/2002 11:49 |
0:02:01 |
Average Time = 3:00:16[4] |
|||
The elapsed times varied in length from less than 1 minute to over 7 months and 20 days. Excluding the elapsed time for manifest # 89, the average elapsed time was 3 hours, 16 seconds. However, five of the eight entries were recorded at approximately 2 minutes or less, suggesting that the picked up and delivered entries were completed sometime after the actual events occurred.
5.2 ESCM Computer System TECHNICAL Performance
The overall performance of the ESCM computer system was assessed using the Microsoft NT Performance Monitor, a software application that reports system performance metrics. Used by the ATA Foundation System Administrator, the NT Performance Monitor measured the performance of the system network computers. Each of the computer’s processes and components has an associated set of counters that provide information about device usage, queue lengths, delays, and information used to measure throughput and internal congestion. Performance Monitor provides charting, alerting, and reporting capabilities that reflect both current and ongoing activity – allowing users to open, browse, and chart log files of current and past activity.
The computer system performance was assessed in three areas: system reliability, system availability, and errors, as described in the following sections.
5.2.1 System Reliability
To assess overall system reliability, two metrics were investigated: system up time and unscheduled system downtime. Using the system up time metric, which measures the total time the computer is operational, the computer was in continuous operation for 182 days. There were no occurrences of unscheduled downtime over 6 months of testing. The most recent scheduled downtime was a computer reboot needed to perform planned maintenance (i.e., service pack installation) on the computer. Consequently, the ESCM system server appears to have functioned reliably for the past 6 months.
5.2.2 System Availability
The ESCM computer resources were also examined to assess the availability of computer resources. Five NT Performance Monitor metrics were investigated:
- Processor Availability
- Interrupts per Second
- Bytes Transferred per Second
- Disk Space Available
- Pages per Second
Following is a description of each performance metric:
Processor Availability indicates the percentage of elapsed time that the processor is busy working on a task (i.e., non-idle thread). The NT Performance Monitor for the ESCM system server reported the average percentage of processor utilization time was 2.6 percent. This value indicates that the server processor was working 2.6 percent of the time and was readily available to perform ESCM system functions.
Interrupts per Second measures how busy the computer processor is in receiving and servicing hardware interrupts. Normal thread execution is suspended during interrupts, so this value is an indirect indicator of system-wide activity of devices, such as the system clock, mouse, disk drivers, data communication lines, network interface cards, and other peripheral devices. These devices normally interrupt the processor when they have completed a task or require attention. The number of interrupts per second for the ESCM system server ranged from a low of 65.99 to a high of 177.24. The average interrupts per second was 89.17, which is not particularly high, and indicates a relatively modest level of hardware activity.
Bytes Transferred per Second provides an overall indication of how busy the server is sending and receiving data across the network. The number of bytes transferred per second ranged from a low of 0 (zero) to a high of 590.99. The average number of bytes transferred per second was relatively low at 19.27 bytes per second.
Disk Space Available measures how much of the total disk space remains. Having little or no disk space can affect the server’s speed in processing tasks and indicates the need as to when to make (or acquire) more disk space. The average percentage of free space available was calculated and found to be 35.1 percent. While it appears that an adequate amount of disk space is available, however, depending on the total disk capacity and rate of disk space consumption, more disk space may need to be added to maintain optimum server performance.
Pages per Second measures the number of pages read from or written to disk to resolve hard page faults. Hard page faults occur when a process requires information that