CHAPTER 7.  CVISN CUSTOMER SATISFACTION

 

 

            Improved customer satisfaction is key to the success of CVISN.  For CVISN to lead to more widespread deployment of the demonstrated technologies and operating procedures, its customers must value the incremental benefits they experience more highly than the incremental costs they bear.

 

            Customers or users of CVISN technologies include independent and company drivers; motor carrier operators; state transportation and CVO administrators; law enforcement, highway, and public safety personnel; and the businesses and industries that engage the services of motor carriers.

 

            To measure customer satisfaction with CVISN, several surveys and other measures were planned and carried out.  These included a national motor carrier survey, a driver survey, and surveys and focus groups involving state inspectors and law enforcement personnel.  Originally, a separate motor carrier survey was planned, to quantify the benefits of electronic credentialing for motor carriers.  However, at the time this study was scheduled, there were not enough carriers with experience in electronic credentialing to constitute a valid study population.

 

            Table 7-1 shows the customer groups who were surveyed to determine their experiences in using CVISN technologies and their satisfaction with those technologies.  Shippers/receivers and the general public are also recognized as stakeholders in CVO.  Benefits to each are discussed indirectly in Chapter 8 on benefit/cost analysis, but direct measures of the satisfaction of these customer groups were beyond the scope of this evaluation.

 

Table 7-1.    General Topics Covered in Surveys and Other Evaluations of Customer Satisfaction

 

Customer Group	Survey Topics Relating to
	Electronic Credentialing	Roadside Inspections/Enforcement
Motor Carriers	· Experience with credentialing · Current credentialing procedures · Awareness and use of electronic credentialing · Opinions about electronic credentialing · Likelihood of using electronic credentialing	· Experience with roadside inspections · Current inspection procedures · Awareness of electronic screening · Opinions about electronic screening
Drivers	· Likelihood of owner-operators to enroll in electronic credentialing	· Opinions about roadside enforcement · Likelihood of owner-operators to enroll in electronic credentialing
State CVO Administrators	· Institutional issues and benefits	· Institutional issues and benefits
State CVO Inspectors		· Inspection systems in use · Satisfaction with equipment · Perceived benefits

 

            Section 7.1 presents a summary of the objectives and methods used.  Section 7.2 summarizes the findings and details the results of the various customer satisfaction measures.  Appendices C.1 through C.6 contain data tables and supporting documentation related to customer satisfaction.

 

 

7.1             Customer Satisfaction Technical Approach

 

Motor Carrier Survey Approach

 

            In planning to appraise customer satisfaction associated with CVISN initiatives, the original plans[1]included a baseline national motor carrier survey (BNMCS), intended

 

·        to collect baseline information[2]concerning the relevant behaviors, awareness, and attitudes[3]of motor carriers;

 

·        to identify the incentives and barriers to more widespread deployment of CVISN‑type initiatives across the country;  and

 

·        to aid potentially in designing subsequent surveys of motor carriers on other topics (such as credentials administration).

 

            We examined several existing databases that were relevant either to the substance of the enquiry or to the methods that might be used to survey motor carriers, and concluded that no existing study met the objectives of the proposed baseline survey.

 

            However, the progress of CVISN deployment has been slowed by a variety of factors, which has limited the ability to monitor, within the evaluation timeframe, the experience of a critical mass of motor carriers affected directly by the new initiatives.  As a result, the objectives and value of this survey have changed somewhat to reflect this slow deployment progress.  This survey provides a picture, as of mid-2000, of the awareness, attitudes, and experience of motor carriers concerning CVISN-type initiatives generally.  To some extent, the survey responses will be colored by on the ongoing CVISN model deployment:  it may be expected to have affected awareness positively (albeit not necessarily to a significant degree), and some respondent firms may well have directly experienced innovations that have been funded in part through the CVISN program.

 

            We designed a mail survey of carriers to be representative of the trucking industry throughout the contiguous 48 states.  The sample was a stratified random one, selected from the mid-1999 records of firms in the federal government’s MCMIS Census database.  Stratification was used both to compensate for the highly skewed distribution of firms in the industry by size, and to place emphasis on firms operating in the states where CVISN deployment was most highly advanced.  Compared to a simple random sample of all firms, the stratified sample contained much higher proportions of larger firms and ones with registered home addresses in five “CVISN focus” states:  Connecticut, Kentucky, Maryland, Oregon, and Virginia.

 

            Experience with the methodologically most closely analogous survey of carriers – one conducted in 1996 by the ATA Foundation [4]– suggested that (1) the response rate was likely to vary inversely with the size of the firm, and (2) in larger firms, obtaining authoritative answers about both credentialing and roadside inspection matters might well involve consulting different employees.  To achieve a target sample size of 150 substantially complete responses would require an issued sample of approximately 1,500 firms.

 

            In practice, only 116 responses were received from this issued sample.  In particular, the response rate from larger firms was disappointing by comparison with that achieved by the ATA Foundation survey. [5] A second sample of 500 firms with 100 or more power units was issued, and achieved a further 31 responses before books were closed at the end of 2000.  A further 11 responses from the pilot survey were also added to the database file, resulting in a final total of 158 achieved responses.

 

            A weight was computed for each sample stratum to restore correct proportionality of the achieved sample by firm size and by geography.[6]  The choice of the weighted total number of observations is arbitrary.  In the cross-tabulations provided in Appendix C.3, we chose to use a weighted base of 10,000 firms. [7] All of the data and commentary provided in this report are based on the weighted data, except (rarely) where information is identified specifically as “unweighted.”

 

            This sample design, while based on a quite small achieved sample size, allows some (limited) disaggregation by firm size and by exposure to CVISN concepts, thought to be important influencing variables for the behaviors and opinions under analysis.

 

Driver Survey Approach

 

            Among the various “customers” of interest are commercial vehicle drivers and owner‑operators.  The purpose of the data collection activity summarized here was to explore the opinions of truck drivers about recent, CVISN-related changes in roadside inspection methods, and the opinions of owner-operators about electronic credentialing.  The operators’ inputs are intended to help color, interpret, and better understand the information gathered in complementary evaluation activities.

 

            A qualitative survey was conducted with 61 truck drivers intercepted at large rest/refueling stops located adjacent to major truck routes in Connecticut and Kentucky.  These two states are ones that have implemented significant electronic credentialing initiatives, and have been the focus of other, complementary evaluation activities.  The interviewing took place at four locations, two in each of the states, in late November and early December 2000.

 

            Sample quotas were set to ensure the representation of owner-operators and of drivers employed by firms of varying sizes.  Using in-depth, semi-structured personal interviews, all of the respondents were asked about roadside safety and weight inspections.  The owner‑operators were also asked about electronic credentialing methods.

 

            Like all qualitative research, the primary objective was to identify the range of opinions on various aspects of these topics, and to form hypotheses about any apparent areas of consensus or disagreement.  Because of the small size of the sample and the method of sample recruitment, the degree to which the people interviewed are representative of any group larger than themselves cannot be determined.

 

            Gathering statistically defensible information about commercial vehicle drivers is a challenging objective, because of myriad practical difficulties in building a sampling frame, and contacting and interviewing people who, by virtue of their occupations, are on the move for large portions of their time.  It is not surprising that many of the surveys of drivers have used highly questionable sampling methods.[8]  A notable exception is the recently-published 1997 survey of commercial vehicle drivers at Midwestern rest/refueling stops, undertaken by the University of Michigan Trucking Industry Program. [9] This study made serious attempts to design a randomized sample within a specified geographical scope for intercept points, and invested significant resources and time into development work to refine the survey procedures and instrument.

 

            The resources available for this survey did not permit comparable efforts, nor a sample size sufficiently large to afford precise quantitative estimates, even were it feasible to design a random sample of drivers.  The approach adopted here was to use appropriate procedures and data from the UMTIP survey to help design a qualitative investigation of driver and owner‑operator opinions about CVISN-type innovations, primarily (1) the use of electronic vehicle identification and information in roadside inspections and (2) electronic credentialing practices.

 

            A further complicating factor in obtaining driver inputs about CVISN initiatives is that the general level of deployment to date has been such that drivers with direct, personal experience of them are likely to have very low incidence at possible intercept points.  In consequence, the interview focus was more concerned with reactions to the general CVISN concepts than to the specific extant deployments of those concepts.

 

            The survey approach was to undertake a number of semi-structured qualitative interviews with drivers at rest and refueling stops selected to provide a relatively high incidence of drivers likely to be familiar with new roadside screening methods.  The respondent selection process sought to randomize the selection of qualified respondents intercepted at those stops at the survey times.  However, because of the sample design and size, the survey does not generate quantitative estimates that are reliably projectable to any larger group than the people interviewed.

 

State Administrator Forums

 

            Methods for evaluating of the satisfaction of state CVO administrators with CVISN technologies were less formal than the measures used to gauge the satisfaction of motor carriers and commercial vehicle drivers.  Evaluation contractor staff participated in many meetings, conferences, and other forums, where the attitudes of state administrators and other CVISN stakeholders were directly solicited and discussed in detail.  Examples of recent conferences and meetings include:

 

·        Various design and planning workshops organized by The Johns Hopkins University Applied Physics Laboratory on behalf of FMCSA.

 

·        CVISN MDI Prototype and Pilot States Program Managers Meetings.  These meetings are held every three months or as needed.

 

·        Institute for Transportation Engineers (ITE) 2000 International Conference, April 2000, Irvine, California.

 

·        Great Lakes and Southeast States CVISN Mainstreaming Conference, May 11‑12, 2000, West Palm Beach, Florida.

 

·        Eastern States ITS-CVO Deployment Forum, November 14‑16, 2000, Crystal City, Virginia.

 

At each of these forums, participants were encouraged to offer opinions on successes, failures, obstacles, lessons learned, and issues to be resolved.  Information gathered from these kinds of meetings was taken into account in all phases of evaluating the CVISN MDI.

 

Roadside Enforcement Staff Surveys

 

            Attitudes and opinions of state motor carrier inspectors regarding the use of CVISN roadside enforcement technologies were addressed through focus groups and a formal survey conducted in collaboration with the evaluation of the I-95 Corridor Coalition and SAFER Data Mailbox FOTs (2000).  Over 50 inspectors from six eastern states (Connecticut, Maryland, New Jersey, New York, Pennsylvania, and Rhode Island) participated in the focus groups, and approximately 370 inspectors from these states completed formal questionnaires.   Topics included background information, system usage, satisfaction, and perceived benefits.

 

 

7.2             Findings

 

Motor Carrier Survey Overview of Findings

 

            On the basis of our investigation of the motor carrier survey data set, we formed the following conclusions:

 

·        The general awareness throughout the national trucking industry of CVISN‑type initiatives is very low:  about 4 percent for electronic credentialing methods, about a third of all firms for electronic screening, and between a quarter and a third of firms for SAFER-type innovations.  Less than a half of one percent of firms currently have any experience using electronic credentialing, and about 6 percent are using electronic screening.

 

·        The effort and time presently involved in credentialing compliance and in roadside inspections is quite considerable.  Across all firms, the reported total in‑house staff time involved in credentialing had a mean of about 73 full-time equivalent days per firm per year, with a median value of about 20 FTE days.  Examining these data on a “per powered unit[10] operated” basis and breaking them down by firm size produces estimates that would intuitively appear to be inflated.  For example, for the firms operating 10 or fewer powered units, the mean claimed credentialing time expended per unit was 34 ("29) [11]FTE days per year, with a median value of 8 FTE days.

 

·        Relatively few firms collect or analyze data about their roadside inspections.  However, the survey respondents’ estimates of the mean amount of time involved per inspection (19 minutes for size/weight checks and 45 minutes for safety checks) are quite similar to earlier estimates from the ATA Foundation survey.

 

·        For the firms that use in-house staff to handle some or all of their credentials work, the most common methods of submitting paperwork are by mail, walk-in, and fax.  About one in ten of these firms claimed to have used the Internet to file information over the preceding twelve months, but this statistic is highly variable by type of firm.

 

·        The respondents’ average levels of satisfaction with their current credentialing methods range from “weakly dissatisfied” to “weakly satisfied.”  OS/OW and HAZMAT permitting generally received the lowest satisfaction ratings, but these were also the types of credentials required by the smallest proportions of all firms.

 

·        The concept of electronic credentialing, of which most responding firms were not aware before the survey, elicited a significantly positive endorsement of the statement “With electronic credentialing, I’d expect the turnaround time to be much quicker.”  On the other hand, responses to other attitudinal statements revealed concerns about possible expansion of state regulation and charges, and the cost-effectiveness of the method for the respondent’s own firm.  Relatively favorable responses tended to be given by those firms currently devoting high levels of in-house staff time to credentialing, or expressing most dissatisfaction with current credentialing methods.  These same firms expressed a relatively high likelihood of adopting electronic methods if given the opportunity to do so, along with firms with a relatively high fleet mileage and intermediate sized firms (in terms of numbers of vehicles and drivers).

 

·        The levels of satisfaction expressed with current roadside check procedures were generally lower than for credentialing, despite the fact that (on a per firm basis) the amount of reported time involved per year was considerably less.  However, there seems to be significant agreement that the types of roadside checks made are appropriate.  The highest levels of dissatisfaction were with the criteria for deciding which vehicles to inspect, the time spent in the inspection itself, and (most markedly) the time spent waiting for inspection.  Several respondents commented also on safety concerns about truck queues for inspection backing up onto the highway.

 

·        Responses to attitudinal statements about electronic screening methods evidenced some similar negative concerns to those we found about electronic credentialing:  concerns about cost-effectiveness for the company and expansion of state regulation, for example.  Possibly because of closeness to the debate about deployment of vehicle monitoring technologies, the phrase “big brotherism” was volunteered by a number of respondents.  Nonetheless, there were significant levels of endorsement for the statements “Even without equipping our vehicles, we would probably benefit if the inspection officials had better information” and “These types of changes will make the roadside inspection system significantly more fair.”

 

·        Despite the relatively small sample size for this survey, it is apparent that examining the attitudinal responses about both electronic credentialing and electronic screening in greater detail, using multivariate clustering methods, could provide important insights into the types of firms most amenable to early adoption of these technologies.

 

Motor Carrier Survey Results

 

The Respondents' Experience of Credentialing

 

            Four out of every five trucking firms had obtained at least one permit within the preceding twelve months.  Roughly half the firms had filed for the most common paperwork:  the IRP/IFTA initial application, annual renewals, or IFTA quarterly tax.  Table 7-2 shows the incidence of credentialing experience (by permit type) for all firms, and for three subgroups expected to require permits and credentials to a greater than average extent:  the “for‑hire truckload carriers,” the firms with over 3 million fleet miles, and those requiring more than five different permit types.

 

Table 7-2.    Experience with Credentialing

 

 

Source:  Table A [Sources designated in tables and footnotes in Chapter 7 correspond to data tables in Appendix C.3].

 

            Roughly 28 percent of all firms use an outside firm to help obtain at least one form of credential.  By far the greatest use of outside firms, as a proportion of all applications, is to help obtain IRP/IFTA annual renewals.  Outside firms are used most heavily by medium-sized fleets (those with 11 to 50 powered units, or 11 to 70 drivers), and those with high annual fleet mileages (over 3 million miles per year).[12]

 

            About 59 percent of all firms use in-house resources in obtaining some or all of their credentials, and two-thirds of those provided some estimates of the staff time involved over the preceding twelve months.  We used the data from Questions 4 and 5 to compute estimates of the numbers of full-time equivalent (FTE) days expended by in-house managerial and clerical staff, and in total.[13]

 

            The mean total staff time expended, across all of the firms that gave us estimates, was approximately 73 person days involved in credentialing.  But because of the highly skewed distribution of firms by size, this mean is heavily influenced by the relatively small number of giant firms (the largest estimate for total staff time was 2.9 person years).  The median level of effort was approximately 20 FTE days.  Not surprisingly, the amount of clerical time spent increases with the size of the fleet, the annual fleet mileage, and the geographical scope of operations.  However, the mean level of “management” time expended is lower for the medium-sized firms (11 to 50 powered units, 11 to 70 drivers) than for the smaller or larger firms, perhaps reflecting the greater use of outside firms by the medium-sized group.

 

            Examining these staff time estimates on a per powered unit operated[14]basis, Table 7‑3 summarizes how this statistic varies by the size of the fleet.  Because the data weights are large and highly variable for firms with small fleets, the mean value for them has a large confidence interval.  The median value for the small firms – 8 person‑days per powered unit – intuitively seems high, which may indicate a tendency (by at least the small firms) to overstate the level of internal effort.

 

Table 7-3.  Total In-house Staff Time Expended per Year per Powered Unit (FTE Days)

 

	Mean value (± 95% confidence limits)	Median value