innovative methods to collect road statistics

Innovative data collection methods for road freight transport statistics

EUROSTAT, Oct 11, 2012

Current situation

Limited relevance of data for Austria’s traffic & transport planners due to sample errors on county or smaller traffic cell levels

Insufficient precision of reported tonne-km as a result of automated imputation of the distance travelled between origin and

destination of a journey using distance matrices, ignoring potential detours in between.

Possible over-estimation of empty trips assuming that distances between places of unloading and subsequent places of loading

are empty trips

Assumed under reporting of both empty & loaded trips respondents assumed to minimize efforts, and report a vehicle to be out-of-order during

the sample week.

Limited accuracy of cargo types Respondents have limited information regarding cargo moved („mixed cargo“) Assignment of goods to NST/R classification by respondents leads to incoherent results

Quality concerns regarding road transport data

Consortium

Gebrüder Weiss GmbH

Paradigma Unternehmensberatung GmbH

Wirtschaftsuniversität Wien – Inst. f. Transportwirtschaft und Logistik

Petschl Transporte Österreich GmbH & Co KG

Austrian Institute of Technology – Department Mobility

Process knowledge and experience of transport companies

Technology, data management und electronic data exchange

Methods and legal environment of road freight statistics in Europe

Goals and Results of the Project

Project Goals Project deliverables

Further reduction of the respondents efforts through automation

Return to a larger sample to meet national requirements

Increase of data quality and actuality

Reduction of required ressources for preparation and processing of the data

Prototypic and fully functional implementation of the connection between data (from companies) to the XML-Interface

Test the applicability of automatic data collection technologies and algorithms to obtain precise measurements.

Legal, economic and methodical evaluation of the results with respect to the road freight transport statistic

Research Objectives

Prove the feasibility of using data available from transport companies building a working prototype, using IT data from transportation companies as a source develop a sufficiently generic standard interface (fleet information, consignment & location

data)

Assess the organizational impact on the respondents obtain empirical information on the benefits as well as potential issues

Use information about goods from transport booking data, to infer cargo types (NST/R) Train a Bayes algorithm, part of the KNIME data mining software to classify goods using

free text

Use GPS location to measure distances travelled and to infer load/unload events. Obtain route information from GPS readings – infer events and compare with order

information

Obtain experience with the technical and economic challenges to implement the standard interface industry software as well as individually developed software

Road Transport Statistics in Austria

EU Regulation 70/2012 provides a general legal and methodological framework for the different national surveys (territoriality principle)

Stratified quarterly sample comprises 6,500 vehicles per quarter out of a total of about 72,000 registered. Meets quality criteria described in the regulation Original sample size of 26000 vehicle weeks per year significantly reduced (since 2006)

Local units, operating vehicles and drawn for the sample use paper based questionnaires or an electronic questionnaire (11% in 2009)

Efforts to complete the questionnaire have been reduced. 27.3 minutes on average to complete a questionnaire 150 work weeks per annum for the Austrian economy …

Main task for completing the questionnaires consists of collecting and preparing the information within their companies.

Target Solution Architecture

Data interfaces are in the public domain and provided to the Software- and System developers

Different information entities are consolidated according to the data model specifications and business logic

Completion of missing data and corrections are performed by the respondent

Based on the transfer-format the specific required structure for the respective country questionaire is generated

Information elements collected

4 XML-based interface specifications are provided

FleetMasterData

Data on lorries and trailers (capacity, axles, odometer, age, license, etc.)

FleetStatusData

Information on specific lorries at certain times (driven distance, fuel usage, etc.)

ConsignmentData

order related data containing information on goods, packaging, origin and destination …

PositionData

GPS readings, country/ZIP codes, activity (loading, border crossing)

Data Collection Service

Protoytped process deployed to move data from IT systems to eQuest

Process activities

Questionnaires as XML files are generated by the eQuest system run by Statistics Austria.

These Questionnaires contain the selection criterias for the data export.

The data export extracts the information out of the ERP/TMS-systems and saves them as XML-files (4 predefined formats).

These files are uploaded to the „SGVS-Konsole“ web-application.

The respondent can now revise the data. The web-application generates a

„completed questionnaire“ and uploads this to the eQuest system.

In the eQuest system the report is finished.

Questionnairewith data

XML-Question-naire

Selection criteria

Data export

ExportedXML data

ERP/TSM

DatabaseAccess

API

Web application „SGVS Console“

eQuest Web-application

Web-Application

Validation Rules (Excerpt)

Every lorry or articulated vehicle mentioned in the NSI’s questionnaire must have an entry in the company’s fleet management system.

Odometer readings at the beginning and the end of the reporting period must be available, where the latter has to be greater or equal than the former. If multiple odometer readings are available over time, the sequence of readings must be non-decreasing.

Every shipment must have been allocated to one or more sections of a journey.

If events and activities such as load, unload are reported or inferred from the position data (see below), corresponding sections of journey’s have to be reported as well.

Reported sections and journeys of a given vehicle must not overlap

Automatic classification of goods

Official statistics

Respondents

InnoRFDat-X

In compliance with national regulations all transported goods are classified by the NST/R

Hauliers and forwarding agents often use free text in their operative data. (i.e. 10 ldm bathtubs, granite, etc.)

Assignment to NST/R classification is conducted manually.

Development of a model for automatic classification according to NST/R categories for all free texts.

Experiences Assignment of goods to NST/R classification through

respondents leads to incoherent results (DE) Hauliers provide free texts, classification is done by NSO (NL)

Autoteile , Coils Stahl und Blech 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Bleche 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Bleche max . 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Bleche Überbreite 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Coils 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)kompl . Profile 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Leitschienen 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)nahtlose Stahlrohre 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Profile 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Profile 12,2 m 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Profile lt . Beilage 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Rohre 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Sonderfahrt , Profile 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stabstahl 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahl 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahl ( S320GD+Z275MB) , 2 Coil 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahl Rohre 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahl Vg . 1/7 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahlbleche 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)

Automatic classification - experiences

Expect improvements when trained using respondents texts

Correct categorization

Pervasive use of product codes in one case

Transport provider has insufficient information („45 parcels“)

Insufficiently discriminating

Find a balance between „rote learning“ and the capability to correctly classify new descriptions

Imprecisions

Based on an algorithm trained on classification texts and applied to a sample of 1000 cargo descriptions from transport orders

Route & Event Detection

Protoytped process deployed to generate trips using GPS data

Objective

Position data file containing the geographic details of every tour where

a tour starts with the loading of an empty lorry

ends with the unloading of the last cargo

Two-Step Heuristics applied

First to detect all stops in the data file

Second to eliminate non-loading/unloading stops

Detect Stops

Speed gradient

Geographic change

Spatial distance

Classify StopsDistance to point of loading

Mandatory rest period

Distance to motorway services, etc.

•Stop-Position•Stop-Duration

•Timestamp•GPS-Coordinates

Loading stopResting stop

Loading point order info:(ZIP-Code)

Event detection performance

Heuristics gave 100% Recall but only 28% Precision

Heuristics + Order details (ZIP-Codes) gave 100% Recall and 85% Precision

Lessons learned

The collection and use of operational available at transport companies to produce road traffic statistics is feasible Standardized interfaces can be cost-effectively implemented

Research implications Use patterns in position data to discriminate between rest and

load/unload stops (independent of consignment information) Respondent specific training sets is expected to increase the precision

of goods classification Generation of data from ERP/TMS-system would allow for continuous

reporting of transport activities

Recommended changes to the legal framework Enable NSI‘s to utilize available IT assets for the collection of raw data Adapt statistical sampling to changes in the market place (freight

forwarders) Uniform collection and production methodologies across EU member

states

TMS Software penetration in Austria

Page 18

32

15

14

40

12

14

Sauer Bespoke SWHypersoft, -sped Keine TMSHelpten COSwareC-Logistic Transporeon (?)

Methodology

Contacts to 55 larger companies, o.w. 29 responded and provided information

Significant proportion has outsourced transport: Strabag, Rail Cargo, Lenzing AG, Magna Steyr, Borealis

Respondents represented 7,3 % of all trucks registered in Austria (SGVS 2007, Q4)

5.230 / 72.000 = 7,3 %

The way forward

Solution is not restricted to the Austria territory. The choice of application architecture and scope has been guided by

the vision of a European rather than only a national application.

A Europe wide adoption of the approach is expected to increase quality of the data collected by the member states. applicability of transport statistics all over Europe will be enhanced comparability of transport statistics across countries will be improved.

Piloted process has shown the potential to substantially reduce the administrative burden on reporting companies

Expectation to further raise the efficiency of statistical production processes leading to cost reductions and time savings.

Summary

Model deployed is capable to automatically classify cargo Sufficient precision achieved after training with respondent

specific datasets Less effort required from the respondents Quality improvements as a consequence of consistent

classification

Implementation aspects Improve „training phase“ using both descriptions and pre-

classifications of a sample of respondents Encourage consigners to provide more descriptive data of their

goods Provide functions to manually override misclassifications

innovative methods to collect road statistics

Business