Efficiency, effectiveness, and impacts assessment in the rail transport sector: a state‐of‐the‐art critical analysis of current research

• Published date: 01 January 2019
• Author: Cinzia Daraio,Marco Diana,Martina Gregori,Giorgio Matteucci,Giuseppe Catalano
• DOI: http://doi.org/10.1111/itor.12551
• Date: 01 January 2019

Intl. Trans. in Op. Res. 26 (2019) 5–40

DOI: 10.1111/itor.12551

INTERNATIONAL

TRANSACTIONS

IN OPERATIONAL

RESEARCH

Efficiency, effectiveness, and impacts assessment in the rail

transport sector: a state-of-the-art critical analysis

of current research

Giuseppe Catalanoa, Cinzia Daraioa, Marco Dianab,

Martina Gregoriaand Giorgio Matteuccia

aDIAG, Sapienza University of Rome, Rome 00185, Italy

bDIATI, Politecnico di Torino, Turin, Italy

E-mail: giuseppe.catalano@uniroma1.it [Catalano]; cinzia.daraio@uniroma1.it[Daraio];

marco.diana@polito.it[Diana]; martina.gregori@uniroma1.it [Gregori]; matteucci@dis.uniroma1.it[Matteucci]

Received 10 January 2018; received in revised form 21 March2018; accepted 24 March 2018

Abstract

This paper aims at being a comprehensivereference for stakeholders, policy makers, and scholars interested in

analyzing the problem of efficiency, effectiveness, and impacts of rail transport systems in a sound empirical

way, paying specific attention to passenger transport services. The paper combines different analyticalframe-

works (engineering, economics, impacts), systematic review techniques, and advanced mappings. Framing

economic efficiency studies into a transport planning perspective permits to move from efficiency to effec-

tiveness issues. In addition, including impacts offers a critical discussion of the existing empirical studies,

relating them to the main methodological approaches used. This analysis can be useful for those interested

in developing new techniques for the evaluation of this sector. The critical analysis developed in this paper

provides a catalogof inputs, outputs, external factors, possible impacts to account for, data, and approaches,

which allows us to identify areas in which new methodologicaldevelopments, new approaches, are needed to

address the relevant societal challenges of the rail transport sector.

Keywords:efficiency; effectiveness; impacts; frontier analysis; DEA; rail transport; quantitative methods

1. Introduction

Railway systems are an important component of the social and economic system of many nations

around the world. From a macroeconomic viewpoint, it has been estimated that the generated

added value of this sector at the European level was €66 billion in 2012, which rises to €142 billion

considering indirect effects, thus representing 1.1% of the European Union’s economy (Molemaker

and Pauer, 2014).

Although such figures seem not too impressive in relative terms, railways are often seen as a key

component in the implementation of more effective transport policies in many countries. They can

C

2018 The Authors.

International Transactionsin Operational Research C

2018 International Federation ofOperational Research Societies

Published by John Wiley & Sons Ltd, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main St, Malden, MA02148,

USA.

6G. Catalano et al. / Intl. Trans. in Op. Res.26 (2019) 5–40

in fact represent a valid competitor to road transport, since they present attractive features such

as their higher transport capacity (both for passengers and freight), higher safety standards, or the

possibility of easily implementing electric traction that can help in limiting environmental impacts.

For instance, CO2emissions per traffic unit in rail transport are less than half than those of road

transport for passengers (40 vs. 115 grams per passenger-km) and less than one third for freight

(20 vs. 75 grams per ton-km) (Molemaker and Pauer, 2014). In congested urban areas, railways

can transport a number of passengers per hour larger by an order of magnitude compared to road

transit lines, and even muchmore if one considers private cars, with comparable space consumption

for the infrastructure. Rail transport is also safer than road transport, averaging 0.3 deaths per

billion persons kilometers against seven deaths per billion persons kilometers for cars (Koornstra

et al., 2003).

Considering those factors, railways systems are more or less heavily subsidized in many countries

to promote their use versus private cars. However, related costs for public finances are generally

high: for example,at the European level, subsidies in 2012 were about €35 billion out of €112 billion

in total revenues (Steer Davies Gleave, 2015). Regulatory reforms have therefore been promoted in

the last decades to increase the efficiency of railways operations,such as both vertical and horizontal

separation and the opening of a former monopolistic market to competition.

The extent to which such reformshave been successful is much debated,since one inherent problem

is related to the difficulty of actually evaluating the efficiency of railways operations (Makovsek

et al., 2015). Many railway companies are multi-product firms (passengers vs. freight, intercity vs.

suburban services, etc.) wherethe mix of outputs can dramatically change from one unit to another,

thus making comparisons difficult (Oum et al., 1999). In addition, much like in other forms of

public transport, subsidies are given to achieve indirect benefits through the implementation of the

service, such as social welfare or the limitation of environmental impacts. In such case, it would be

more appropriate to consider the extent to which such transport policy goals have been achieved

through an effectivenessanalysis, rather than limiting oneself to check if either technical or allocative

efficiency has been reached through the measurement of outputs and inputs, like for an ordinary

industrial process in the private sector (Diana and Daraio, 2014).

In a related study dealing with urban public transport, Daraio et al. (2016) assessed the potential

of filling the above research gap, namely the need to shift from an efficiency to an effectiveness

analysis, by running an efficiency analysis where the measures of inputs and especially of outputs

are not strictly related to the service production process, but are rather measuring the attainment

of some policy goal. As hypothetical example, rather than considering production factors such as

labor as input and traffic units such as vehicleࢩkilometersor seatsࢩkilometers as output, one might

consider some indicators related to the emissions of pollutants or to accessibility improvements to

assess how effectivelyrelated policy targets are reached by investingpublic money on transit services.

In more general terms, efficiency analyses could be turned into a more powerful decision support

tool for different stakeholders in the transport sector if they considered some of the wealth of

indicators that are normally employedin transport planning studies to monitor the implementation

of policy measures, such as in Sustainable Urban Mobility Plans (Wefering et al., 2014).

Building upon that knowledge base, the starting objective of the present paper is to perform

a study similar to Daraio et al. (2016) in the railway sector, with additional analyses from the

insights gained so far. Therefore we focused mainly on passenger railway services serving both

suburban and intercity trips. While we do not consider those researches exclusively focused on

C

2018 The Authors.

International Transactionsin Operational Research C

2018 International Federation ofOperational Research Societies

G. Catalano et al. / Intl. Trans. in Op. Res.26 (2019) 5–40 7

freight services, for which the evaluation framework would be completely different, the goal is to

assess the extent to which the existing efficiency analyses related to such services are considering

input and output measures that represent broader policy goals by encompassing effectiveness and

therefore be considered as monitoring tools for transport plans.

Considering the increasing importance of impacts in the evaluation of the performances of any

transport system, and of railways in particular, we extend the analysis to include impacts. While

a systematic review of transport planning studies dealing with the impacts of railways is outside

the scope of this research, it is deemed important to check the extent to which existing economic

analyses in this sector are enlarging their perspective beyond the mere consideration of efficiency

issues that are related to the service production process.

To achieve this objective, we systematically review, in an original way, the scientific literature

dealing with efficiency analyses in the railway sector being published up to 2016, therefore updating

a previous review in this sector by Oum et al. (1999).

There are several reviews on efficiency and operational research methods applied to the public

sector (see, e.g., Rais and Viana 2011; Narb´

on-Perpi˜

n´

a and De Witte, 2017a, 2017b). Given the

increasing importance of Operations Researchmethods such as Data Envelopment Analysis (DEA)

to drive such studies (Lampe and Hilgers, 2015; Emrouznejad and Yang, 2017; Mariz et al., 2018),

and the increasing use of DEA in the transport sector (for a recent review, see Cavaignac and Petiot,

2017), we believe that this activity would be relevant to readers of this journal.

The paper unfolds as follows. The next section illustrates the methodology followed to review

the existing studies. Section 3 describes the main impacts, which could be relevant to account

for. Section 4 analyzes the keywords of the reviewed papers using the software tool VOSviewer. In

Section 5 a description of the main inputs, outputs,external variables adopted in the different papers

is reported. Section 6 reviews the main techniques applied in the surveyed papers, while Section 7

analyzes their evolution over time together with the data used. Section 8 concludes the paper.

2. Methodology of the review

The objectives of this paper are framing economic efficiency studies into a transport planning

perspective including impacts. By doing this, we aim at offering a critical discussion of the exist-

ing empirical studies, relating them to the main methodological approaches which have been used

so far. These objectives are challenging. In order to achieve them we developed an original re-

view approach. It combines integrationof different analytical frameworks (engineering, economics,

impacts), systematic review techniques, and advanced mappings. It offers a systematic and compre-

hensive classification (catalog)of inputs, outputs external factors used together with methodological

insights for developing new methods for facing the complexity of the field (rail transport efficiency)

within its extensive environment and face the societal challenges.

According to Petticrew and Roberts (2006), a systematic literature review “strives to comprehen-

sively identify, appraise, and synthesize all the relevant studies on a given topic.” The main steps of

the survey implemented are the following:

1. Identification of the objectives of the survey (in our case framing economic efficiency studies

into a transport planning perspective, deepened to include impacts).

C

2018 The Authors.

International Transactionsin Operational Research C

2018 International Federation of OperationalResearch Societies